Click command not working on a remote stack

[trevix]

LC 9.6.8 OSX12.5.1

I am trying to implement a quick and dirty sort of "remote control" using sockets, with a Server and a Client stacks.
Both stacks have a library stack with all the communications scripts and are on two different computers on a local network.
The idea is that ,once established the communication, the server sends a screen shot of the current card (in which there are a few buttons) to the client .
The client has a mouse down script on the image control, that send the ClickLoc data to the server.

If the clickloc data corresponds to the position of a button control (the 2 stacks are of the same size) I should be able to run a mouse down/up from remote.
Everything works fine and fast on local and I can see that the position of the click on the server correspond to the mousedown position done on the client.

What is not working is that the button on the server is not running its mouse down/mouseUp handler.
If, on the server msg box, I run a "click at 275,229" or a send "Click at 275,229" to the card, the button works ok.
But this doesn't apply to the server script, which is something like this:

Code: Select all

click at TheMessage["DATA"] --location of the mouseDown sent by the client
set the loc of graphic "FakePointer" to TheMessage["DATA"]

Please note:
- in the server the FakePointer moves correctly, as to prove that the current card is correct.
- Using a send "Click at tLoc" to card xxx in the server script does not work.
- The idea is not to use on the server script the control names, in order to send them mouseDown command. Just their position, like any desktop remote control software would do.
Any idea?

Also, I am curious to know (which is related) why this works from the msg box

Code: Select all

send "Click at 426,255" to this stack

while this doesn't

Code: Select all

send "Click at 426,255" to this stack in 0 seconds

Regards
Trevix

[dunbarx]

Trevix.

It may well be a syntax issue. For example, the reason your one-liner does not work is because it needs to look something like this:

Code: Select all

 do "send" && quote & "Click at 426,255" & quote && "to this card"

But I as well could not append a time to the line.

Craig

[trevix]

I did a small test stack, to understand the behaviour of "click at".
Not sure if I can make sense of a few things...

Anyway, none of the solutions seen in the test stack seem to be working on my Server stack, that has a code similar to this:

Code: Select all

on chatServerMessageReceived_new p_Socket, pMsg --this script is in the library script of the server, opened with a "start using..."
--- here I convert pMsg back to array
--- I then have pMsg["ACTIONTYPE"] = "MOUSEDOWN" 
-- and pMsg["DATA"] = "426,255" or whatever loc has been sent by the client

click at pMsg["DATA"] --position sent by client
set the loc of graphic "FakePointer" to pMsg["DATA"]
write "CONF" & kEndOfFile  to socket p_Socket   
---

The "FakePointer" moves correctly to the sent position, but the button does not do a mouseUp or mouse down

[Klaus]

Hi trevix,

maybe something like this will do?
Works at least in your test stack on the desktop:

Code: Select all

...
put "426,255" into tLoc  
put controlatloc(tLoc) into tObject
## answer tObject -> control 1
send "mousedown" to tObject
...

Best

Klaus

[trevix]

Great: thanks Klaus, it works.You always have a solution...
I was hoping to streamline the script, to avoid lag, but this will do.

Any idea on how to treat the LC Answer dialog on the remote server, without rebuilding all the dialogs of my standalone?
If a dialog has "OK" and "Cancel", I don't think I can send them a MouseUp and, probably, the Server script that receives the info from the socket connection will not run if the dialog is open.

Regards
Trevix

[Klaus]

Hi Trevix,

great that it works!

Any idea on how to treat the LC Answer dialog on the remote server, without rebuilding all the dialogs of my standalone?

Hm, out of my head:

Code: Select all

...
put "Answer Dialog" into tStack
## Do whatever you need to do here
## and here...
send "close stack tStack" to stack "your serverstack here" in 1 secs
...

Will close that namely stack one second later and will NOT throw an error if the Dialog is not open!
You get the picture.


Best

Klaus

[trevix]

I not sure to understand.
I must be able, from the client, do dismiss the answer dialog that has been opened on the server. That is, pressing the "OK" or "Cancel" buttons of the dialog on the server, just having the position of the twos on the client (clickLoc).
I guess that there is a way to send a mouse down to the answer dialog.
The problem is that no socket code will run while the dialog is open.

[Klaus]

Hi Trevix,

I not sure to understand.

no problem, my answer will not help you, i just didn't understand what and WHEN you are trying to do something.

I must be able, from the client, do dismiss the answer dialog that has been opened on the server. That is, pressing the "OK" or "Cancel" buttons of the dialog on the server, just having the position of the twos on the client (clickLoc).
I guess that there is a way to send a mouse down to the answer dialog.
The problem is that no socket code will run while the dialog is open.

Hm, can you add a little script to the server stack?
Like:

Code: Select all

command dismissdialog tLoc
   wait 1 secs with messages
   if "Answer Dialog" is in the openstacks() then
     set the defaultstack to "Answer Dialog"
     put controlatloc(tLoc) into tObject
     send "mouseup" to tObject
  end if
end dismissdialog

Then:

Code: Select all

...
put "123,456" into tLoc
send "dismissdialog tLoc" to stack "your serverstack"
## Do whatever may open that dialog here...
## You may need to adjust the WAIT duration
...

Know what I mean?
I'm just guessing since I have no idea about the exact functionality of your stack(s).


Best

Klaus

[trevix]

Think of a small version of AnyDesk or TeamViewer.
I establich a socket channel beteen the client and the server. Then I must be able to do things on the server, clicking from the client.
Example:

• communication established: the server send a screen shot to the client

• I click on a image control on the client (containing the screenshot) where it shows a button

• the client send using sockets the click position to the server, telling it to press at the loc

• the script of the button on the server runs

• the server send back to the client a screenshot of the updated screen

I don't think all of this can happens if a answer dialog is open on the server.
The line on the server

Code: Select all

read from socket p_Socket until kEndOfFile with message "chatServerMessageReceived_new"

will not run if the answer dialog is still visible.
My understanding is that the answer and ask dialog "freeze" any other script (not being LC "multi processing" or what is the word for it)

[dunbarx]

The answer dialog, without fiddling with it, is always presented at the same loc on screen. On that local machine, it is not possible to click at the loc of the "OK" button, since the dialog is blocking. But from the outside, not a problem. For example, I use a macro program called "KeyBoard Maestro". I made a quick macro where the mouse was moved to the loc of the LC "OK" button and clicked.

With the LC answer dialog open, I switched to another app and invoked the macro. The button in LC was clicked, and the dialog dismissed.

The question is this. Does the remote sending stack, in communication with the local LC program, act like an "outside" app? If not, can this be exploited in another way?

Craig

[trevix]

The question is this. Does the remote sending stack, in communication with the local LC program, act like an "outside" app?

Yes: server and client are on two different computers, connected to the same wifi

[trevix]

To make it more clear:
the server App runs on a Android TvBox, attached to a TV set that may be at 6-8 meters from the ground and connected to the wifi local network.
In the past we were using a small remote bluetooth/radio keyboard, in order to change the setting of the app running in the tvBox: but this is very difficult, particularly if the ambient light is not optimal or your fingers must deal with a 10$ chinese keyboard.

We then started using AnyDesk from a PC: works fine, but requires a PC on the same network.On a cell phone AnyDesk is barely useful. But from a marketing point of view we don't think it is a good choice (the need of a third party App in order to use our App).

The idea then became, in order to modify settings when needed (for example to enter the names of players, since it is a Sport App), to be able FROM a cell phone to clik here and there on the tvbox screens, : showing the tvbox screens on the cell phone and sending mouse clicks to the tv box.

On a local network, for what I tested, it works very well and very fast over WiFi.
But I am afraid of the difficulties that I will encounter in particular cases, like Answer/Ask dialogs, or entering text, or emulate finger slide.
What do you think?

[dunbarx]

Have you considered rolling your own Answer/Ask dialog boxes? They can look any way you want them to, will do the same tasks and more, and are not blocking. In other words, a homemade dialog box can send and receive messages in the usual way.

Craig

[bwmilby]

If you are writing a custom companion mobile app then I would change the way of going about it. Use the socket communications but actually reproduce the UI in the mobile app and just send data. Your TV app would provide the screen ID and then when the user entered data it would magically appear in the correct place. Commands (clicks) would be sent by name and not location.

I agree about rolling your own dialogs. Just have a group that you show for the dialog and then hide when done. That method works well on mobile when you want full control of the presentation.

[trevix]

About rolling my Answer dialog:
I thought about doing it. It is just that I have quite a number LC default dialogs already. I would have to to do more then just opening a small modal stack with 2 buttons: would have to accomodate for variable number of buttons, changes in text length, etc.
Not so fast...the standalone has grown over time more than I imagined, with more the 20 substacks and around 100MB.

I forgot to mention that the App on the cell phone and the App on the TvBox are "exactly" the same. They only differentiate on opening by the screen size and, according to the hardware, slightly changing the behaviour.
My server and client socket libraries are both on the App and get loaded accordingly when the App detect to be running on cell or tvbox.

The original idea of using ClickLoc over an image sent by the server, is because the user somehow would enter a sort of "mirror" mode, operating on the tvbox/server objects, not on its own UI. Like on desktop remote control software: the client software does not need to know anything about the UI of the controlled device.
Using names, I would have to intercept on the cell the script beginning of every object that has a mousedown/keydown (to send its name and action) and also allow the cell UI to run the rest of the script, otherwise server and client would not be synchronised (like a card change). But this would provide changes to the cell UI also (like a different value on a field) and this is not possible.

But I understand that using long names of control, would be easier even if a little rigid.

bwmilby@:

Your TV app would provide the screen ID...

I don't understand