LiveCode Forums

Looking for guidance about sounds and LiveCode. For instance, is voice activation possible? Example, if my app is running and I say something, can the app send out a “beep” to show that it can respond. If this is possible, is there a way for a LC app to, usually, distinguish between voices vs other sounds?

I just dreamed up an app but do not want to start working on it if this is not possible with LC. Pointers to documentation would help if it is available. I cannot seem to find anything in the dictionary.

Linda

Hi.

I have never done this, but check out Monte's "Merg" gadgets in the dictionary, especially the "mergMicrophoneStartRecording" command.

Craig

Voice recognition is complex stuff, usually handled by OS services. Most such mechanisms provide a way for developers to integrate voice support into their apps, sometimes with low-level APIs that would require LiveCode Builder to use, sometimes through resources definable at a higher level to register phrases to be recognized and response triggers.

I haven't looked deeply into it myself, so that's as much as I know.

Just seemed worth confirming that it is indeed a non-trivial task in any language, and not just technically but also ethically:

In order to be able to respond to commands directed to the device, the device must listen to EVERYTHING. At all times. And some of that gets sent to the OS vendor for processing to determine if indeed the device was being invoked.

Most voice service providers have a place in your account settings where you can review everything they've recorded. I found it super creepy.

I used to not think about such things, happily enjoying voice control. But after reviewing the snippets of conversations picked up from my home, office, restaurants, and anywhere else I'd been, I'm no longer comfortable having an always-on microphone connected to the Internet.

I've turned off all such services, deleted the recordings, and never looked back.

That is creepy. No, I do not want to record anything. I would even make sure that the app only responds if it is running in the foreground.

It would be an app that wakes the user up if they are talking in their sleep. Sometimes...I also sleepwalk and that is extremely dangerous. Sleep walking, for me, is always proceeded by sleep talking.

I was hoping to distinguish between voice and other noises so that the user would not be woken up just because there was some other noise occurring.

If the complicated part is determining if it is a voice or something else then, what about the app responding if a noise continues for, say, 10 seconds?

Linda

anmldr wrote: ↑

Mon May 10, 2021 5:25 pm

That is creepy. No, I do not want to record anything.

Periodic recording of ambient sound to send to the OS servers for further processing isn't a choice, it's just how voice control systems work.

Your app sounds valuable. I wonder if motion detection might be a less invasive option.

Periodic recording of ambient sound to send to the OS servers for further processing isn't a choice, it's just how voice control systems work.

Thanks. I did not know that.

I wonder if motion detection might be a less invasive option.

I can’t see that working unless it is an app for the Apple watch. When I sleepwalk, I don’t reach for my phone, iPad or laptop

Linda

I have never done this, but check out Monte's "Merg" gadgets in the dictionary, especially the "mergMicrophoneStartRecording" command

.

Thanks. “Merg” is for iOS only, I think. I would want to have an Android version of the app since that is what I have for a phone. I do have an iPad too.

Linda

Could you use an Apple Airtag?

If adding hardware is okay, external motion detectors may be an option.

capellan wrote: ↑

Tue May 11, 2021 10:20 pm

Could you use an Apple Airtag?

Perhaps. I could clip it to my pajamas. But, again, this is only for Apple. And, I am not sure if the motion detection info is available to anything other than the “Find My” app.

FourthWorld wrote: ↑

Tue May 11, 2021 10:35 pm

If adding hardware is okay, external motion detectors may be an option.

Trying to understand...add the motion detection hardware to what? Do you also mean something like the Air Tag?

Even though most people would not want their voice to be recorded, the more that I think about this, I am almost sure that sleeptalkers/sleepwalkers would not care. The reason? Sleepwalking is a very serious safety issue. For me, sleepwalking is almost always proceeded by sleeptalking. It would be better to wake up BEFORE the walking occurs.

Linda

Sounds like a terrific project and a useful one. I'm not sure you can do it with LC though. Motion detection may be a workable solution. You could put a motion detector on the nightstand or somewhere where it isn't pointing at the bed but would catch movement in the surrounding area. If you use a smart detector and Google routines, you can have it only trigger between certain hours when you're sleeping, or start watching when you say the "bedtime" routine keyword. The routine could have Assistant say something at high volume, or turn on a light, or whatever else would wake you up. (You'd need a smart bulb for the light if you choose that method. Having Assistant scream at you might work better anyway.)

Unfortunately I don't think you can use speech detection with this method unless you happen to say "hey Google" in your sleep.

Edit: You're right, AirTags only report location when you monitor them yourself.