Text-to-speech TTS for iOS
Text-to-speech TTS for iOS
I am trying, with LCB FFI, to implement a simple Text To Speech on iOS standalone.
As of yet, ripping some examples and with the help of Trevor DeVore, I have been able to make it work on OSX (LC 9.6.0 DP2, OSX 10.14.6).
See my post: https://forums.livecode.com/viewtopic.p ... 20#p187920
I guess it should work more or less the same on iOS, using the "AVSpeechSynthesizer" instead of the OSX "NSSpeechSynthesizer"
I know nothing of Objc, I am struggling with Livecode Builder and there is so much to be read, so it would be nice to receive some help in a collaborative way so that we can finally come up with a solution for something, in my opinion, so useful.
Note:
- Thierry Douez has ceased developing its TTS external, so no solution here.
- TTS should be available off-line
- Here a list of useful links
http://www.informit.com/articles/articl ... 8&seqNum=7
https://useyourloaf.com/blog/synthesize ... from-text/
https://developer.apple.com/documentati ... ynthesizer
Thanks to anybody who will share some light.
As of yet, ripping some examples and with the help of Trevor DeVore, I have been able to make it work on OSX (LC 9.6.0 DP2, OSX 10.14.6).
See my post: https://forums.livecode.com/viewtopic.p ... 20#p187920
I guess it should work more or less the same on iOS, using the "AVSpeechSynthesizer" instead of the OSX "NSSpeechSynthesizer"
I know nothing of Objc, I am struggling with Livecode Builder and there is so much to be read, so it would be nice to receive some help in a collaborative way so that we can finally come up with a solution for something, in my opinion, so useful.
Note:
- Thierry Douez has ceased developing its TTS external, so no solution here.
- TTS should be available off-line
- Here a list of useful links
http://www.informit.com/articles/articl ... 8&seqNum=7
https://useyourloaf.com/blog/synthesize ... from-text/
https://developer.apple.com/documentati ... ynthesizer
Thanks to anybody who will share some light.
Trevix
OSX 14.3.1 xCode 15 LC 10 DP7 iOS 15> Android 7>
OSX 14.3.1 xCode 15 LC 10 DP7 iOS 15> Android 7>
-
- Posts: 854
- Joined: Wed Nov 04, 2009 11:41 am
- Location: Gunthorpe, North Lincs, UK
Re: Text-to-speech TTS for iOS
Hi Trevix,
I have just started with LCB and like you have next to no knowledge of objective-c / cocoa or even c. With that in mind I am happy to try and help.
I have just started with LCB and like you have next to no knowledge of objective-c / cocoa or even c. With that in mind I am happy to try and help.
best wishes
Skids
Skids
Re: Text-to-speech TTS for iOS
Welcome to "Via Crucis".
First I guess we need someone to confirm some facts, that I report from various site (that will probably embarasse me with more experienced developers):
Otherwise, implementing this in a foreign handler would be a good starting point:
First I guess we need someone to confirm some facts, that I report from various site (that will probably embarasse me with more experienced developers):
AV Foundation is Apple’s advanced Objective-C framework for working with time-based media on OS X and iOS
A software framework is an abstraction in which software providing generic functionality can be selectively changed by additional user-written code
In object-oriented programming, a class is an extensible program-code-template for creating objects, providing initial values for state (member variables) and implementations of behavior (member functions or methods) (??!!).
Give the above, we need first to understand if in the LCB code there is the need to call(import?) the AV Foundation framework and how. Like you do "use com.livecode.objc". That is, if the LCB library for AVSpeechSynthesizer we are going to build requires something that may not be available to LCB . (And that would be the end of it I guess).In Swift parlance, a module is a compiled group of code that is distributed together. A framework is one type of module. The AVSpeechSynthesizer is a class of the AV Foundation framework
Otherwise, implementing this in a foreign handler would be a good starting point:
Code: Select all
AVSpeechSynthesizer *synthesizer = [[AVSpeechSynthesizer alloc] init];
AVSpeechUtterance *utterance =
[[AVSpeechUtterance alloc] initWithString:@"Hello World!"];
[synthesizer speakUtterance:utterance];
Trevix
OSX 14.3.1 xCode 15 LC 10 DP7 iOS 15> Android 7>
OSX 14.3.1 xCode 15 LC 10 DP7 iOS 15> Android 7>
-
- Posts: 854
- Joined: Wed Nov 04, 2009 11:41 am
- Location: Gunthorpe, North Lincs, UK
Re: Text-to-speech TTS for iOS
Well with the understanding that I am an inexperienced developer here goes.
1. Yes it seems so: with specific frameworks being accessed via foreign handlers in the script. See in the examples below that the EventKit framework is referenced in the handler.
3. Yes a Class is a template that may contain default values along with other variables and "methods" to manipulate them. Objects are based on a Class template e.g. the Class "person" , may have properties such as sex. A person object will have a name and be created with a line similar to Set tStephen as new person. Then the property is set through a setter method e.g. tStephen.SetSex = "male". Note that the dot notation is quite common but it is not used in objective-c.
4. I don't know about Swift apart from it looks nicer than Objective-C. This link confirms that AVSpeechSynthesizer is part of the AVFoundation.https://www.hackingwithswift.com/exampl ... hesisvoice
Looking at Objective-c examples you often see the Alloc method inside an init method. This is how Apple recommend it be done. However, LCB does not allow nested calls like this. So two foreign handlers have to be written and used. The first calls the +alloc method which is a Class method (+). the second calls the -init method which is an object instance method (-). Have a close look at how I create an Eventkit eventstore object in my LCB script :
Next I set up a native variable to store the reference to the Objective-C object:
These are called later in the LCB script (is it script or code?) with :
The result of this handler gets stored in the variable sEventStore for later use.
I'll try and look at your code snip in detail later today.
1. Yes it seems so:
2. A framework is to Apple as a Dynamic Linked Library (DLL) is to Windows. They both add additional commands that programs may use. Often these commands are added to program code via a use library or similar command. For example in LCB this isAVFoundation is the full featured framework for working with time-based audiovisual media on iOS, macOS, watchOS and tvOS. Using AVFoundation, you can easily play, create, and edit QuickTime movies and MPEG-4 files, play HLS streams, and build powerful media functionality into your apps.
Code: Select all
use.com.livecode.objc
3. Yes a Class is a template that may contain default values along with other variables and "methods" to manipulate them. Objects are based on a Class template e.g. the Class "person" , may have properties such as sex. A person object will have a name and be created with a line similar to Set tStephen as new person. Then the property is set through a setter method e.g. tStephen.SetSex = "male". Note that the dot notation is quite common but it is not used in objective-c.
4. I don't know about Swift apart from it looks nicer than Objective-C. This link confirms that AVSpeechSynthesizer is part of the AVFoundation.https://www.hackingwithswift.com/exampl ... hesisvoice
Looking at Objective-c examples you often see the Alloc method inside an init method. This is how Apple recommend it be done. However, LCB does not allow nested calls like this. So two foreign handlers have to be written and used. The first calls the +alloc method which is a Class method (+). the second calls the -init method which is an object instance method (-). Have a close look at how I create an Eventkit eventstore object in my LCB script :
Code: Select all
-- bind the memory allocation handler - ok
private foreign handler ObjC_EKEventStoreAlloc() \
returns optional ObjcRetainedID \
binds to "objc:EventKit>EKEventStore.+alloc"
-- bind the initialisation of new object - ok
private foreign handler ObjC_EKEventStoreInit(in pObj as ObjcRetainedID,in EKEntityMaskEvent as CUInt) \
returns optional ObjcID \
binds to "objc:EventKit>EKEventStore.-initWithAccessToEntityTypes:"
Next I set up a native variable to store the reference to the Objective-C object:
Code: Select all
-- declare static variable to store the long lived EventStore object
private variable sEventStore as optional ObjcObject
Code: Select all
private handler InitialiseEventStore() returns ObjcObject
variable tEventStore as optional ObjcObject
unsafe
put ObjC_EKEventStoreAlloc() into tEventStore
return ObjC_EKEventStoreInit(tEventStore, 0)
end unsafe
end handler
I'll try and look at your code snip in detail later today.
best wishes
Skids
Skids
-
- Posts: 854
- Joined: Wed Nov 04, 2009 11:41 am
- Location: Gunthorpe, North Lincs, UK
Re: Text-to-speech TTS for iOS
Oh one hot tip from Trevor was to look for the header files of the framework on github. This was the only way to discover that the EKEntityMaskEvent was in fact an unsigned integer with a value of zero.
best wishes
Skids
Skids
-
- Posts: 854
- Joined: Wed Nov 04, 2009 11:41 am
- Location: Gunthorpe, North Lincs, UK
Re: Text-to-speech TTS for iOS
Probably worth expanding on the foreign handler shown above:
The first parameter pObj is a reference to the object that the method call is being sent to. We know that the call is going to an instance of the EKEventStore Class because the method name starts with a minus sign. The colon signifies that a parameter is also being sent. The parameter sent is the second in the list of our foreign handler i.e. EKEntityMaskEvent in this example.
The next example shows multiple parameters being sent to an objects method:
Note that the full method name is "predicateForEventsWithStartDate:endDate:calendars:" colons included. The colons indicate points where variables are inserted and believe it or not are suppose to make Objective-C code self documenting.
Code: Select all
-- bind the initialisation of new object - ok
private foreign handler ObjC_EKEventStoreInit(in pObj as ObjcRetainedID,in EKEntityMaskEvent as CUInt) \
returns optional ObjcID \
binds to "objc:EventKit>EKEventStore.-initWithAccessToEntityTypes:"
The next example shows multiple parameters being sent to an objects method:
Code: Select all
private foreign handler ObjcPredicateForEventsBetweenTwoDates(pObj as objcRetainedID, in pStart as objcID, pEnd as objcID, in pCalArray as objcID) \
returns optional objcID \
binds to "objc:EventKit>EKEventStore.-predicateForEventsWithStartDate:endDate:calendars:"
best wishes
Skids
Skids
Re: Text-to-speech TTS for iOS
Hello.
Trying to implement the choice of voice for this iOS TTS, I am still struggling understanding how to go from Objc examples (found on the net) to LCB implementation.
Like here: https://useyourloaf.com/blog/synthesize ... from-text/
where I read:
Or first I have to do a alloc/init initialisation of "AVSpeechSynthesisVoice"?
But after that?
Trying to implement the choice of voice for this iOS TTS, I am still struggling understanding how to go from Objc examples (found on the net) to LCB implementation.
Like here: https://useyourloaf.com/blog/synthesize ... from-text/
where I read:
So, I just have to find out how to add the above to the following Utterance Init block?The following code snippet would, for example, set the utterance to use an Australian voice:
utterance.voice = [AVSpeechSynthesisVoice voiceWithLanguage:@"en-AU"];
Code: Select all
private foreign handler ObjC_AVSpeechUtteranceInitWithString(in pObj as ObjcID, in pText as objcID) \
returns optional ObjcRetainedID \
binds to "objc:AVFoundation>AVSpeechUtterance.-initWithString:"
Code: Select all
private foreign handler ObjC_AVSpeechSynthesisVoiceAlloc() \
returns optional ObjcID \
binds to "objc:AVFoundation>AVSpeechSynthesisVoice.+alloc"
private foreign handler ObjC_AVSpeechSynthesisVoice(in pObj as ObjcID) \
returns optional ObjcRetainedID \
binds to "objc:AVFoundation>AVSpeechSynthesisVoice.-init"
Trevix
OSX 14.3.1 xCode 15 LC 10 DP7 iOS 15> Android 7>
OSX 14.3.1 xCode 15 LC 10 DP7 iOS 15> Android 7>
-
- Posts: 854
- Joined: Wed Nov 04, 2009 11:41 am
- Location: Gunthorpe, North Lincs, UK
Re: Text-to-speech TTS for iOS
Hi,
Before using the init with string method you should call the alloc method of the class.
When you call this as a function call you store the returned ObjcID in a LCB variable that you have declared as a objcObject tyoe :
Next you pass this as the ObjID to your initwithvoice handler which returns a pointer to an objc Objectthat you store in an LCB variable. This variable can and probably be the same one as you used for the alloc call.
Next you can use your variable as an AVSpeechSynthesizer object in calls to methods of AVSpeechSynthesizer . Of course you have to write a foreign object handler for each method that you wish to use.
Several other issues present themselves. The first is that most of the method parameters are passed as objects. This is because ObjectiveC is all about objects. The exceptions are that some parameters may be passed in as integers. So if you with to pass an integer you may declare a variable of type Cint or any of these :
Another issue is that methods often return objects or arrays of objects. These objects have to be "unpicked" using yet more foreign handlers to eventually arrive at variables that LCB is able to deal with i.e. NSstring, Cints etc.
So looking at your code snip above the parameter ptext most likely should be a NSstring object when its passed. See https://developer.apple.com/documentati ... guage=objc
So in summary you only need to call one form of -init, but you need to be careful how parameters are typed. One last thing is that the initwithvoice method is only available with the NsSpeechSynthesizer object and I don't believe that this object works on iOS. A quick look of the documentation indicates that on iOS the voice can be passed as a property of the utterance to a AVSpeechSynthesizer object. The snag is that it is as an object, https://developer.apple.com/documentati ... guage=objc so more research required.
I hope that the above is of some help.
Before using the init with string method you should call the alloc method of the class.
Code: Select all
-- bind the memory allocation handler - ok
private foreign handler ObjC_AVSpeechSynthAlloc() \
returns optional ObjcID \
binds to "objc:AVFoundation>AVSpeechSynthesizer.+alloc"
Code: Select all
variable tSpeechSynth as objcObject
Next you can use your variable as an AVSpeechSynthesizer object in calls to methods of AVSpeechSynthesizer . Of course you have to write a foreign object handler for each method that you wish to use.
Several other issues present themselves. The first is that most of the method parameters are passed as objects. This is because ObjectiveC is all about objects. The exceptions are that some parameters may be passed in as integers. So if you with to pass an integer you may declare a variable of type Cint or any of these :
You should note that there is no such thing as a string in C, the nearest the language gets is a character. However, Objective-C uses a NSstring object as its way of working with strings, and these objects are often used as parameters in Objective-C methods. Thankfully LCB provides two methods for converting native strings to foreign NSstring objects. These are described in the dictionary.There are the standard C primitive types (defined in the foreign module)
• CBool maps to 'bool'
• CChar, CSChar and CUChar map to 'char', 'signed char' and 'unsigned char'
• CShort/CSShort and CUShort map to 'signed short' and 'unsigned short'
• CInt/CSInt and CUInt map to 'signed int' and 'unsigned int'
• CLong/CSLong and CULong map to 'signed long' and 'unsigned long'
• CLongLong/CSLongLong and CULongLong map to 'signed long long' and 'unsigned long long'
• CFloat maps to 'float'
• CDouble maps to 'double'
There are types specific to Obj-C types (defined in the objc module):
• ObjcObject wraps an obj-c 'id', i.e. a pointer to an objective-c object
• ObjcId maps to 'id'
• ObjcRetainedId maps to 'id', and should be used where a foreign handler argument expects a +1 reference count, or where a foreign handler returns an id with a +1 reference count.
Another issue is that methods often return objects or arrays of objects. These objects have to be "unpicked" using yet more foreign handlers to eventually arrive at variables that LCB is able to deal with i.e. NSstring, Cints etc.
So looking at your code snip above the parameter ptext most likely should be a NSstring object when its passed. See https://developer.apple.com/documentati ... guage=objc
So in summary you only need to call one form of -init, but you need to be careful how parameters are typed. One last thing is that the initwithvoice method is only available with the NsSpeechSynthesizer object and I don't believe that this object works on iOS. A quick look of the documentation indicates that on iOS the voice can be passed as a property of the utterance to a AVSpeechSynthesizer object. The snag is that it is as an object, https://developer.apple.com/documentati ... guage=objc so more research required.
I hope that the above is of some help.
best wishes
Skids
Skids
-
- Posts: 854
- Joined: Wed Nov 04, 2009 11:41 am
- Location: Gunthorpe, North Lincs, UK
Re: Text-to-speech TTS for iOS
ooops I misread your post and my own code....
So with this code :
This creates an AVSpeechSynthesizer object and stores in variable, then a AVSpeech Utterance object which is stored in another variable. Lastly the utterance object is passed to the speechSynthesazer object.
So you want to specify the voice to use. The Speech Utterance object has a voice property which is an AVSpeechSynthesisVoice object. So this needs creating before the utterance and a number of foreign handlers will have to be written to create the object and then assign the object to the Utterance object before it is passed to AVSpeechSynthesizer object.
How? No idea at present but I will have a think about it.
So with this code :
Code: Select all
unsafe
-- convert the string passed into a NSString object
put StringToNSString(pText) into tTextToSpeak
-- create instance of SpeechSynth
put ObjC_AVSpeechSynthAlloc() into tSpeechSynth
put ObjC_AVSpeechSynthInit(tSpeechSynth) into tSpeechSynth
-- create instance of Utterance
put ObjC_AVSpeechUtteranceAlloc() into tUtterance
put ObjC_AVSpeechUtteranceInitWithString(tUtterance,tTextToSpeak) into tUtterance
-- Speak the Utterance
ObjC_SpeakUtterance(tSpeechSynth,tUtterance)
end unsafe
So you want to specify the voice to use. The Speech Utterance object has a voice property which is an AVSpeechSynthesisVoice object. So this needs creating before the utterance and a number of foreign handlers will have to be written to create the object and then assign the object to the Utterance object before it is passed to AVSpeechSynthesizer object.
How? No idea at present but I will have a think about it.
best wishes
Skids
Skids
-
- Posts: 854
- Joined: Wed Nov 04, 2009 11:41 am
- Location: Gunthorpe, North Lincs, UK
Re: Text-to-speech TTS for iOS
For clarity here is the complete working yet unfinished library that I started :
library net.anvic.skids.speechsynth
use com.livecode.foreign
use com.livecode.objc
metadata title is "Speech Synth Library"
metadata author is "Simon (Skids) Knight"
metadata version is "1.0.0"
-- bind the memory allocation handler - ok
private foreign handler ObjC_AVSpeechSynthAlloc() \
returns optional ObjcID \
binds to "objc:AVFoundation>AVSpeechSynthesizer.+alloc"
-- bind the initialisation of new object - ok
private foreign handler ObjC_AVSpeechSynthInit(in pObj as ObjcID) \
returns optional ObjcRetainedID \
binds to "objc:AVFoundation>AVSpeechSynthesizer.-init"
------
-- Now create an 'Utterance' to be passed to the AVSpeechSynthesizer
private foreign handler ObjC_AVSpeechUtteranceAlloc() \
returns optional ObjcID \
binds to "objc:AVFoundation>AVSpeechUtterance.+alloc"
private foreign handler ObjC_AVSpeechUtteranceInitWithString(in pObj as ObjcID, in pText as objcID) \
returns optional ObjcRetainedID \
binds to "objc:AVFoundation>AVSpeechUtterance.-initWithString:"
-- Now the method to send the utterance to the AVSpeechSynthesizer
private foreign handler ObjC_SpeakUtterance(in pObj as ObjcID, in pUtterance as ObjcID) \
returns nothing \
binds to "objc:AVFoundation>AVSpeechSynthesizer.-speakUtterance:"
/*
-- Need to be able to create an NSString object
private foreign handler ObjC_NSStringAlloc() \
returns objcRetainedID \
binds to "objc:NSString.+alloc"
private foreign handler ObjC_NSStringInitWithString(in pObj as ObjcID, in pString as String) \
returns objcID \
binds to "objc:NSString.-initWithString:"
*/
public handler smkSpeakText(in pText as String) returns String
variable tSpeechSynth as objcObject
variable tUtterance as objcObject
variable tTextToSpeak as objcObject
unsafe
-- convert the string passed into a NSString object
put StringToNSString(pText) into tTextToSpeak
-- create instance of SpeechSynth
put ObjC_AVSpeechSynthAlloc() into tSpeechSynth
put ObjC_AVSpeechSynthInit(tSpeechSynth) into tSpeechSynth
-- create instance of Utterance
put ObjC_AVSpeechUtteranceAlloc() into tUtterance
put ObjC_AVSpeechUtteranceInitWithString(tUtterance,tTextToSpeak) into tUtterance
-- Speak the Utterance
ObjC_SpeakUtterance(tSpeechSynth,tUtterance)
end unsafe
return "Complete"
end handler
end library
best wishes
Skids
Skids
-
- Posts: 854
- Joined: Wed Nov 04, 2009 11:41 am
- Location: Gunthorpe, North Lincs, UK
Re: Text-to-speech TTS for iOS
So the first step is revealed here : https://developer.apple.com/documentati ... jc#1668668
The "Finding Voices" link points back to the AVspeechSynthesizer class and informs us that the class provide a list of voice objects. This means that with the right foreign handler we can just ask the class and it will return a list of AVSpeechSynthesisVoice objects.
Once we get these the code will have to extract the names and return these to LCS so that the user may choose a name.
Then the chosen name will be passed back with the text to speak, the AVSpeechSynthesisVoice object extracted and passed as a parameter to the utterance object along with the text to speak.
Well its a plan....
To select a voice for use in speech, obtain an AVSpeechSynthesisVoice instance using one of the methods in Finding Voices, and then set it as the value of the voice property on an AVSpeechUtterance instance containing text to be spoken.
The "Finding Voices" link points back to the AVspeechSynthesizer class and informs us that the class provide a list of voice objects. This means that with the right foreign handler we can just ask the class and it will return a list of AVSpeechSynthesisVoice objects.
Once we get these the code will have to extract the names and return these to LCS so that the user may choose a name.
Then the chosen name will be passed back with the text to speak, the AVSpeechSynthesisVoice object extracted and passed as a parameter to the utterance object along with the text to speak.
Well its a plan....
best wishes
Skids
Skids
-
- Posts: 854
- Joined: Wed Nov 04, 2009 11:41 am
- Location: Gunthorpe, North Lincs, UK
Re: Text-to-speech TTS for iOS
Here is some progress using the following handlers:
code:
A call to smkListOfVoices() returns the following on my computer:
The handler AV_ReadVoices(in pNSArrayOfVoices as objcID) returns String does the work first converting the NSArray of voice objects passed to it into a native list. It then loops through the list calling the second foreign handler to read the name which is returned as an NSstring object.
Fortunately LCB is able to convert to and from NSstrings and the result of each conversion is added to the variable tFoundVoices which gets returned to the LCS caller.
Traps for young and not so young players include mistakes in the syntax of the foreign handler binding string. a "+" or a "-" indicates that a method is being called. A ":" indicates that a parameter is being used. No colon means no parameter.
code:
Code: Select all
-- return a list of voices in an array of AVSpeechSynthesisVoice objects
private foreign handler ObjC_AVSpeechSynthesisVoices() \
returns ObjcID \
binds to "objc:AVFoundation>AVSpeechSynthesisVoice.+speechVoices"
private foreign handler Objc_NameOfVoiceNSstring (in pObj as objcID) \
returns ObjcID \
binds to "objc:AVFoundation>AVSpeechSynthesisVoice.name"
public handler smkListOfVoices() returns String
variable tArrayOfVoices as objcObject // array of AVSpeechSythesisVoice Objects
variable tVoices as String
unsafe
put ObjC_AVSpeechSynthesisVoices() into tArrayOfVoices
put AV_ReadVoices(tArrayOfVoices) into tVoices
end unsafe
return tVoices
end handler
private handler AV_ReadVoices(in pNSArrayOfVoices as objcID) returns String
variable tVoicesList as List -- native List LCB type
variable tVoiceObj as objcObject -- AVSpeechSythesisVoice object
variable tNameObj as objcObject -- NSString
variable tFoundVoices as String -- native string that will be returned
if pNSArrayOfVoices is not nothing then
put listFromNSArray(pNSArrayOfVoices) into tVoicesList
else
return "Empty Array Passed into AV_ReadVoices"
end if
-- Loop through the voice objects in the list reading the name
repeat for each element tVoiceObj in tVoicesList
unsafe
if Objc_NameOfVoiceNSstring(tVoiceObj) is not nothing then
put Objc_NameOfVoiceNSstring(tVoiceObj) into tNameObj
put StringFromNSString(tNameObj) & ";" after tFoundVoices
end if
end unsafe
end repeat
-- tidy up return string
if tFoundVoices ends with ";" then
delete the last char of tFoundVoices
end if
return tFoundVoices
end handler
The code follows the rough design I outlined above. The first foreign handler calls the class method as indicated by the "+" character prefix of the method name. This returns an NSArray of Voice objects.Alex;Alice;Alva;Amelie;Anna;Carmit;Damayanti;Daniel;Diego;Ellen;Fiona;Fred;Ioana;Joana;Jorge;Juan;Kanya;Karen;Kyoko;Laura;Lekha;Luca;Luciana;Maged;Mariska;Mei-Jia;Melina;Milena;Moira;Monica;Nora;Paulina;Rishi;Samantha;Sara;Satu;Sin-ji;Tessa;Thomas;Ting-Ting;Veena;Victoria;Xander;Yelda;Yuna;Yuri;Zosia;Zuzana
The handler AV_ReadVoices(in pNSArrayOfVoices as objcID) returns String does the work first converting the NSArray of voice objects passed to it into a native list. It then loops through the list calling the second foreign handler to read the name which is returned as an NSstring object.
Fortunately LCB is able to convert to and from NSstrings and the result of each conversion is added to the variable tFoundVoices which gets returned to the LCS caller.
Traps for young and not so young players include mistakes in the syntax of the foreign handler binding string. a "+" or a "-" indicates that a method is being called. A ":" indicates that a parameter is being used. No colon means no parameter.
best wishes
Skids
Skids
-
- Posts: 854
- Joined: Wed Nov 04, 2009 11:41 am
- Location: Gunthorpe, North Lincs, UK
Re: Text-to-speech TTS for iOS
The next step is to obtain a voice based on a name passed in from Livecode Script (LCS).
This will use handlers very much like those just created. It would be useful to know if the array of voices is destroyed once the LCB handler that uses it completes. If, as I suspect it is then the LCB variable declaration should be moved outside the handlers to make it a static. This means that the next call from LCS to speak this text with this voice will be able to access the array of voices.
to be continued..........
This will use handlers very much like those just created. It would be useful to know if the array of voices is destroyed once the LCB handler that uses it completes. If, as I suspect it is then the LCB variable declaration should be moved outside the handlers to make it a static. This means that the next call from LCS to speak this text with this voice will be able to access the array of voices.
to be continued..........
best wishes
Skids
Skids
-
- Posts: 854
- Joined: Wed Nov 04, 2009 11:41 am
- Location: Gunthorpe, North Lincs, UK
Re: Text-to-speech TTS for iOS
After some hours of head scratching here is an update to the library:
The following additions have been made:
1) The declaration of the list of voice objects has been moved to make it a static (line 53).
2) The handler smkSpeakText has been modified to allow the name of the required voice to be passed into it (line 104).
3) Lines 112 calls a new handler to find and return the voice object with the required name.
4) Line 134 calls a new foreign handler that sets the voice property of the Utterance object.
5) Lines 140-164 are the new handler called from line 112 - this loops the list seeking a voice object of the correct name.
6) Lines 33 to 35 define the new foreign handler:
The learning point for me as a non-objectiveC programmer is that the documentation lists details of the voice property and that to assign a value to the property the property name has to be capitalised and prefixed with "set". This to set the voice the code uses "setVoice".
Note that this code works but is far from complete. For example there is limited error handling so the name must exist. Also the handler smkListOfVoices() has to be called once to build the list of voice objects before the second handler is used e.g.
From Livecode Script do the following
I hope what I have written makes some sense and shows how I worked out how to access the various methods.
Code: Select all
library net.anvic.skids.speechsynth
use com.livecode.foreign
use com.livecode.objc
metadata title is "Speech Synth Library"
metadata author is "Simon (Skids) Knight"
metadata version is "1.0.0"
-- bind the memory allocation handler - ok
private foreign handler ObjC_AVSpeechSynthAlloc() \
returns optional ObjcID \
binds to "objc:AVFoundation>AVSpeechSynthesizer.+alloc"
-- bind the initialisation of new object - ok
private foreign handler ObjC_AVSpeechSynthInit(in pObj as ObjcID) \
returns optional ObjcRetainedID \
binds to "objc:AVFoundation>AVSpeechSynthesizer.-init"
-- Now create an 'Utterance' to be passed to the AVSpeechSynthesizer
private foreign handler ObjC_AVSpeechUtteranceAlloc() \
returns optional ObjcID \
binds to "objc:AVFoundation>AVSpeechUtterance.+alloc"
private foreign handler ObjC_AVSpeechUtteranceInitWithString(in pObj as ObjcID, in pText as objcID) \
returns optional ObjcRetainedID \
binds to "objc:AVFoundation>AVSpeechUtterance.-initWithString:"
private foreign handler ObjC_AVSpeechUtteranceSetVoice(in pUtteranceObj as ObjcID, in pVoiceObject as objcID) \
returns optional ObjcRetainedID \
binds to "objc:AVFoundation>AVSpeechUtterance.setVoice:"
-- Now the method to send the utterance to the AVSpeechSynthesizer
private foreign handler ObjC_SpeakUtterance(in pObj as ObjcID, in pUtterance as ObjcID) \
returns nothing \
binds to "objc:AVFoundation>AVSpeechSynthesizer.-speakUtterance:"
--
-- return a list of voices in an array of AVSpeechSynthesisVoice objects
private foreign handler ObjC_AVSpeechSynthesisVoices() \
returns ObjcID \
binds to "objc:AVFoundation>AVSpeechSynthesisVoice.+speechVoices"
private foreign handler Objc_NameOfVoiceNSstring (in pObj as objcID) \
returns ObjcID \
binds to "objc:AVFoundation>AVSpeechSynthesisVoice.name"
variable sVoicesList as List -- of objcObjects that should stay available between calls
public handler smkListOfVoices() returns String
// Note this should always be called even if LCS knows the name of a voice
// it wants to use.
variable tArrayOfVoices as objcObject // array of AVSpeechSythesisVoice Objects
variable tVoices as String
unsafe
put ObjC_AVSpeechSynthesisVoices() into tArrayOfVoices
put AV_ReadVoices(tArrayOfVoices) into tVoices
end unsafe
return tVoices
end handler
private handler AV_ReadVoices(in pNSArrayOfVoices as objcID) returns String
variable tVoicesList as List -- native List LCB type
variable tVoiceObj as objcObject -- AVSpeechSythesisVoice object
variable tNameObj as objcObject -- NSString
variable tFoundVoices as String -- native string that will be returned
if pNSArrayOfVoices is not nothing then
put listFromNSArray(pNSArrayOfVoices) into sVoicesList
else
return "Empty Array Passed into AV_ReadVoices"
end if
-- Loop through the voice objects in the list reading the name
repeat for each element tVoiceObj in sVoicesList
unsafe
if Objc_NameOfVoiceNSstring(tVoiceObj) is not nothing then
put Objc_NameOfVoiceNSstring(tVoiceObj) into tNameObj
put StringFromNSString(tNameObj) & ";" after tFoundVoices
end if
end unsafe
end repeat
-- tidy up return string
if tFoundVoices ends with ";" then
delete the last char of tFoundVoices
end if
return tFoundVoices
end handler
public handler smkSpeakText(in pText as String, in pVoiceName as String) returns String
variable tSpeechSynth as objcObject
variable tUtterance as objcObject
variable tTextToSpeak as objcObject
variable tVoiceObj as objcObject
-- call function to look up and return voice object
put GetVoiceObjectNamed(pVoiceName) into tVoiceObj
if tVoiceObj is nothing then
return " error getting voice object to use"
end if
unsafe
-- convert the string passed into a NSString object
put StringToNSString(pText) into tTextToSpeak
-- create instance of SpeechSynth
put ObjC_AVSpeechSynthAlloc() into tSpeechSynth
put ObjC_AVSpeechSynthInit(tSpeechSynth) into tSpeechSynth
-- create instance of Utterance
put ObjC_AVSpeechUtteranceAlloc() into tUtterance
put ObjC_AVSpeechUtteranceInitWithString(tUtterance,tTextToSpeak) into tUtterance
-- new code to set voice
ObjC_AVSpeechUtteranceSetVoice(tUtterance, tVoiceObj)
-- Lastly Speak the Utterance
ObjC_SpeakUtterance(tSpeechSynth,tUtterance)
end unsafe
return "Complete"
end handler
private handler GetVoiceObjectNamed(in pVoiceName as String) returns objcObject
variable tVoiceObj as ObjcObject -- this will be returned
variable tNameObj as ObjcObject -- An NSstring object
variable tVoiceName as String
-- Loop through the voice objects in the list reading the name
repeat for each element tVoiceObj in sVoicesList
unsafe
if Objc_NameOfVoiceNSstring(tVoiceObj) is not nothing then
put Objc_NameOfVoiceNSstring(tVoiceObj) into tNameObj
put StringFromNSString(tNameObj) into tVoiceName
if tVoiceName is pVoiceName then
-- correct object found so stop repeating
exit repeat
end if
else
-- an error has occured
return nothing
end if
end unsafe
end repeat
-- this will either return the correct voice or the last voice in list
return tVoiceObj
end handler
end library
1) The declaration of the list of voice objects has been moved to make it a static (line 53).
2) The handler smkSpeakText has been modified to allow the name of the required voice to be passed into it (line 104).
3) Lines 112 calls a new handler to find and return the voice object with the required name.
4) Line 134 calls a new foreign handler that sets the voice property of the Utterance object.
5) Lines 140-164 are the new handler called from line 112 - this loops the list seeking a voice object of the correct name.
6) Lines 33 to 35 define the new foreign handler:
Code: Select all
private foreign handler ObjC_AVSpeechUtteranceSetVoice(in pUtteranceObj as ObjcID, in pVoiceObject as objcID) \
returns optional ObjcRetainedID \
binds to "objc:AVFoundation>AVSpeechUtterance.setVoice:"
Note that this code works but is far from complete. For example there is limited error handling so the name must exist. Also the handler smkListOfVoices() has to be called once to build the list of voice objects before the second handler is used e.g.
From Livecode Script do the following
Code: Select all
put smkListOfVoices() into tListOfNames
put put smkSpeakText("Hello world my name is Simon", "Anna")
best wishes
Skids
Skids
-
- Posts: 627
- Joined: Wed Apr 24, 2013 4:53 pm
- Contact:
Re: Text-to-speech TTS for iOS
Correct, if you define the variables within your handler then they are local variables only accessible to that handler they're in. If you define the variables outside of a handler then they're global scope and accessible to all of your handlers (but not "static" like a literal/constant, they can still be modified). There's also some other situations with certain control structures having to do with the scope of variables defined within them. I can't remember if it was if/then, safe/unsafe, or some other structure, I just remember I've hit that problem before. Also, with Objective C FFI and creating global objects there is some situations where you want 'ObjCRetainedId' instead of the regular 'ObjCId'. This is somewhat of a mystery to me, it has to do with "reference counting", auto vs manual memory management, and creating instances of objects. I usually will try the regular 'ObjCId' way and if it crashes the LC engine, maybe because the object suddenly doesn't exist anymore, then I see if ObjCRetainedId helps (trial & error).Simon Knight wrote: ↑Tue Feb 18, 2020 8:09 pmIf, as I suspect it is then the LCB variable declaration should be moved outside the handlers to make it a static. This means that the next call from LCS to speak this text with this voice will be able to access the array of voices.