Text-to-speech TTS for iOS

Moderators: LCMark, LCfraser

trevix
Posts: 480
Joined: Sat Feb 24, 2007 11:25 pm
Location: Italy
Contact:

Text-to-speech TTS for iOS

Post by trevix » Wed Feb 12, 2020 9:56 am

I am trying, with LCB FFI, to implement a simple Text To Speech on iOS standalone.
As of yet, ripping some examples and with the help of Trevor DeVore, I have been able to make it work on OSX (LC 9.6.0 DP2, OSX 10.14.6).
See my post: https://forums.livecode.com/viewtopic.p ... 20#p187920

I guess it should work more or less the same on iOS, using the "AVSpeechSynthesizer" instead of the OSX "NSSpeechSynthesizer"
I know nothing of Objc, I am struggling with Livecode Builder and there is so much to be read, so it would be nice to receive some help in a collaborative way so that we can finally come up with a solution for something, in my opinion, so useful.

Note:
- Thierry Douez has ceased developing its TTS external, so no solution here.
- TTS should be available off-line
- Here a list of useful links
http://www.informit.com/articles/articl ... 8&seqNum=7
https://useyourloaf.com/blog/synthesize ... from-text/
https://developer.apple.com/documentati ... ynthesizer

Thanks to anybody who will share some light.
Trevix
OSX 10.14.6 LC 9.6.0 Dp2 iOs 9.3>

Simon Knight
Posts: 539
Joined: Wed Nov 04, 2009 11:41 am
Location: Gunthorpe, North Lincs, UK

Re: Text-to-speech TTS for iOS

Post by Simon Knight » Wed Feb 12, 2020 7:36 pm

Hi Trevix,

I have just started with LCB and like you have next to no knowledge of objective-c / cocoa or even c. With that in mind I am happy to try and help.
best wishes
Skids

trevix
Posts: 480
Joined: Sat Feb 24, 2007 11:25 pm
Location: Italy
Contact:

Re: Text-to-speech TTS for iOS

Post by trevix » Thu Feb 13, 2020 12:19 pm

Welcome to "Via Crucis".
First I guess we need someone to confirm some facts, that I report from various site (that will probably embarasse me with more experienced developers):
AV Foundation is Apple’s advanced Objective-C framework for working with time-based media on OS X and iOS
A software framework is an abstraction in which software providing generic functionality can be selectively changed by additional user-written code
In object-oriented programming, a class is an extensible program-code-template for creating objects, providing initial values for state (member variables) and implementations of behavior (member functions or methods) (??!!).
In Swift parlance, a module is a compiled group of code that is distributed together. A framework is one type of module. The AVSpeechSynthesizer is a class of the AV Foundation framework
Give the above, we need first to understand if in the LCB code there is the need to call(import?) the AV Foundation framework and how. Like you do "use com.livecode.objc". That is, if the LCB library for AVSpeechSynthesizer we are going to build requires something that may not be available to LCB . (And that would be the end of it I guess).
Otherwise, implementing this in a foreign handler would be a good starting point:

Code: Select all

AVSpeechSynthesizer *synthesizer = [[AVSpeechSynthesizer alloc] init];
AVSpeechUtterance *utterance =
    [[AVSpeechUtterance alloc] initWithString:@"Hello World!"];
[synthesizer speakUtterance:utterance];
Trevix
OSX 10.14.6 LC 9.6.0 Dp2 iOs 9.3>

Simon Knight
Posts: 539
Joined: Wed Nov 04, 2009 11:41 am
Location: Gunthorpe, North Lincs, UK

Re: Text-to-speech TTS for iOS

Post by Simon Knight » Thu Feb 13, 2020 3:40 pm

Well with the understanding that I am an inexperienced developer here goes.

1. Yes it seems so:
AVFoundation is the full featured framework for working with time-based audiovisual media on iOS, macOS, watchOS and tvOS. Using AVFoundation, you can easily play, create, and edit QuickTime movies and MPEG-4 files, play HLS streams, and build powerful media functionality into your apps.
2. A framework is to Apple as a Dynamic Linked Library (DLL) is to Windows. They both add additional commands that programs may use. Often these commands are added to program code via a use library or similar command. For example in LCB this is

Code: Select all

use.com.livecode.objc
with specific frameworks being accessed via foreign handlers in the script. See in the examples below that the EventKit framework is referenced in the handler.

3. Yes a Class is a template that may contain default values along with other variables and "methods" to manipulate them. Objects are based on a Class template e.g. the Class "person" , may have properties such as sex. A person object will have a name and be created with a line similar to Set tStephen as new person. Then the property is set through a setter method e.g. tStephen.SetSex = "male". Note that the dot notation is quite common but it is not used in objective-c.

4. I don't know about Swift apart from it looks nicer than Objective-C. This link confirms that AVSpeechSynthesizer is part of the AVFoundation.https://www.hackingwithswift.com/exampl ... hesisvoice

Looking at Objective-c examples you often see the Alloc method inside an init method. This is how Apple recommend it be done. However, LCB does not allow nested calls like this. So two foreign handlers have to be written and used. The first calls the +alloc method which is a Class method (+). the second calls the -init method which is an object instance method (-). Have a close look at how I create an Eventkit eventstore object in my LCB script :

Code: Select all

-- bind the memory allocation handler - ok
private foreign handler ObjC_EKEventStoreAlloc() \
			returns optional ObjcRetainedID \
			binds to "objc:EventKit>EKEventStore.+alloc"

-- bind the initialisation of new object - ok
private foreign handler ObjC_EKEventStoreInit(in pObj as ObjcRetainedID,in EKEntityMaskEvent as CUInt) \
			returns optional ObjcID \
			binds to "objc:EventKit>EKEventStore.-initWithAccessToEntityTypes:"



Next I set up a native variable to store the reference to the Objective-C object:

Code: Select all

-- declare static variable to store the long lived EventStore object
private variable sEventStore as optional ObjcObject
These are called later in the LCB script (is it script or code?) with :

Code: Select all

private handler InitialiseEventStore() returns ObjcObject
	variable tEventStore as optional ObjcObject
	unsafe
		put ObjC_EKEventStoreAlloc() into tEventStore
		return ObjC_EKEventStoreInit(tEventStore, 0)
	end unsafe
end handler
The result of this handler gets stored in the variable sEventStore for later use.

I'll try and look at your code snip in detail later today.
best wishes
Skids

Simon Knight
Posts: 539
Joined: Wed Nov 04, 2009 11:41 am
Location: Gunthorpe, North Lincs, UK

Re: Text-to-speech TTS for iOS

Post by Simon Knight » Thu Feb 13, 2020 3:44 pm

Oh one hot tip from Trevor was to look for the header files of the framework on github. This was the only way to discover that the EKEntityMaskEvent was in fact an unsigned integer with a value of zero.
best wishes
Skids

Simon Knight
Posts: 539
Joined: Wed Nov 04, 2009 11:41 am
Location: Gunthorpe, North Lincs, UK

Re: Text-to-speech TTS for iOS

Post by Simon Knight » Thu Feb 13, 2020 3:57 pm

Probably worth expanding on the foreign handler shown above:

Code: Select all

-- bind the initialisation of new object - ok

private foreign handler ObjC_EKEventStoreInit(in pObj as ObjcRetainedID,in EKEntityMaskEvent as CUInt) \
			returns optional ObjcID \
			binds to "objc:EventKit>EKEventStore.-initWithAccessToEntityTypes:"
The first parameter pObj is a reference to the object that the method call is being sent to. We know that the call is going to an instance of the EKEventStore Class because the method name starts with a minus sign. The colon signifies that a parameter is also being sent. The parameter sent is the second in the list of our foreign handler i.e. EKEntityMaskEvent in this example.

The next example shows multiple parameters being sent to an objects method:

Code: Select all

private foreign handler ObjcPredicateForEventsBetweenTwoDates(pObj as objcRetainedID, in pStart as objcID, pEnd as objcID, in pCalArray as objcID) \
			returns optional objcID \
			binds to "objc:EventKit>EKEventStore.-predicateForEventsWithStartDate:endDate:calendars:"
Note that the full method name is "predicateForEventsWithStartDate:endDate:calendars:" colons included. The colons indicate points where variables are inserted and believe it or not are suppose to make Objective-C code self documenting.
best wishes
Skids

trevix
Posts: 480
Joined: Sat Feb 24, 2007 11:25 pm
Location: Italy
Contact:

Re: Text-to-speech TTS for iOS

Post by trevix » Tue Feb 18, 2020 11:10 am

Hello.
Trying to implement the choice of voice for this iOS TTS, I am still struggling understanding how to go from Objc examples (found on the net) to LCB implementation.

Like here: https://useyourloaf.com/blog/synthesize ... from-text/
where I read:
The following code snippet would, for example, set the utterance to use an Australian voice:
utterance.voice = [AVSpeechSynthesisVoice voiceWithLanguage:@"en-AU"];
So, I just have to find out how to add the above to the following Utterance Init block?

Code: Select all

private foreign handler ObjC_AVSpeechUtteranceInitWithString(in pObj as ObjcID, in pText as objcID) \
      	returns optional ObjcRetainedID \
      	binds to "objc:AVFoundation>AVSpeechUtterance.-initWithString:"
Or first I have to do a alloc/init initialisation of "AVSpeechSynthesisVoice"?

Code: Select all

private foreign handler ObjC_AVSpeechSynthesisVoiceAlloc() \
    returns optional ObjcID \
    binds to "objc:AVFoundation>AVSpeechSynthesisVoice.+alloc"

private foreign handler ObjC_AVSpeechSynthesisVoice(in pObj as ObjcID) \
    returns optional ObjcRetainedID \
   binds to "objc:AVFoundation>AVSpeechSynthesisVoice.-init"
But after that?
Trevix
OSX 10.14.6 LC 9.6.0 Dp2 iOs 9.3>

Simon Knight
Posts: 539
Joined: Wed Nov 04, 2009 11:41 am
Location: Gunthorpe, North Lincs, UK

Re: Text-to-speech TTS for iOS

Post by Simon Knight » Tue Feb 18, 2020 12:18 pm

Hi,

Before using the init with string method you should call the alloc method of the class.

Code: Select all

-- bind the memory allocation handler - ok
private foreign handler ObjC_AVSpeechSynthAlloc() \
			returns optional ObjcID \
			binds to "objc:AVFoundation>AVSpeechSynthesizer.+alloc"
When you call this as a function call you store the returned ObjcID in a LCB variable that you have declared as a objcObject tyoe :

Code: Select all

variable tSpeechSynth as objcObject
Next you pass this as the ObjID to your initwithvoice handler which returns a pointer to an objc Objectthat you store in an LCB variable. This variable can and probably be the same one as you used for the alloc call.

Next you can use your variable as an AVSpeechSynthesizer object in calls to methods of AVSpeechSynthesizer . Of course you have to write a foreign object handler for each method that you wish to use.

Several other issues present themselves. The first is that most of the method parameters are passed as objects. This is because ObjectiveC is all about objects. The exceptions are that some parameters may be passed in as integers. So if you with to pass an integer you may declare a variable of type Cint or any of these :
There are the standard C primitive types (defined in the foreign module)
• CBool maps to 'bool'
• CChar, CSChar and CUChar map to 'char', 'signed char' and 'unsigned char'
• CShort/CSShort and CUShort map to 'signed short' and 'unsigned short'
• CInt/CSInt and CUInt map to 'signed int' and 'unsigned int'
• CLong/CSLong and CULong map to 'signed long' and 'unsigned long'
• CLongLong/CSLongLong and CULongLong map to 'signed long long' and 'unsigned long long'
• CFloat maps to 'float'
• CDouble maps to 'double'
There are types specific to Obj-C types (defined in the objc module):
• ObjcObject wraps an obj-c 'id', i.e. a pointer to an objective-c object
• ObjcId maps to 'id'
• ObjcRetainedId maps to 'id', and should be used where a foreign handler argument expects a +1 reference count, or where a foreign handler returns an id with a +1 reference count.
You should note that there is no such thing as a string in C, the nearest the language gets is a character. However, Objective-C uses a NSstring object as its way of working with strings, and these objects are often used as parameters in Objective-C methods. Thankfully LCB provides two methods for converting native strings to foreign NSstring objects. These are described in the dictionary.

Another issue is that methods often return objects or arrays of objects. These objects have to be "unpicked" using yet more foreign handlers to eventually arrive at variables that LCB is able to deal with i.e. NSstring, Cints etc.

So looking at your code snip above the parameter ptext most likely should be a NSstring object when its passed. See https://developer.apple.com/documentati ... guage=objc

So in summary you only need to call one form of -init, but you need to be careful how parameters are typed. One last thing is that the initwithvoice method is only available with the NsSpeechSynthesizer object and I don't believe that this object works on iOS. A quick look of the documentation indicates that on iOS the voice can be passed as a property of the utterance to a AVSpeechSynthesizer object. The snag is that it is as an object, https://developer.apple.com/documentati ... guage=objc so more research required.

I hope that the above is of some help.
best wishes
Skids

Simon Knight
Posts: 539
Joined: Wed Nov 04, 2009 11:41 am
Location: Gunthorpe, North Lincs, UK

Re: Text-to-speech TTS for iOS

Post by Simon Knight » Tue Feb 18, 2020 12:32 pm

ooops I misread your post and my own code....

So with this code :

Code: Select all

unsafe

		-- convert the string passed into a NSString object
		put StringToNSString(pText) into tTextToSpeak

		-- create instance of SpeechSynth
		put ObjC_AVSpeechSynthAlloc() into tSpeechSynth
		put ObjC_AVSpeechSynthInit(tSpeechSynth) into tSpeechSynth

		-- create instance of Utterance
		put ObjC_AVSpeechUtteranceAlloc() into tUtterance
		put ObjC_AVSpeechUtteranceInitWithString(tUtterance,tTextToSpeak) into tUtterance

		-- Speak the Utterance
		ObjC_SpeakUtterance(tSpeechSynth,tUtterance)

	end unsafe
This creates an AVSpeechSynthesizer object and stores in variable, then a AVSpeech Utterance object which is stored in another variable. Lastly the utterance object is passed to the speechSynthesazer object.

So you want to specify the voice to use. The Speech Utterance object has a voice property which is an AVSpeechSynthesisVoice object. So this needs creating before the utterance and a number of foreign handlers will have to be written to create the object and then assign the object to the Utterance object before it is passed to AVSpeechSynthesizer object.

How? No idea at present but I will have a think about it.
best wishes
Skids

Simon Knight
Posts: 539
Joined: Wed Nov 04, 2009 11:41 am
Location: Gunthorpe, North Lincs, UK

Re: Text-to-speech TTS for iOS

Post by Simon Knight » Tue Feb 18, 2020 12:41 pm

For clarity here is the complete working yet unfinished library that I started :
library net.anvic.skids.speechsynth

use com.livecode.foreign
use com.livecode.objc

metadata title is "Speech Synth Library"
metadata author is "Simon (Skids) Knight"
metadata version is "1.0.0"



-- bind the memory allocation handler - ok
private foreign handler ObjC_AVSpeechSynthAlloc() \
returns optional ObjcID \
binds to "objc:AVFoundation>AVSpeechSynthesizer.+alloc"

-- bind the initialisation of new object - ok
private foreign handler ObjC_AVSpeechSynthInit(in pObj as ObjcID) \
returns optional ObjcRetainedID \
binds to "objc:AVFoundation>AVSpeechSynthesizer.-init"

------

-- Now create an 'Utterance' to be passed to the AVSpeechSynthesizer
private foreign handler ObjC_AVSpeechUtteranceAlloc() \
returns optional ObjcID \
binds to "objc:AVFoundation>AVSpeechUtterance.+alloc"

private foreign handler ObjC_AVSpeechUtteranceInitWithString(in pObj as ObjcID, in pText as objcID) \
returns optional ObjcRetainedID \
binds to "objc:AVFoundation>AVSpeechUtterance.-initWithString:"

-- Now the method to send the utterance to the AVSpeechSynthesizer

private foreign handler ObjC_SpeakUtterance(in pObj as ObjcID, in pUtterance as ObjcID) \
returns nothing \
binds to "objc:AVFoundation>AVSpeechSynthesizer.-speakUtterance:"
/*
-- Need to be able to create an NSString object
private foreign handler ObjC_NSStringAlloc() \
returns objcRetainedID \
binds to "objc:NSString.+alloc"

private foreign handler ObjC_NSStringInitWithString(in pObj as ObjcID, in pString as String) \
returns objcID \
binds to "objc:NSString.-initWithString:"
*/
public handler smkSpeakText(in pText as String) returns String
variable tSpeechSynth as objcObject
variable tUtterance as objcObject
variable tTextToSpeak as objcObject

unsafe

-- convert the string passed into a NSString object
put StringToNSString(pText) into tTextToSpeak

-- create instance of SpeechSynth
put ObjC_AVSpeechSynthAlloc() into tSpeechSynth
put ObjC_AVSpeechSynthInit(tSpeechSynth) into tSpeechSynth

-- create instance of Utterance
put ObjC_AVSpeechUtteranceAlloc() into tUtterance
put ObjC_AVSpeechUtteranceInitWithString(tUtterance,tTextToSpeak) into tUtterance

-- Speak the Utterance
ObjC_SpeakUtterance(tSpeechSynth,tUtterance)

end unsafe
return "Complete"
end handler
end library
best wishes
Skids

Simon Knight
Posts: 539
Joined: Wed Nov 04, 2009 11:41 am
Location: Gunthorpe, North Lincs, UK

Re: Text-to-speech TTS for iOS

Post by Simon Knight » Tue Feb 18, 2020 12:57 pm

So the first step is revealed here : https://developer.apple.com/documentati ... jc#1668668
To select a voice for use in speech, obtain an AVSpeechSynthesisVoice instance using one of the methods in Finding Voices, and then set it as the value of the voice property on an AVSpeechUtterance instance containing text to be spoken.

The "Finding Voices" link points back to the AVspeechSynthesizer class and informs us that the class provide a list of voice objects. This means that with the right foreign handler we can just ask the class and it will return a list of AVSpeechSynthesisVoice objects.

Once we get these the code will have to extract the names and return these to LCS so that the user may choose a name.

Then the chosen name will be passed back with the text to speak, the AVSpeechSynthesisVoice object extracted and passed as a parameter to the utterance object along with the text to speak.

Well its a plan....
best wishes
Skids

Simon Knight
Posts: 539
Joined: Wed Nov 04, 2009 11:41 am
Location: Gunthorpe, North Lincs, UK

Re: Text-to-speech TTS for iOS

Post by Simon Knight » Tue Feb 18, 2020 8:03 pm

Here is some progress using the following handlers:

code:

Code: Select all

-- return a list of voices in an array of AVSpeechSynthesisVoice objects
private foreign handler ObjC_AVSpeechSynthesisVoices() \
		returns ObjcID \
		binds to "objc:AVFoundation>AVSpeechSynthesisVoice.+speechVoices"

private foreign handler Objc_NameOfVoiceNSstring (in pObj as objcID) \
			returns ObjcID  \
			binds to "objc:AVFoundation>AVSpeechSynthesisVoice.name"

public handler smkListOfVoices() returns String
	variable tArrayOfVoices as objcObject  // array of AVSpeechSythesisVoice Objects
	variable tVoices as String
	unsafe
		put ObjC_AVSpeechSynthesisVoices() into tArrayOfVoices
		put AV_ReadVoices(tArrayOfVoices) into tVoices
	end unsafe

	return tVoices

end handler

private handler AV_ReadVoices(in pNSArrayOfVoices as objcID) returns String
	variable tVoicesList as List  -- native List LCB type
	variable tVoiceObj as objcObject -- AVSpeechSythesisVoice object
	variable tNameObj as objcObject -- NSString
	variable tFoundVoices as String -- native string that will be returned

	if pNSArrayOfVoices is not nothing then
		put listFromNSArray(pNSArrayOfVoices) into tVoicesList
	else
		return "Empty Array Passed into AV_ReadVoices"
	end if

	-- Loop through the voice objects in the list reading the name
	repeat for each element tVoiceObj in tVoicesList
		unsafe
			if Objc_NameOfVoiceNSstring(tVoiceObj) is not nothing then
				put Objc_NameOfVoiceNSstring(tVoiceObj) into tNameObj
				put StringFromNSString(tNameObj) & ";" after tFoundVoices
			end if
		end unsafe
	end repeat

	-- tidy up return string
	if tFoundVoices ends with ";" then
		delete the last char of tFoundVoices
	end if

	return tFoundVoices
end handler
A call to smkListOfVoices() returns the following on my computer:
Alex;Alice;Alva;Amelie;Anna;Carmit;Damayanti;Daniel;Diego;Ellen;Fiona;Fred;Ioana;Joana;Jorge;Juan;Kanya;Karen;Kyoko;Laura;Lekha;Luca;Luciana;Maged;Mariska;Mei-Jia;Melina;Milena;Moira;Monica;Nora;Paulina;Rishi;Samantha;Sara;Satu;Sin-ji;Tessa;Thomas;Ting-Ting;Veena;Victoria;Xander;Yelda;Yuna;Yuri;Zosia;Zuzana
The code follows the rough design I outlined above. The first foreign handler calls the class method as indicated by the "+" character prefix of the method name. This returns an NSArray of Voice objects.

The handler AV_ReadVoices(in pNSArrayOfVoices as objcID) returns String does the work first converting the NSArray of voice objects passed to it into a native list. It then loops through the list calling the second foreign handler to read the name which is returned as an NSstring object.

Fortunately LCB is able to convert to and from NSstrings and the result of each conversion is added to the variable tFoundVoices which gets returned to the LCS caller.

Traps for young and not so young players include mistakes in the syntax of the foreign handler binding string. a "+" or a "-" indicates that a method is being called. A ":" indicates that a parameter is being used. No colon means no parameter.
best wishes
Skids

Simon Knight
Posts: 539
Joined: Wed Nov 04, 2009 11:41 am
Location: Gunthorpe, North Lincs, UK

Re: Text-to-speech TTS for iOS

Post by Simon Knight » Tue Feb 18, 2020 8:09 pm

The next step is to obtain a voice based on a name passed in from Livecode Script (LCS).

This will use handlers very much like those just created. It would be useful to know if the array of voices is destroyed once the LCB handler that uses it completes. If, as I suspect it is then the LCB variable declaration should be moved outside the handlers to make it a static. This means that the next call from LCS to speak this text with this voice will be able to access the array of voices.

to be continued..........
best wishes
Skids

Simon Knight
Posts: 539
Joined: Wed Nov 04, 2009 11:41 am
Location: Gunthorpe, North Lincs, UK

Re: Text-to-speech TTS for iOS

Post by Simon Knight » Tue Feb 18, 2020 10:03 pm

After some hours of head scratching here is an update to the library:

Code: Select all

library net.anvic.skids.speechsynth

use com.livecode.foreign
use com.livecode.objc

metadata title is "Speech Synth Library"
metadata author is "Simon (Skids) Knight"
metadata version is "1.0.0"



-- bind the memory allocation handler - ok
private foreign handler ObjC_AVSpeechSynthAlloc() \
			returns optional ObjcID \
			binds to "objc:AVFoundation>AVSpeechSynthesizer.+alloc"

-- bind the initialisation of new object - ok
private foreign handler ObjC_AVSpeechSynthInit(in pObj as ObjcID) \
			returns optional ObjcRetainedID \
			binds to "objc:AVFoundation>AVSpeechSynthesizer.-init"



-- Now create an 'Utterance' to be passed to the AVSpeechSynthesizer
private foreign handler ObjC_AVSpeechUtteranceAlloc() \
			returns optional ObjcID \
			binds to "objc:AVFoundation>AVSpeechUtterance.+alloc"

private foreign handler ObjC_AVSpeechUtteranceInitWithString(in pObj as ObjcID, in pText as objcID) \
      	returns optional ObjcRetainedID \
      	binds to "objc:AVFoundation>AVSpeechUtterance.-initWithString:"

private foreign handler ObjC_AVSpeechUtteranceSetVoice(in pUtteranceObj as ObjcID, in pVoiceObject as objcID) \
      	returns optional ObjcRetainedID \
      	binds to "objc:AVFoundation>AVSpeechUtterance.setVoice:"

-- Now the method to send the utterance to the AVSpeechSynthesizer

private foreign handler ObjC_SpeakUtterance(in pObj as ObjcID, in pUtterance as ObjcID) \
			returns nothing \
			binds to "objc:AVFoundation>AVSpeechSynthesizer.-speakUtterance:"
--
-- return a list of voices in an array of AVSpeechSynthesisVoice objects
private foreign handler ObjC_AVSpeechSynthesisVoices() \
		returns ObjcID \
		binds to "objc:AVFoundation>AVSpeechSynthesisVoice.+speechVoices"

private foreign handler Objc_NameOfVoiceNSstring (in pObj as objcID) \
			returns ObjcID  \
			binds to "objc:AVFoundation>AVSpeechSynthesisVoice.name"


variable sVoicesList as List  -- of objcObjects that should stay available between calls

public handler smkListOfVoices() returns String
	// Note this should always be called even if LCS knows the name of a voice
	// it wants to use.
	variable tArrayOfVoices as objcObject  // array of AVSpeechSythesisVoice Objects
	variable tVoices as String
	unsafe
		put ObjC_AVSpeechSynthesisVoices() into tArrayOfVoices
		put AV_ReadVoices(tArrayOfVoices) into tVoices
	end unsafe

	return tVoices

end handler

private handler AV_ReadVoices(in pNSArrayOfVoices as objcID) returns String
	variable tVoicesList as List  -- native List LCB type
	variable tVoiceObj as objcObject -- AVSpeechSythesisVoice object
	variable tNameObj as objcObject -- NSString
	variable tFoundVoices as String -- native string that will be returned

	if pNSArrayOfVoices is not nothing then
		put listFromNSArray(pNSArrayOfVoices) into sVoicesList
	else
		return "Empty Array Passed into AV_ReadVoices"
	end if

	-- Loop through the voice objects in the list reading the name
	repeat for each element tVoiceObj in sVoicesList
		unsafe
			if Objc_NameOfVoiceNSstring(tVoiceObj) is not nothing then
				put Objc_NameOfVoiceNSstring(tVoiceObj) into tNameObj
				put StringFromNSString(tNameObj) & ";" after tFoundVoices
			end if
		end unsafe
	end repeat

	-- tidy up return string
	if tFoundVoices ends with ";" then
		delete the last char of tFoundVoices
	end if

	return tFoundVoices
end handler






public handler smkSpeakText(in pText as String, in pVoiceName as String) returns String
	variable tSpeechSynth as objcObject
	variable tUtterance as objcObject
	variable tTextToSpeak as objcObject

	variable tVoiceObj as objcObject

	-- call function to look up and return voice object
	put GetVoiceObjectNamed(pVoiceName) into tVoiceObj
	if tVoiceObj is nothing then
		return " error getting voice object to use"
	end if

	unsafe

		-- convert the string passed into a NSString object
		put StringToNSString(pText) into tTextToSpeak

		-- create instance of SpeechSynth
		put ObjC_AVSpeechSynthAlloc() into tSpeechSynth
		put ObjC_AVSpeechSynthInit(tSpeechSynth) into tSpeechSynth

		-- create instance of Utterance
		put ObjC_AVSpeechUtteranceAlloc() into tUtterance
		put ObjC_AVSpeechUtteranceInitWithString(tUtterance,tTextToSpeak) into tUtterance

		-- new code to set voice
		ObjC_AVSpeechUtteranceSetVoice(tUtterance, tVoiceObj)

		-- Lastly Speak the Utterance
		ObjC_SpeakUtterance(tSpeechSynth,tUtterance)

	end unsafe
	return "Complete"
end handler

private handler GetVoiceObjectNamed(in pVoiceName as String) returns objcObject
	variable tVoiceObj as ObjcObject  -- this will be returned
	variable tNameObj as ObjcObject -- An NSstring object
	variable tVoiceName as String

	-- Loop through the voice objects in the list reading the name
	repeat for each element tVoiceObj in sVoicesList
		unsafe
			if Objc_NameOfVoiceNSstring(tVoiceObj) is not nothing then
				put Objc_NameOfVoiceNSstring(tVoiceObj) into tNameObj
				put StringFromNSString(tNameObj) into tVoiceName

				if tVoiceName is pVoiceName then
					-- correct object found so stop repeating
					exit repeat
				end if
			else
				-- an error has occured
				return nothing
			end if
		end unsafe
	end repeat
	-- this will either return the correct voice or the last voice in list
	return tVoiceObj
end handler

end library
The following additions have been made:
1) The declaration of the list of voice objects has been moved to make it a static (line 53).
2) The handler smkSpeakText has been modified to allow the name of the required voice to be passed into it (line 104).
3) Lines 112 calls a new handler to find and return the voice object with the required name.
4) Line 134 calls a new foreign handler that sets the voice property of the Utterance object.
5) Lines 140-164 are the new handler called from line 112 - this loops the list seeking a voice object of the correct name.
6) Lines 33 to 35 define the new foreign handler:

Code: Select all

private foreign handler ObjC_AVSpeechUtteranceSetVoice(in pUtteranceObj as ObjcID, in pVoiceObject as objcID) \
      	returns optional ObjcRetainedID \
      	binds to "objc:AVFoundation>AVSpeechUtterance.setVoice:"
The learning point for me as a non-objectiveC programmer is that the documentation lists details of the voice property and that to assign a value to the property the property name has to be capitalised and prefixed with "set". This to set the voice the code uses "setVoice".

Note that this code works but is far from complete. For example there is limited error handling so the name must exist. Also the handler smkListOfVoices() has to be called once to build the list of voice objects before the second handler is used e.g.

From Livecode Script do the following

Code: Select all

put smkListOfVoices() into tListOfNames
put put smkSpeakText("Hello world my name is Simon", "Anna")
I hope what I have written makes some sense and shows how I worked out how to access the various methods.
best wishes
Skids

PaulDaMacMan
Posts: 224
Joined: Wed Apr 24, 2013 4:53 pm
Contact:

Re: Text-to-speech TTS for iOS

Post by PaulDaMacMan » Tue Feb 18, 2020 10:05 pm

Simon Knight wrote:
Tue Feb 18, 2020 8:09 pm
If, as I suspect it is then the LCB variable declaration should be moved outside the handlers to make it a static. This means that the next call from LCS to speak this text with this voice will be able to access the array of voices.
Correct, if you define the variables within your handler then they are local variables only accessible to that handler they're in. If you define the variables outside of a handler then they're global scope and accessible to all of your handlers (but not "static" like a literal/constant, they can still be modified). There's also some other situations with certain control structures having to do with the scope of variables defined within them. I can't remember if it was if/then, safe/unsafe, or some other structure, I just remember I've hit that problem before. Also, with Objective C FFI and creating global objects there is some situations where you want 'ObjCRetainedId' instead of the regular 'ObjCId'. This is somewhat of a mystery to me, it has to do with "reference counting", auto vs manual memory management, and creating instances of objects. I usually will try the regular 'ObjCId' way and if it crashes the LC engine, maybe because the object suddenly doesn't exist anymore, then I see if ObjCRetainedId helps (trial & error).
https://github.com/PaulMcClernan/

Post Reply

Return to “LiveCode Builder”