To build an Intelligent Azure Voice Bot, we need to install some libraries in python and also we need to get the key from azure portal and use it in our code.
Azure Speech Services Integration
The bot starts by integrating Azure Speech SDK, which converts spoken language into text
In this we need to captures user speech and translates it into text, which is then passed to the natural language processing (NLP) engine
Language Understanding (LUIS) for Intent Detection
After speech recognition, the bot sends the transcribed text to LUIS (Language Understanding Intelligent Service) to identify the user’s intent.
This function extracts the top-scoring intent and entities from the user query, which is crucial for understanding what the user wants to achieve
Azure Bot Framework for Dialogue Management
The next step is handling the conversation flow using the Azure Bot Service. The bot connects to Azure Bot Framework to manage conversation states and provide responses based on the detected intent.
Key Technologies Used:
- Azure Speech Services: Speech-to-text conversion for user input
- LUIS: Natural language understanding for identifying intents and entities.
- Azure Bot Service: Manages conversation flow and interactions.
Source Code –
import azure.cognitiveservices.speech as speechsdk
from azure.core.credentials import AzureKeyCredential
from azure.ai.language.questionanswering import QuestionAnsweringClient
AZURE_SPEECH_KEY = ’52b8a7964e3b4a42ba4aefac4549e991′
AZURE_SPEECH_REGION = ‘eastus’
AZURE_QUESTIONANSWERING_ENDPOINT = ‘https://languageserv101.cognitiveservices.azure.com/’
AZURE_QUESTIONANSWERING_KEY = ‘bf0d258fd39748859cba4969baa007db’
client = QuestionAnsweringClient(AZURE_QUESTIONANSWERING_ENDPOINT,AzureKeyCredential(AZURE_QUESTIONANSWERING_KEY))
speech_config = speechsdk.SpeechConfig(subscription=AZURE_SPEECH_KEY,region=AZURE_SPEECH_REGION)
speech_config.speech_recognition_language = ‘en-us’
speech_config.speech_synthesis_voice_name = ‘en-US-JaneNeural’
audio_config = speechsdk.audio.AudioConfig(use_default_microphone=True)
speech_recognizer = speechsdk.SpeechRecognizer(speech_config=speech_config,audio_config=audio_config)
#print(“Speak now, I am listening…”)
#speech_recognition_result = speech_recognizer.recognize_once_async().get()
#result = speech_recognition_result.text
#print(“result———-“,result)
def ask_openai(prompt):
print(prompt)
#client = QuestionAnsweringClient(endpoint,AzureKeyCredential(key))
output = client.get_answers(
question = prompt,
project_name=”Wav2Lip”,
deployment_name=”production”
)
for candidte in output.answers:
synthesize_result = candidte.answer
print(candidte.answer)
#print(“({}) {}”.format(candidte.confidence, candidte.answer))
#print(“Source: {}”.format(candidte.source))
audio_config = speechsdk.audio.AudioOutputConfig(use_default_speaker=True)
speech_synthesizer = speechsdk.SpeechSynthesizer(speech_config=speech_config,audio_config=audio_config)
speech_syntesize_result = speech_synthesizer.speak_text_async(synthesize_result).get()
def chat_with_open_ai():
while True:
print(“Azure OpenAI is listening. Say ‘Stop’ or press Ctrl-Z to end the conversation.”)
try:
# Get audio from the microphone and then send it to the TTS service.
speech_recognition_result = speech_recognizer.recognize_once_async().get()
# If speech is recognized, send it to Azure OpenAI and listen for the response.
if speech_recognition_result.reason == speechsdk.ResultReason.RecognizedSpeech:
if speech_recognition_result.text == “Stop.”:
print(“Conversation ended.”)
break
print(“Recognized speech: {}”.format(speech_recognition_result.text))
ask_openai(speech_recognition_result.text)
elif speech_recognition_result.reason == speechsdk.ResultReason.NoMatch:
print(“No speech could be recognized: {}”.format(speech_recognition_result.no_match_details))
break
elif speech_recognition_result.reason == speechsdk.ResultReason.Canceled:
cancellation_details = speech_recognition_result.cancellation_details
print(“Speech Recognition canceled: {}”.format(cancellation_details.reason))
if cancellation_details.reason == speechsdk.CancellationReason.Error:
print(“Error details: {}”.format(cancellation_details.error_details))
except EOFError:
break
# Main
try:
chat_with_open_ai()
except Exception as err:
print(“Encountered exception. {}”.format(err))
If you have any doubts Please feel free to ping here or in the YouTube Comment Box.