Bringing Amazon Lex into your Amazon Connect flows

In this blog we’ll continue our discussion around Amazon Lex. Talk about a few things to keep in mind when integrating your Amazon Lex bot with your Amazon Connect flow. In my particular use case I wanted to use Amazon Lex to look at my Gmail calendar and book a meeting if I’m available. If you want to skip to the very end you can see the end result via video. You’ll see one video of the voice interaction and one of the Facebook Messenger interaction.

First, you might want to reference my previous post around Lex validation. Now let’s talk about our use case:

  • Lex easily allows you to build a bot which understand both voice and text, so our bot needs to handle calls into our call center as well as Facebook Messenger interactions.
  • Bot needs to to ask a few question in order to find out what time the user would like to meet.
  • Bot should only schedule calls between Monday-Friday and 10 AM – 4 PM Easter Time
  • Bot (using Lambda) should schedule a meeting and if slot already taken then suggest an alternate time to meet.

Second, let’s take a quick look at the Lex screen. The bot I created is very simple and it follows closely the Flowers example provided by Amazon. These are the slots I’m requiring my bot to confirm.

image

I used two different Lambda functions. One for validation and one for fulfillment. While most examples seem to focus on using the same function for both, for me it was easier to have different code bases for each with the added benefit of keeping the code manageable. As it is both validation and fulfillment both came in at around 250 lines of code, but fulfillment had around 9 megabytes of dependencies.

image

Finally, here are sample utterances I used for the main intent.

image

What this gets us is the following. The first video is the voice interaction. I went about it the long way to show some of the validation rules being set by the bot, such as no weekend meetings and no meetings too early in the day. At the end of the video you see I refresh the Gmail calendar to show the new appointment has been saved.

In the second video I go through the same Lex bot using Facebook Messenger and then show the calendar to prove that the appointment was saved.

Ultimately, Amazon makes it extremely easy to create a mutli channel bot, however the integration to back end systems is the tricky part. This bot needs a lot of tuning to make it more natural, but for just a few hours of work there’s very little out there that can get your call center to have some bot integration for self service.

~david

Creating a Lambda Function to Validate Lex Input

In this blog we’re going to step a bit away from Amazon Connect and focus on building a conversational interface using Amazon Lex. As you can probably guess down the line, this interface/bot is going to be connected to Amazon Connect for even more contact center goodness. Here we’re going to focus on creating a Lambda function strictly for validation, not for fulfillment.

First, let’s talk about what I’m building. I’m building a bot which can schedule a time to have a call with me. You tell your intention to the bot “schedule a meeting/call” and the bot will then ask you a few questions using directed language to figure out when you want to meet. Once Lex has all the information it needs it goes out to my calendar to figure out if I’m free or busy. Second, the validation code I have is mainly based on one of Amazon’s great blueprint for ordering flowers. I recommend you start with that before trying to write your own from scratch. Finally, read through the code and pay close attention to the comments marked in bold as these were the biggest gotchas as I went through.

A couple of things to keep in mind when building a conversation interface with Amazon Lex and you’re using validation.

– Have a clear scope of the conversation. I’m not a VUI designer by any means, but if you’re planning on going with an open-ended prompt “How may I help you?” you will be working on this for a long time. Instead try to focus on the smallest possible outcome. Ultimately, it is my opinion that no IVR is really NLU and they are all just directed speech apps with a lot more money sunk into them so they can be called NLU IVRs.

– If you’re going to use input validation, every user input will be ran through Lambda. This means that you must account for people saying random things which aren’t related to what your bot does and these random things will be processed through the validation function and might generate errors. Thus, you need to ignore this input and direct the customer to answer your question, so you can move on.

– Separating validation from fulfillment makes the most sense. Other than making your code easier to read and manage, you’re also able to separate responsibilities and permissions between your two Lambda functions.

– Play around with the examples Amazon provides. They are a great tool to get started and give you a ton of building blocks you can use in your own bot.

Here’s the validation code as well as some notes, hopefully this helps someone else along the way.

'use strict';

// --------------- Helpers to build responses which match the structure of the necessary dialog actions -----------------------

//elicitSlot is in charge of building the request back to Lex and tell Lex what slot needs to be re-filled.
function elicitSlot(sessionAttributes, intentName, slots, slotToElicit, message) {
return {
sessionAttributes,
dialogAction: {
type: 'ElicitSlot',
intentName,
slots,
slotToElicit,
message,
},
};
}

function close(sessionAttributes, fulfillmentState, message) {
return {
sessionAttributes,
dialogAction: {
type: 'Close',
fulfillmentState,
message,
},
};
}

function delegate(sessionAttributes, slots) {
return {
sessionAttributes,
dialogAction: {
type: 'Delegate',
slots,
},
};
}

function confirm(sessionAttributes, intentName, slots){
return{
sessionAttributes,
dialogAction:{
type: 'ConfirmIntent',
intentName,
slots,
message: {
contentType: 'PlainText',
content: 'We are set, do you want to schedule this meeting?'
}
},
};
}
// ---------------- Helper Functions --------------------------------------------------

function isDateWeekday(date) {
const myDate = parseLocalDate(date);
if (myDate.getDay() == 0 || myDate.getDay() == 6) {
console.log("Date is a weekend.");
return false;
} else {
console.log("Date is a weekday.");
return true;
}
}

function parseLocalDate(date) {
/**
* Construct a date object in the local timezone by parsing the input date string, assuming a YYYY-MM-DD format.
* Note that the Date(dateString) constructor is explicitly avoided as it may implicitly assume a UTC timezone.
*/

const dateComponents = date.split(/\-/);
return new Date(dateComponents[0], dateComponents[1] - 1, dateComponents[2]);
}

function isValidDate(date) {
try {
return !(isNaN(parseLocalDate(date).getTime()));
} catch (err) {
return false;
}
}
function buildValidationResult(isValid, violatedSlot, messageContent) {
if (messageContent == null) {
return {
isValid,
violatedSlot,
};
}

return {
isValid,
violatedSlot,
message: { contentType: 'PlainText', content: messageContent },
};
}

function validateMeeting(meetingDate, meetingTime, meetingLength) {

if (meetingDate) {
if (!isValidDate(meetingDate)) {
return buildValidationResult(false, 'MeetingDate', 'That date did not make sense. What date would you like to meet?');
}

if (parseLocalDate(meetingDate) < new Date()) {
return buildValidationResult(false, 'MeetingDate', 'You can only schedule meetings starting the next business day. What day would you like to meet?');
}

if (!isDateWeekday(meetingDate)) {
return buildValidationResult(false, 'MeetingDate', 'You can only schedule meetings during the normal weekday. What day would you like to meet?');
}

if (meetingTime) {
if (meetingTime.length !== 5) {
// Not a valid time; use a prompt defined on the build-time model.
return buildValidationResult(false, 'MeetingTime', null);
}
const hour = parseInt(meetingTime.substring(0, 2), 10);
const minute = parseInt(meetingTime.substring(3), 10);
if (isNaN(hour) || isNaN(minute)) {
//Not a valid time; use a prompt defined on the build-time model.
return buildValidationResult(false, 'MeetingTime', null);
}
if (hour < 10 || hour > 16) {
//Outside of business hours

return buildValidationResult(false, 'MeetingTime', 'Meetings can only be scheduled between 10 AM and 4 PM. Can you specify a time during this range?');

}
}

if(!meetingLength){
return buildValidationResult(false, 'MeetingLength', 'Will this be a short or long meeting?');
}
}
return buildValidationResult(true, null, null);
}

//--------------- Functions that control the bot's behavior -----------------------
function orderFlowers(intentRequest, callback) {
const source = intentRequest.invocationSource;
//get appointment slots
const meetingDate = intentRequest.currentIntent.slots.MeetingDate;
const meetingTime = intentRequest.currentIntent.slots.MeetingTime;
const meetingLength = intentRequest.currentIntent.slots.MeetingLength;

//For fullfilment source will NOT be DialogCodeHook
if (source === 'DialogCodeHook') {
//Perform basic validation on the supplied input slots. Use the elicitSlot dialog action to re-prompt for the first violation detected.
const slots = intentRequest.currentIntent.slots;
const validationResult = validateMeeting(meetingDate, meetingTime, meetingLength);

if (!validationResult.isValid) {
slots[`${validationResult.violatedSlot}`] = null;
callback(elicitSlot(intentRequest.sessionAttributes, intentRequest.currentIntent.name, slots, validationResult.violatedSlot, validationResult.message));
return;
}

//Pass the price of the flowers back through session attributes to be used in various prompts defined on the bot model.
const outputSessionAttributes = intentRequest.sessionAttributes || {};
callback(delegate(outputSessionAttributes, intentRequest.currentIntent.slots));
return;
}
}

// --------------- Intents -----------------------
function dispatch(intentRequest, callback) {
const intentName = intentRequest.currentIntent.name;

//Dispatch to your skill's intent handlers
if (intentName === 'MakeAppointment') {
return orderFlowers(intentRequest, callback);
}
throw new Error(`Intent with name ${intentName} not supported`);
}

//--------------- Main handler -----------------------
//Route the incoming request based on intent.
//The JSON body of the request is provided in the event slot.
//Execution starts here and moves up based on function exports.handler => dispatch =>orderFlowers=>validateMeeting=>buildValidationResult is the most typical path a request will take.

exports.handler = (event, context, callback) => {
try {
dispatch(event, (response) => callback(null, response));
} catch (err) {
callback(err);
}
};

Right way to block ANIs using Amazon Connect

In this blog I’ll cover a potential financial issue you might face if you try to ANI block customers and they are calling you through a SIP trunk.

As I continue my journey of getting familiar with Amazon Connect I ran into an interesting and a bit worrisome issue. The use case I was working on was to create a table which blocks or allows specific ANIs to call in. Ultimately, when a blocked caller came in I wanted to just hang up on the call. My original flow looked like this:

image

Pretty straight forward, invoke Lambda, check attribute and if blocked = true, disconnect the call. When calling from my cellphone this worked great. However, when calling from my home phone (using a Flowroute SIP trunk) I got a nice surprise in the logs:

image

What you’re seeing is a partial log of my home phone constantly retrying to connect to Amazon Connect and generating a new call each time. Since there was no prompt play and no ring back heard I assume the network believes there as a connection issue and continues to try and connect. Which means that you could easily incur a huge expense both on your phone provider and on your Amazon AWS bill.

The way to fix this was to play a 1 second of silence prompt before disconnecting the call.

image

~david

Amazon Connect and Sticky Queue

In this blog I’ll discuss how to achieve a sticky queue using Amazon Connect.

When a customer calls back within a short amount of time, it’s fairly safe to assume they are calling back for the same reason as before. This is often referred as sticky agent or sticky queue. Because you’re trying to “stick” a queue or agent to a specific customer. As a best practice avoid using anything sticky as it could force your customer down the wrong path or create long hold times when too many callers are stuck to a single agent or queue, but for my use case it’s safe to use because I want to use it. :)

I assume you already have a Lambda function or two working with your flow, if you don’t then you might want to skip this functionality until you get that working. The first thing you need to do is find the ARN for your queues. I’m going to be honest, this is not intuitive at all and I wish the Connect team would allow you to retrieve this information via the flow without having to do the following steps.

To get your Queue ARN go to your queues via https://<your instance>.awsapps.com/connect/queues, select a single queue, and notice the bold section of the url https://<your instance>.awsapps.com/connect/queues/edit?id=arn:aws:connect:us-east-1:64:instance/4d92ab25-8XXX-4bXX-aXXX-XXXXXXX/queue/XXXXXX-2022-XXXX-a275-xxXXXXxxxXXX That’s your queue’s ARN.

– Set an attribute (e.g. Queue) with your ARN.
– Set your queue name to the attribute you just set.

image

– Save your attribute to your DB.

Next time your customer calls, you can retrieve the last queue they went through and give them the option to go to that queue again, hopefully saving your customer some frustration.

image

~david

Amazon Connect Flow Designer Review

I’m trying to capture my initial notes and reactions to Amazon’s contact center offering. In this blog I’m going to focus solely on their flow designer tool. I’ll provide a brief overview of the tool, some best practices I’ve come up with, as well as some things I wish were different. Remember that I come from the Cisco contact center world, so my view is slightly tainted and what I’ve lived and loved has been the Cisco tools.

Amazon Connect provides a web based call flow tool called flow designer. Those of you familiar with ICM Script Editor and CVP Studio will feel at home. Below is one of the flows I’ve created. Note that the designer allows you to snap steps or what Amazon calls “action blocks” into the grid for cleaner looking flows.

Flow designer with flow

In the left hand side of the designer is your “palette” you can find an explanation of each action block here.

Flow Designer Palete

Building your first flow is truly easy and requires very little technical knowledge. The Play prompt block allows both playing audio files as well as text-to-speech (TTS) in a variety of voices and languages. Setting a queue and building a queue is just as easy.

Now a few items which bother me about contact flow as well as some best practices I’ve found. I touched on a few of these in my earlier post.

  • DO NOT hit the back button or navigate away from the flow designer without saving. There is no auto save!
  • You can’t copy and paste a block. You must build a block from scratch every time. I keep a file with Lambda names and variables I’m using for easy copy/paste.
  • You can’t have the block properties of multiple blocks open at the same time.
  • There is no move of multiple blocks. You must move each one at a time.
  • Build your flow strictly with TTS and only add audio files once you’re happy with the product. If you’re using dynamic speech you’ll have a better sense of what the audio files need to say.
  • Plan your error conditions flow early. This is important when handling error/default/timeout from menus, but applies across multiple different types of blocks. You should come up with a few standard error correction flows and branch out all your error conditions appropriately based on where you are in the flow. This will also avoid a spider web flow.
  • No easy way to get from flow to flow. Once you’re in the designer, you click the back button in your browser or go through the main menu to jump to another flow. Ideally Amazon provides a drop down in the designer to switch between flows to save a few clicks.
  • No infinite scroll. Specifically you can’t scroll and build your flow up or to the left. This means that you should think of starting your flows somewhere in the middle of your screen to give you a bit of real estate for last minutes changes/branches. When you create a new flow Amazon “conveniently” starts you off like in the left image, but you should move your fist block off to the right a bit, like right image. Also make sure you immediately enable “Snap to grid” for cleaner looking flows.

image image

  • You are able to move blocks behind the unmovable left hand margin. The only reason I discovered this is because I wanted to add a log block and didn’t want to pile up blocks on top of each other.

Flow designer with nodes hidden.

  • You need to be aware of where your lines are going and try to avoid overlap and tight spaces, specially when using the Get customer input block. Trying to modify a line in the middle of the block can be difficult and will require for you to delete other lines to get to the line you want to delete or modify.

image

  • DO NOT hit the back button or navigate away from the flow designer without saving. There is no auto save! Yes, it’s a repeat, happened to me multiple times.
  • When saving a flow or publishing a flow you get the same confirmation. It would be nice to be reminded what was the last action you took for those of us who are jumping from screen to screen.

Flow designer save message.

~david

Initial Observations of Amazon Connect

If you don’t know what Amazon Connect is and you’re in the contact center world, you might want to consider a new career path. Amazon Connect is AWS’s answer to the Ciscos, Genesys, Avayas of the world. Not only that, but also a competitor for Twilio, Microsoft, and anyone who carries voice from point A to point B. Needless to say, when the Amazon giant moves everyone pays attention. A lot of these are just a brain dump so pardon the brevity. I’m still trying to dig a bit deeper and come up with specific ideas to blog specially comparing Amazon’s solutions with Cisco’s offering(s).

Things which are awesome:

– Agent logs are in JSON format, holy crap that’s awesome!
– Hours of operations are available out of the box and are granular to the minute. Ability to add exceptions for the same day is a nice touch.
– If you associate an email with your agent your agent can reset their own password.

Things which are strange:

– Can’t change agent state while reserved or talking.
– If you use a desk phone, you can’t reject the call.  
– Changes take about a minute or two to propagate and there’s no notification if your changes are live or not.      
– If you create a new agent and then login as that agent using the same browser as before your admin session will be moved over to the new agent credentials. Painful when trying to test permissions on agents.     
– You can’t re-route a connector by clicking on the start point, you must first delete the existing line and then create your new connector.

Things which absolutely make no sense:

– Every step should have a Lambda invocation option. This would make the scripting a lot cleaner.
– If you reject a call and you’re the only agent you’re automatically set back to ready. Queue must be drained before last agent can change states out of available.
– No default routing? I disabled the only queue and calls just dropped when I tried to route to that queue. You would think that the system would force some sort of default routing option just in case you make a mistake.  
– Contact flow editor, no easy way to get back to all your contact flows.
– Agent auto accept takes about 12 seconds to trigger using softphone, this would impact agent stats and I really don’t see the point of having this feature if it’s going to take this long to connect an agent.
– When you save or publish a contact flow you get the same message "Contact flow saved successfully!" Different message for publish would be nice.
– No easy way to move the whole script. Work area should have infinite scroll to all sides.
– You can’t select multiple nodes and move them, you must move one by one.
– Flows don’t auto save drafts, if for some reason you don’t remember to save you’re SOL.
– How draft flows and published flows are handled is confusing. Not very user friendly.
– Checking contact attributes doesn’t offer a NULL or NOT NULL condition check.
– When a connector goes behind a flow node, you can’t delete the connector.
– No way to duplicate nodes. You must configure a new node from scratch every time.   

~david

Cisco Spark Webhooks using Node.js

I’ve been trying to come up with a reason to play around with node.js. I’ve also been trying to make some time to play around with the Cisco Spark API. So I figured I would merry the two of them. My intention was to setup a webhook on a channel, then in my node application display the messages being posted in my room. Pretty simple example, but it touches a few different things.

First, you need to setup your webhook to point to your server.

Second, you need to have node installed, in this case I’m using Ubuntu 16.04.

Third, write some code. Check out my comments to follow along.

//will use these for the POST request

var express = require("express");
var myParser = require("body-parser");
var app = express();

//will use this for the GET request

var http = require(‘https’);

app.use(myParser.json());
app.post("/", function(request, response){

console.log(request.method);

//console the webhook name as well as the name of the author of the message
console.log(‘Webhook:’+request.body.name);
console.log(‘Email:’+request.body.data.personEmail);

//setup your GET request to find out the actual message posted by the author

var options = {
host: ‘api.ciscospark.com’,
path: ‘/v1/messages/’+request.body.data.id,
headers: {
  ‘User-Agent’: ‘request’,
  ‘Authorization’: ‘Bearer MyAuth’
}
}

//make the request

http.request(options, OnResponse).end();
});

//capture the request response

function OnResponse(response){
var data = ”;
response.on(‘data’, function(chunk){
  data += chunk;
});

response.on(‘end’, function(){
  data = JSON.parse(data);

//console out the actual text
  console.log(data.text);
});
}

app.listen(8080);

~david

Microsoft Azure and Laravel 5.1 from Scratch

While most people use AWS, I like to use Azure and there’s just not a lot of information relating Azure and Laravel, so figured this might be useful for others. This assumes a level of comfort with Linux and Azure.

  • Login: https://portal.azure.com/#
  • Click on Virtual machines (classic) and search for Ubuntu. My preference has always been Ubuntu server 14.04 LTS, select this.
  • Select deployment model: Resource Manager and click Create.
  • Give your VM some basic information: Name, User name, Authentication Type (SSH public key is my preference). Create a new resource group, I always use the same name as the VM and Location. I usually do the A1 Standard size.
  • Click ok to choose the defauls under Settings.
  • Click OK on summary and your VM will start deploying, this will take about 10-15 minutes to complete.
  • Once running, click the VM, Public Ip address/DNS name label, All Settings, Configuration, and give it a DNS name. I do this to not have to deal with IP addresses and use actual names.

At this point, I created a CustomScript Extension using this https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-linux-script-lamp/ and http://tutsnare.com/how-to-install-laravel-on-ubuntu-lamp/ and http://tecadmin.net/install-laravel-framework-on-ubuntu/ This will install everything you need. The script is below. To run the script through the CLI you need to use this command: azure vm extension set -c "./public_config.json" <yourresourcegroup> <yourvm> CustomScriptForLinux Microsoft.OSTCExtensions 1.4
If you want to run the script manually, just SSH into your VM and go through every step below starting at "sudo apt-get -y".

  • SSH into your machine.
  • Instal composer: curl -sS https://getcomposer.org/installer | php
  • Move composer globally: sudo mv composer.phar /usr/local/bin/composer
  • Install Laravel: sudo composer create-project laravel/laravel /var/www/laravel 5.1.*
    • sudo mkdir laravel/app/storage
    • sudo chown -R www-data.www-data /var/www/laravel
    • sudo chmod -R 755 /var/www/laravel
    • sudo chmod -R 777 /var/www/laravel/app/storage
  • Create VirtualHost and use sample below: vim /etc/apache2/sites-available/yourhost.com.conf
  • Enable your site config: sudo a2ensite yourhost.com
  • Reload Apache: sudo service apache2 reload
  • Disable default site: sudo a2dissite 0000-default
  • Reload Apache: sudo service apache2 reload
  • Open VM firewall: sudo iptables -A INPUT -p tcp –dport 80 -j ACCEPT
  • Restart VM: sudo shutdown -r now
  • In Azure Portal Network Security Group find "inbound security rules" and add a rule for port 80.
  • Go to your http://yourhost.com Welcome!

Azure LAMP install script:
#!/bin/bash
# set up a silent install of MySQL
dbpass="YourPasswordHere"

export DEBIAN_FRONTEND=noninteractive
echo mysql-server-5.6 mysql-server/root_password password $dbpass | debconf-set-selections
echo mysql-server-5.6 mysql-server/root_password_again password $dbpass | debconf-set-selections

# install the LAMP stack
sudo apt-get -y install apache2 mysql-server php5 php5-mysql php5-curl

#install extensions
#installing json extension
sudo apt-get install php5-json
#installing unzip extension
sudo apt-get install unzip
#installing curl extension
sudo apt-get install curl
#installing openssl extension
sudo apt-get install openssl
#installing mcrypt extension
sudo apt-get install php5-mcrypt
#enable mcrypt extension
sudo php5enmod mcrypt
#installing git
sudo apt-get install -y git-core
#enable mod rewrite extension
sudo a2enmod rewrite

# write some PHP
echo \<center\>\<h1\>My Demo App\</h1\>\<br/\>\</center\> > /var/www/html/phpinfo.php
echo \<\?php phpinfo\(\)\; \?\> >> /var/www/html/phpinfo.php

# restart Apache
sudo apachectl restart

Yourhost.com.conf:
<VirtualHost *:80>
       
        ServerName laravel.example.com
        DocumentRoot /var/www/laravel/public

        <Directory />
                Options FollowSymLinks
                AllowOverride None
        </Directory>
        <Directory /var/www/laravel>
                AllowOverride All
        </Directory>

        ErrorLog ${APACHE_LOG_DIR}/error.log
        LogLevel warn
        CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

For posterity.

This is case sensative.
public_config.json:{"fileUris":["/scripts/Laravel.sh"]’>/scripts/Laravel.sh"]’>/scripts/Laravel.sh"]’>https://<yourstorage>/scripts/Laravel.sh"], "commandToExecute":"sh Laravel.sh" }