How to automate your home with your voice

Using voice to control things in your home is one of the coolest things in home automation. It is more natural to use voice rather than keyboard or a touch screen. Wouldn’t it be fun to use a voice command to turn off the lights or switch on the AC or even open the door? All without leaving the comfort of your chair !

With the availability of cloud based technologies it is not as difficult as it sounds. This project demonstrates the concepts and introduces you to two powerful network based technologies - Amazon Alexa and PubNub.

This blog post was originally published as a project on Hackster.

Things you need

Hardware

• Raspberry Pi 3 model B (model 2B is supported by Amazon but I have not tested it) to run Amazon Alexa app

• BeagleBone Black - to run a GPIO controller and PubNub listener code

• USB Microphone

• Earphones or speakers with 3.5mm jack

• LEDs and resistors (470 Ω) for demonstration

• Make sure you have all required accessories for the Raspberry Pi like USB keyboard, mouse, HDMI monitor and power adapter or micro USB cable.If you are running a headless Pi, then use VNC to log in from your PC/Laptop. SSH won’t be enough as we need the GUI for Alexa app. For the BeagleBone Black, SSH login is enough as we will run the code from command line.

Software

• Alexa-AVS Sample app from GitHub

• Alexa skill kit and web service code from GitHub

• PubNub listener and Beaglebone GPIO contoller code from GitHubhttps://github.com/gopal-amlekar/pubnub-alexa-beagleboneblack.git

Other

• Developer account on Amazon (Needs a valid credit card although the service is free for 12 months)
  • https://developer.amazon.com
• Account with PubNub
  • https://www.pubnub.com
• Account with Heroku (for running a web service)
  • https://signup.heroku.com/dc

Theory of operation

What is Alexa

Alexa is a voice service from Amazon which gives a Voice User Interface (VUI) to customers to interact with the world. You can ask Alexa range of questions from information about weather in your city to the distance between sun and earth. You can also ask Alexa to play music or set an alarm or timer etc. However what makes it more powerful is the ability to develop additional skills using the service. This is the Amazon Skill Kit. The skill kit is a set of API and tools which help you create your own capabilities using voice commands. There are some pre-defined skills like Smart home skill, Flash briefing skill but in this project we will be using a custom skill and write our code from scratch. Using a custom skill is simpler and more flexible. The custom skill does not need any user authentication or account linking as is required by the smart home skill. You can read more about Alexa  voice service and Alexa skills kit here.

What is PubNub

PubNub is a real time data streaming network. It works on the principle of publish-subscribe. Devices publish messages to network using a particular topic and these messages are received by devices who are subscribed to that topic. PubNub acts as a broker in this architecture to receive published messages and pass them on to subscriber devices. The device can be a mobile phone or tablet or Raspberry Pi, Arduino like board and so on. You can read more about real time data streaming networks on this blog by Joe Hanson and find out how PubNub works from here.

In this project, we use PubNub for message passing from Alexa skill adapter code to the BeagleBone responsible for controlling devices in the home.

With this basic knowledge about Alexa and PubNub, the following presentation will help understand the flow and working of this project.

Get it going

Enough theory and introduction, let’s get to the execution phase. Follow the steps below.

1. Create all accounts
2. Get Publish/Subscribe keys from PubNub
3. Get Alexa voice service running on Raspberry Pi
4. Create a custom skill
5. Run skill adapter code as web service
6. Configure skill
7. Run code on BeagleBone
8. Have fun

Create all accounts

If not done already, create all accounts required (Amazon, PubNub and Heroku).

PubNub publish/subscribe keys

Create a new app within PubNub. This will also create a demo keyset within the app. This keyset is a pair of publish and subscribe keys. We need these keys later on so copy these keys somewhere in a text file. The publish key is needed for publishing messages to the PubNub network from the web service. The subscribe key is needed for code running on BeagleBone listening to messages published on a channel. Note that we don’t need to create any channel on PubNub website. Whenever a publish is done, the channel gets automatically created and other devices can subscribe to the channel.

Get Alexa voice service running on Raspberry Pi

The wiki https://github.com/alexa/alexa-avs-sample-app/wiki/Raspberry-Pi contains all instructions in detail with screen shots and hence I am not repeating them here. Following these steps will install the Alexa app on your Raspberry Pi and will also register your Raspberry Pi as an authenticated device to access the Alexa voice service.

A couple of things to note:

1. Usage of wakeword is optional and you can always talk to Alexa by clicking on ‘Listen' button in the sample app.

2. The automated installer takes quite a lot of time especially on a fresh Raspbian install. In my case it took more than 4 hours as it downloaded a number of packages including upgrade Raspbian OS.

Once you are able to access the Alexa voice service, i.e. you got correct answers to questions you asked, you are ready to take next step for developing a skill set.

Create a custom skill

1. Login to https://developer.amazon.com/

2. Click on tab Alexa and then click on Get Started button in Alexa Skills Kit box. Click on Add a New Skill button.

3. Keep Language as English (U.S.) and select Custom Interaction Mode as skill type.

4. Enter a descriptive name in the Name dialog box such as Home automation using PubNub.

5. In the Invocation Name, enter a short name such as My home. This is the name you will use to invoke the skill. For e.g. you will say “Alexa, ask My home to turn on kitchen light”.

6. Set Audio Player option to No.

7. Click on Save button.

8. Copy the application ID created as we are going to need this later. We will return to this point after creating web service on Heroku.

Create a web service

The actions our custom skill executes based on voice commands we speak, are all executed in the code running as web service. The web service receives requests from Alexa voice service in the form of intents and slots. It processes the intents and publishes messages to PubNub network.

You can carry out these steps on your host machine running either Linux / Windows / Mac.

It is easier to work with command line interface when using Heroku. Follow the steps in the setup section here to download the Heroku CLI and verify you have correct versions of node, npm and git installed on your host system.

If not done already, checkout source code from GitHub and navigate to folder alexa-pubnub-heroku.

1
2

git clone https://github.com/gopal-amlekar/alexa-pubnub-heroku.git
cd alexa-pubnub-heroku

Login to heroku.

1
2
3
4
5

heroku login
Enter your Heroku credentials:
Email: test@xyz.com
Password:
Logged in as test@xyz.com

Create a web app with following commands

1
2
3

heroku create
Creating app... done, demo-app-96357
https://demo-app-96357.herokuapp.com/ | https://git.heroku.com/demo-app-96357.git

Heroku creates a web app for you with a random name. Note this name. It also creates one git repository for the app code on Heroku. A remote is added to your local git repository so you can push code changes to the web app via git.
Next, we need to set some configuration parameters such as PubNub publish/subscribe keys, web app route etc.

1
2
3
4
5

heroku config:set AMAZON_APP_KEY=YOUR_AMAZON_SKILL_APPLICATION_ID
heroku config:set WEB_APP_ROUTE=/alexa_web_service
heroku config:set PUB_NUB_SUBSCRIBE_KEY=YOUR_PUBNUB_SUBSCRIBE_KEY
heroku config:set PUB_NUB_PUBLISH_KEY=YOUR_PUBNUB_PUBLISH_KEY
heroku config:set PUB_NUB_CHANNEL_KEY=alexa_world</pre>

The WEB_APP_ROUTE configuration parameter is part of the https endpoint route to which Alexa service will pass on the requests. You can choose any random name for this. In this example, the endpoint becomes
https://demo-app-96357.herokuapp.com/alexa_web_service.

We need this endpoint later to configure the Alexa skill.
Also, you can choose any name for PUB_NUB_CHANNEL_KEY but it should be kept same in the BeagleBone environment as well.

Finally, depoly the Heroku web app which runs our web service.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

git push heroku master
Counting objects: 20, done.
...
Writing objects: 100% (20/20), 6.10 KiB | 0 bytes/s, done.
... 
remote: Building source:
remote: -----> Node.js app detected
remote: -----> Creating runtime environment https://iotality.com/wp-admin/post-new.php#
... 
... 
remote: -----> Build succeeded! 
... 
... 
remote: -----> Launching... 
remote: Released v5 
remote: https://demo-app-96357.herokuapp.com/ deployed to Heroku 
remote: Verifying deploy... done.
To https://git.heroku.com/demo-app-96357.git 
* [new branch] master -> master

Configure the skill

Skill configuration consists of 3 parts:

1. Configuring Voice User Interface
2. Mapping voice commands to intents and slots
3. Configuring Skill adapter endpoints

We need to tell our skill what voice commands are expected from user and how to match the commands to intents and slots. The skill will then process the voice commands and pass on requests to our web service in the form of intents and slots. Slot is a list of additional objects in our voice command. Use of slot is optional but we are going to need this to specify the gadgets to be turned on/off.
The skill therefore needs intent schema which is a schema of the intents along with slots that our skill adapter can process. It also needs a list of sample utterances (i.e. the voice commands that the user will speak). Alexa then tries to match the voice commands spoken to the sample utterances and map them to the intents.

1. Go back to the Alexa skill you created on https://developer.amazon.com and edit it.
2. Click on Next or click on the Interaction model tab.
3. In the Intent schema, copy/paste contents of file intent_schema.json from the the folder skill-interaction in the GitHub project checked out earlier.
4. In the Sample utterances, copy/paste contents of file sample_utterances.txt from the same folder.

Note: The sample utterances and intent schema are in fact not needed for the web app. But I added them to this repository only for convenience.

1. Click on Add slot type button. Enter the type as LIST_OF_ITEMS' and key in following values one on each line: Drawing room Light, Bedroom Light, Garage Light. These are not case-sensitive.
2. Add another slot type and enter type as LIST_OF_DOORS and key in the value garge door there. I used only one value here but you can add more specific to your project.
3. Click on Next button.
4. In the configuration section, select the service endpoint type as HTTPS and geographical region North America.
5. Copy / paste the web service path created above (https://demo-app-96357.herokuapp.com/alexa_web_service) in the text box. Make sure to select the SSL certificate option there as My development endpoint is a sub-domain of a domain that has a wildcard certificate from a certificate authority.
6. Leave Account linking option to No and click on Next.

At this point, you should be able to test your skill via a simulator. In the test section, ensure that your skill has been enabled for testing. If not, something is missing and generally an error message will be shown there. Go back and correct it.

In Enter utterance text box in Service Simulator section, enter Turn on kitchen light. This utterance will be converted by the skill to an intent and passed on to the skill adapter code. Response from the skill adapter code should be shown in the Lambda response box.

If this works fine, proceed to test the skill using voice commands. Start the Alexa app on your Raspberry Pi and say Alexa Tell my home to turn on kitchen light. You should get a voice acknowledgement like Turning on the gadget.

You can additionally test the skill by using PubNub’s debug console as well. Go to Debug console section of your PubNub app on PubNub website. Add a client with the channel name Alexa_World. When you speak into the Alexa app on the Pi, you should see the message from the skill adapter in the client window.

Run code on BeagleBone Black

Let’s now fit together the only remaining piece of the puzzle. We will run code on BeagleBone Black so it subscribes to channel Alexa_world on PubNub and keeps listening to the messages on this channel. Once it receives a message, it validates the message and takes necessary action to turn on /off the corresponding GPIO pin or set PWM etc.

1. A little bit of hardware work is needed. I used LEDs to demonstrate the concept. Wire up three LEDs as shown in the schematic below. Also wire up a servo motor as shown in the schematic.

• Carry out the following steps on BeagleBone. Use SSH from your host machine or Raspberry Pi to log in to BeagleBone.

• Check out the GitHub code. This code is also in Node.js using PubNub SDK. Make sure you have Node.js V4 or greater running on the BeagleBone.

• Make sure you have Node.js V4 or greater running on the BeagleBone. Follow instructions here to update Node.js if required.

• Go to the checked out folder and issue following command to install required modules. The required modules are - PubNub, dotenv and bonescript.

1
2
3

git clone https://github.com/gopal-amlekar/pubnub-alexa-beagleboneblack.git
cd pubnub-alexa-beagleboneblack
npm install

• Create a file named “.env” in the same folder and put in following details in that file. I used this approach instead of sharing the publish / subscribe keys in source code.

1
2

PUB_NUB_CHANNEL_KEY=alexa_world
PUB_NUB_SUBSCRIBE_KEY=YOUR_PUBNUB_SUBSCRIBE_KEY</pre>

Execute the code on beaglebone.

1
2

npm start
Subscribing to channel alexa_world

The app will now start listening to messages on PubNub channel alexa_world.

Now if you speak into the Alexa app Turn on garage light or Turn on bedroom light, you should see corresponding LEDs turning on/off as per your commands.

Demonstration

Do let me know if you replicate this project.

Have fun !