Quick summary: In this tutorial, we are going to program a voice-controlled drone using Node.js and Web speech API. We will use Parrot ARDrone 2.0 as our drone and Node.js + web speech API to build our application.
Introduction
Drones are amazing. I personally love the ability to sit down and fly the drone to either capture pictures/videos or just having fun.
Drones are not just for fun though. Drones are used heavily in the movie production industry, glacier exploration, military, agriculture and many more.
Consumer Drones are getting popular due to ease in regulation by many countries and great quality drones manufactured by DJI and other providers.
In this tutorial, we are going to create a program that will allow us to control and maneuver our drone using voice commands. That’s right, the drone will do as you command.
Before diving into development, do have a look at this one minute video where I am flying the drone in my hotel room (Not safe at all).
Let’s look at the things you require beforehand.
Source code and demo
Hardware
You need the following hardware:
You need to have a standard laptop running Windows/Mac/Ubuntu. I have tested it on Mac and Ubuntu 18.04.
Software
You must have the latest version of Node.js installed in your system. Visit this site to download Node.js.
You need to have the latest version of Google Chrome installed in your system. Visit this site to download chrome.
Understanding the drone
Let’s try to understand the built and structure of the Parrot ARDrone.
The Parrot ARDrone is a quadcopter i.e there are 4 rotating motors attached to each corner.
Each pair of opposite rotors is turning the same way. One pair is turning clockwise and the other anti-clockwise.
The drone movements are obtained by the changing pitch, roll and yaw angles.
As you can observe in the diagram above, by changing the left and right rotors speed the opposite way yields roll movement. This allows the drone to go forth and back.
By changing front and rear rotors speeds the opposite way yields pitch movement.
By changing each rotor pair speed the opposite way yields yaw movement. This allows turning left and right.
You really don’t need to learn aerodynamics to fly the drone, this is just for the basic information about the drone.
How Parrot ARDrone works
Parrot ARDrone provides its own WiFi access point. One needs to connect to it in order to send the command to the drone.
There are various mobile apps which you can use to fly the drone using your mobile. We will also use the WiFi access point to connect to the drone.
Let’s do the basic connection first. Connect the battery in the drone and it will boot the drone system.
Check your WiFi connection. You should an access point naming similar to one shown in the screenshot below.
Connect to the WiFi network. Once connected, open up the terminal and try to do a telnet to 192.168.1.1 – this is the IP address of the drone.
You should see successful telnet into the drone operating system running Linux Busybox.
The App Architecture
Our app codebase is divided into the following section:
- User interface with Web speech API to detect voice.
- Command filtering and matching.
- Sending commands to the drone.
- Live video streaming
The web speech API works with an internet connection. The reason why we are adding ethernet connection because we will connect to the drone WiFi access point and we need internet to detect our voice using web speech API.
Let’s build our app.
Coding the app
Let’s code our app. First, create a new folder and switch to it using the terminal.
mkdir drone && cd drone
Create a new Node project using the following command.
npm init --y
Let’s install the required dependency.
npm install --save express body-parser ar-drone
We are going to support the following commands.
- Takeoff
- Land
- Up – go up about half meter distance then stops.
- Down – go down about half meter distance then stops.
- Go left – go left about half meter distance then stops.
- Go right – go right about half meter distance then stops.
- Turn – turn clockwise 90 degrees.
- Go forward – go forward about a meter distance then stops.
- Go backward – go backward about a meter distance then stops.
- stop
Let’s code our server.
Here is our code to receive the command, filter it and control the drone.
const express = require('express');
const bodyparser = require('body-parser');
const arDrone = require('ar-drone');
const router = express.Router();
const app = express();
const commands = ['takeoff', 'land','up','down','goleft','goright','turn','goforward','gobackward','stop'];
var drone = arDrone.createClient();
// disable emergency
drone.disableEmergency();
// express
app.use(bodyparser.json());
app.use(express.static(__dirname + '/public'));
router.get('/',(req,res) => {
res.sendFile('index.html');
});
router.post('/command',(req,res) => {
console.log('command recieved ', req.body);
console.log('existing commands', commands);
let command = req.body.command.replace(/ /g,'');
if(commands.indexOf(command) !== -1) {
switch(command.toUpperCase()) {
case "TAKEOFF":
console.log('taking off the drone');
drone.takeoff();
break;
case "LAND":
console.log('landing the drone');
drone.land();
break;
case "UP":
console.log('taking the drone up half meter');
drone.up(0.2);
setTimeout(() => {
drone.stop();
clearTimeout();
},2000);
break;
case "DOWN":
console.log('taking the drone down half meter');
drone.down(0.2);
setTimeout(() => {
drone.stop();
clearTimeout();
},2000);
break;
case "GOLEFT":
console.log('taking the drone left 1 meter');
drone.left(0.1);
setTimeout(() => {
drone.stop();
clearTimeout();
},1000);
break;
case "GORIGHT":
console.log('taking the drone right 1 meter');
drone.right(0.1);
setTimeout(() => {
drone.stop();
clearTimeout();
},1000);
break;
case "TURN":
console.log('turning the drone');
drone.clockwise(0.4);
setTimeout(() => {
drone.stop();
clearTimeout();
},2000);
break;
case "GOFORWARD":
console.log('moving the drone forward by 1 meter');
drone.front(0.1);
setTimeout(() => {
drone.stop();
clearTimeout();
},2000);
break;
case "GOBACKWARD":
console.log('moving the drone backward 1 meter');
drone.back(0.1);
setTimeout(() => {
drone.stop();
clearTimeout();
},2000);
break;
case "STOP":
drone.stop();
break;
default:
break;
}
}
res.send('OK');
});
app.use('/',router);
app.listen(process.env.port || 3000);
Here is the HTML and JavaScript code to detect speech and send a command to Node server.
Voice Controlled Drone
A tiny app that allows you to control AR drone using voice
Sorry, Your Browser Doesn't Support the Web Speech API. Try Opening This Demo In Google Chrome.
Give the command
Press the Start Recognition button and allow access.