Sheldon robot: remote control

With any robot, it’s always useful to have some form of remote connection, if not full remote control. The question is how to achieve this.

By the way, in this post, I’m definitely in ‘thinking out loud/workshop notebook’ mode, because nothing described here is finished. I’m just playing with concepts.

Now, when I mention ‘remote control’, I’m not talking about controlling the robot like you might a radio-controlled toy car. Personally, I’m always disappointed when I see a bit of video of a ‘robot’ manoeuvring in interesting ways only to find out that it’s being directly steered. Yes, sometimes the creator has shown technical flair by, say, writing an Android app and maybe some communications protocols, and is controlling the machine via wifi or Bluetooth. That’s all fine and dandy; but to me, a machine only becomes a true robot when it gains a degree of autonomy.

So, in this scheme of things, the purpose of having a remote connection to Sheldon is two-fold: one, to issue ‘tasks’ that Sheldon can then go off and fulfil; and two, it’s to receive telemetry.

In the process of getting to that state, though, I may veer towards the ‘remote control’ end of the spectrum. I’m a long way from having a semi-, let alone fully, autonomous robot. In the immediate future – quite soon, I hope – I’ll have a base vehicle with collision avoidance capabilities. For that, I’ve decided it would be nice to remotely start and stop the vehicle, select a speed, tell it whether to go forward or backward, and maybe issue basic turn commands (not steering, per se) to help it if I can see it’s about to run into trouble.

Means of communication

So, how to talk to it? There are just so many options these days. Here are some of the things I considered:

1. Direct Bluetooth connection between an app on my laptop and the motion control node (a Teensy). I’ve done this before with a Bluetooth module and an app written in Processing. It worked well whenever a Bluetooth connection could be established, which wasn’t always.
2. A Raspberry Pi on the robot running its wifi in Access Point (AP) mode. An HTML/JavaScript app running under Apache on the RPi would provide a web interface – accessible from any of my devices with a browser – and talk to the motion control node via serial, I2C or SPI.
3. A Raspberry Pi or ESP8266 device on the robot running a websockets server. An HTML/JavaScript app on a web server on my local network would provide the interface. This would be accessible from any of my devices with a web browser.
4. A Raspberry Pi on the robot running a web server and providing the web app to control the robot but connected via the local network rather than in AP mode.
5. Something involving MQTT. But the latency can be horrible.

I really like solution 2 because it eliminates the need for a local network. Solution 4 is the next-best option and possibly easier to implement. But just now I implemented solution 3. Huh?

Overview of how it works.

Baby steps

The fact is, I’m just not ready to put a Raspberry Pi on Sheldon. Or maybe he’s not ready.

I’m still grappling with the fundamentals of the base vehicle. I have the motion control node, using the Teensy 3.5, up and running, even if not all its functionality is there yet. I just wanted to add some quick’n’dirty remote control to make like easier.

Adafruit Huzzah ESP8266

I have a bunch of ESP8266 boards (the Adafruit Huzzah) so why not use them?

Now, you might point out that you can put the ESP8266 into AP mode and access it directly, as per solution 2. And you’d be right. Indeed, that was my first line of enquiry. But creating web pages on an ESP8266 is hair-shirted and horrible. There’s simply no way you’re going to create a decent UI with lots of JavaScript and JQuery loveliness that way. (Yeah, I know, you could add an SD card reader and put the files there, but it’s still clumsy.)

You could just create an HTTP-based API, but I really wanted a decent UI for this. Another option I considered was creating the UI on a web server running on localhost on my laptop, and having that interact with the API on the ESP8266, but that was getting unnecessarily complex, and isn’t a million miles away from solution 3, other than not needing a local network.

The network solution

The app as it currently looks on my iPhone in Safari.

This is how it works with the current prototype. I have a web server running on a machine (my iMac) on my local network (which has the IP address 10.0.0.15). This serves up a virtual host on port 8080. So, if I’m working on the iMac, I just use localhost:8080 as the address in the browser. If I’m using another device, such as my Android tablet (which is what I mostly plan to use), I go to 10.0.0.15:8080.

Incidentally, the root document directory for this virtual host is on Dropbox. I’ve configured Apache on my MacBook Pro to also have a virtual host pointing to the same files. So I can use the laptop as the server instead.

The web page uses JavaScript, JQuery and HTML5 websockets to connect to the ESP8266. This receives commands from the web app, ignores any it doesn’t recognise and passes on the ones it does to the Teensy motion control node via serial.

Yes, there’s going to be some latency in there on account of going via the network plus the serial connection, plus websockets seems to nap sometimes. But remember, I’m not planning to use this for anything real-time, such as precise steering.

Getting the message

One decision I needed to make was exactly what kind of messages to send over websockets. Efficiency and speed would suggest using one-byte numeric codes – eg, sending the value 64 to mean ‘select FORWARD direction’. And there’s no problem with doing that other than that such codes are a tad obscure when coding and debugging.

Instead, I decided on four-character strings, such as STOP, STRT and what have you. Four characters is enough to get a proper sense of the meaning of the command while keeping the amount of data that needs to be sent reasonably small. (I’m not even sure the latter point is an issue – maybe I’m being unnecessarily terse, but what the hell.)

Working with websockets

I’m no expert on websockets – in fact, this is the first time I’ve used them having had only the vaguest notion of what they do. But it seems that the way you use them is that the client (the web app in my case) sends a message to the server (the ESP8266) and, if the latter so desires, gets a response back.

Using websockets turned out to be very simple. I won’t post the whole code because it’s still in a state of flux, but here are some JavaScript snippets from the server.

//during setup
let socket = null; // for websocket
let ip = '10.0.0.170'; // default IP on my network
let port = '8181'; // because why not?
let connected = false;
let recvdMsg = '';

function socketConnect() {
	if(ip != '' && !connected) {
		var url = 'ws://' + ip + ':' + port;
		socket = new WebSocket(url);
		socket.onopen = function() {
			//
		};
		socket.onmessage = function(event) {
			recvdMsg = event.data;
			$('#msg').html(recvdMsg); // shows on page
		};
		socket.onclose = function() {
			reset(); // a function elsewhere that returns page to default
			alert('WebSocket closed!');
		};
		socket.onerror = function() {
			//
		};
		connected = true;
	}
}

On the web page, there’s a ‘connect’ button that calls the socketConnect() function (and then switches to being a ‘disconnect’ button). There’s also a form to enter the IP of the ESP8266. Once connected, other buttons use socket.send() to send messages – eg, the ‘START’ button triggers the command: socket.send(‘STRT’);.

On the ESP8266, the crucial bits of code are:

#include <ESP8266WiFi.h>
#include <WebSocketsServer.h>
#define WEBSOCKET_PORT 8181
String motorCmds[] = {"STRT", "STOP", "FRWD", "RVRS", "CRWL", "SLOW", "NRML", "FAST", "ROTL", "ROTR", "VERL", "VERR"};

void webSocketEvent(uint8_t num, WStype_t type, uint8_t * payload, size_t length) {
  if (type == WStype_TEXT) {
    String msg = String((const char *)payload);
    String response = "";
    // in the following tests, we're trying to limit the amount of time spent performing
    // the tests and responding. First we check for commands that aren't in the motorCmds[]
    // array because they need specific responses and also it saves us going through the loop
    // looking for motor commands (which is why that's last).
    if (msg.equals("PING")) { // might get this frequently, so test for this first
      response = "PONG";
    } else {
      for (uint8_t i = 0; i < 12; i++) {
        if (msg.equals(motorCmds[i])) {
          response = ">MT:" + msg;
          Serial.println(response);    // forward via serial port to Teensy
        }
      }
    }
    if(response.length() > 0) {
      webSocket.broadcastTXT(response);
    }
  }
}

void setup() {
  Serial.begin(115200);
  // ... other stuff ...
  wifiConnect();
  webSocket.begin();
  webSocket.onEvent(webSocketEvent);
}

void loop() {
  webSocket.loop();
  // ... some other stuff, but not much ...
}

Now what if I want the ESP8266 to initiate a message? For example, the ESP8266 might receive a message over its serial connection from the motion control node, perhaps saying that the robot is stalled. There appears to be no way to send this to the web app via our websockets setup. But no matter: in this scenario, I’m running a web server anyway. I can simply set up an API on that (probably via a PHP script). Any such incoming messages can be queued in a text file to be read by the web app at yet-to-be-determined intervals. That might work.

Progress report

I’ve got as far as hacking together a basic web app that can send commands to the ESP8266. And I’ve programmed the ESP8266 to receive these commands and pass on appropriate ones via the serial connection.

The next step will be to connect the ESP8266 to the Teensy. Watch this space.