I was very fortunate and lucky to receive a deep racer for attending one of the first Deep Racer workshops at AWS re:Invent 2018.
The AWS DeepRacer is a physical computer on wheels, with a webcam. It’s a fun way to get started with re-inforcement learning.
Re-inforcement learning is a type of machine learning that works when an ‘agent’ is allowed to act on a trial-and-error basis within an interactive environment, using feedback from those actions to learn over time in order to reach a predetermined goal or to maximize some type of score or reward.
When you start up the car, connect it to a WiFi network and open the web interface, it loads with two driving modes: Automated or Manual.
Automated requires a model and the deployment of that model to your device via a USB stick. A model can be created from the AWS DeepRacer Console within the AWS Management Console, however, at the time of writing this blog post, the AWS DeepRacer console is still in Preview. While I wait for my personal AWS account to be accepted for the Preview, I can still play with the car using the Manual control method.
I was reading through the AWS DeepRacer Developer Guide and came across this little golden snippet.
Naturally, I wanted to see if I could push some boundaries.
So, the AWS DeepRacer only works over WiFi. What if I could remote control the vehicle without it being tethered to a fixed WiFi point? My phone can be a WiFi Hotspot, and it also uses 4G…
2 minutes later…
The AWS DeepRacer comes pre-installed with Ubuntu 16.04 and ROS (Robot Operating System). It’s basically a mini-computer on wheels - a super cool setup. I can install any software from the Ubuntu repository.
In most countries, generally, mobile networks are implemented with CG-NAT (Carrier Grade NAT). That means that you don’t get a publicly routable IP Address… you’re behind a big NAT system machine. How am I going to open the AWS DeepRacer web interface to control it? That’s right… A VPN.
I ended up with something that looks like this:
To get this working, I needed to install VPN software that performs connection facilitation. A public EC2 instance is required because it’s accessible by both my laptop and the DeepRacer through the 4G connection. My laptop is behind another NAT without any port forwards.
Initially, I started with Wireguard, because I heard some good things about it and wanted to try it out. Unfortunately the DeepRacer has SecureBoot enabled, we means can’t use WireGuard because it implements the VPN as a kernel extension. I pulled out my trusty, userspace friendly, tinc vpn and installed it on the DeepRacer, my laptop and a publicly accessible EC2 instance.
tinc is a Virtual Private Network (VPN) daemon that uses tunnelling and encryption to create a secure private network between hosts on the Internet. It’s similar to OpenVPN, with the benefit that it is much simpler to configure and has a lot less bloat.
I enabled the WiFi Hotspot on my phone, updated the wifi-creds.txt file in the root of the USB stick and restarted the DeepRacer. tinc started up during the boot process, and initiated a connection to the EC2 instance without a hitch! Once it was connected, the EC2 instance was able to mediate UDP holepunching to faciliate direct connectivity from my laptop to the DeepRacer over UDP.
While I had the web interface open, I monitored the traffic going over the VPN, it was pushing nearly 1-2MB/s over 4G. Looking at the implementation of ROS, I could see that the WebCam mpeg stream was being delivered at a really high quality. Which is fine for WiFi, but not great for 4G where data is quite a lot more expensive.
The webpage is calling the OpenROV Geo Video Server with the default topic configured. https://192.168.53.3/route?topic=/video_mjpeg&width=480&height=360
&quality=9 to the end of the query string. I adjusted the webpage to request this very poor quality encoding, and the bandwidth dropped to about 80KB/s. You can still see the webcam output, which is good enough for me.
Happy with the result, I took it for a spin. I wasn’t too keen to send it out by itself, so I followed it all the way around the neighborhood, while my wife controlled it on my laptop from my home-office.
It was a fun experience while the DeepRacer console is still in preview.