Using the Particle Photon as a wireless game controller

Andreas LundquistBlogs, Tutorials

photon-kvad
Tweet about this on TwitterShare on FacebookShare on LinkedInShare on Google+

Two years ago, Particle (formerly Spark) launched their Kickstarter campaign to fund the development of the Core. It proved to be a fine idea, they got funding and since then we’ve seen a lot of things have changed in the IoT-market. Now the Particle Photon is out, a sequel to the Core, and its a bigger brother in several ways:

photon

The major changes includes a shiny 120 MHz (STM32F205) microprocessor and a 6x RAM from to 128 KB. The development board is also equipped with both DAC and CAN, features fairly unusual to this kind of development board.

In short, the team behind Particle has developed an impressive and thought-out offering, where their development environment is mainly cloud based (Particle Build) although more experienced users may choose to opt-in for the “offline” Particle Dev software.

SparkFun were kind enough to provide us with a Photon and a couple of their shields. Among the shields I got my hands on were the Battery Shield and the IMU Shield. Ever since I begun developing examples at Evothings, I have thought that it would be interesting to try to add connectivity and external controls to an already existing, and normally stand-alone mobile application using Evothings Studio. Enter the pacman-canvas application developed by Platzh1rsch. This entire tutorial is based on work from this repository. The pacman-canvas application is a Pacman game implemented using nothing but HTML5, the application can be tested here. Since it is pure HTML5 it is possible to run the application on a mobile phone using the Evothings Studio. In this tutorial we will show you how you can edit the application to connect to a Photon equipped with an IMU shield that will be used to control Pacman himself in the game.

In order to follow this tutorial you need a Particle Photon and a smart phone (iOS/Android). I will also assume that you are familiar with Particle Build. If you’re not, Particle provide an excellent introduction on their web page to bring you up to speed.

Source Code

You can browse the source code for this tutorial in the Evothings GitHub repository.

Also you will find Platzh1rsch pacman-canvas in its original version in his GitHub repository.

What You Need

In order to follow this tutorial I assume that you are familiar with Particle Build. You need the following hardware:

  • An iOS or Android smartphone
  • A local network with a DHCP server

Step 1 – Hardware

The most simple step in this tutorial, is hardware preparation. Simply put the shields together and you are ready to start coding. I prefer to put the stack in a breadboard to minimize the risk of unintentionally shortcut any of the exposed pins. You can view my stack in the picture below.

Particle Photon

Step 2 – Embedded Software

Preparation

As stated earlier I assume that you are familiar with Build so we dive directly into the source code.

Source Code

The complete source code described below can be found on the Github repository.

What you are trying to do is to build a small server on the Photon. When a client connects to the device it will start to send a simple JSON-object containing acceleration data to the connected client. You will also implement a functionality that prints the network information over a serial connection. In this tutorial I will assume that the Photon and the smartphone are connected to the same local network. But it is just a matter of networking skills to make this example usable over the internet if one is interested in that.

// This #include statement was automatically added by the Particle IDE.
#include "SparkFunLSM9DS1/SparkFunLSM9DS1.h"

// Configure SparkFun Photon IMU Shield
#define LSM9DS1_M 0x1E 
#define LSM9DS1_AG  0x6B

// Configure example
#define DELAY_BETWEEN_TRANSFERS 50 // In milliseconds
#define SERVER_PORT 23
#define RGB_BRIGHTNESS 128
#define RGB_R 0
#define RGB_G 255
#define RGB_B 0
 
TCPServer server = TCPServer(SERVER_PORT);
TCPClient client;
LSM9DS1 imu;

void setup()
{

	// Enable serial communication
	Serial.begin(9600);

	// Start listening for clients
	server.begin();
	
	// Initialize SparkFun Photon IMU Shield
	imu.settings.device.commInterface = IMU_MODE_I2C;
	imu.settings.device.mAddress = LSM9DS1_M;
	imu.settings.device.agAddress = LSM9DS1_AG;
	imu.begin();

}

In the code above we have included the SparkFunLSM9DS1 library that allows us to communicate with the SparkFun IMU Shield. Then we configure the shield so that it communicates using the I2C bus. Then there is some defines that can be used to tweak the application to fit your needs. In this case we assume that it is left untouched. The next step is to define the server and client which we use to communicate with clients over the WiFi-connection. And lastly we define the imu which is our interface to the IMU shield.

In setup() we enable the serial communication that we will use to print the received networks settings. Then we start listening for TCP connections by executing the server.begin() method. The IMU shield is initiated after some settings have been updated. Now we have configured and initiated the serial connection, TCP server and the IMU shield.

The following part covers the loop() function that is executed continuously on the development board.

void loop()
{

	if (client.connected()) {

		// Discard data not read by client 
		client.flush();

		// Take control of the led
		RGB.control(true);
		RGB.color(RGB_R, RGB_G, RGB_B);

		// Read IMU data
		imu.readAccel(); 
		
		// Create JSON-object 
		char buffer [40]; 
		size_t length = sprintf(buffer, 
					"{\"ax\": %.3f, \"ay\": %.3f, \"az\": %.3f}\n",
					imu.calcAccel(imu.ax),
					imu.calcAccel(imu.ay),
					imu.calcAccel(imu.az)
						);   

		// Flash LED     
		RGB.brightness(RGB.brightness() == RGB_BRIGHTNESS ? 0 : RGB_BRIGHTNESS);

		// Transfer JSON-object
		server.write((uint8_t *)buffer, length);

	} else {

		// Turn on LED and release control of LED
		RGB.brightness(RGB_BRIGHTNESS);
		RGB.control(false);

		// Check if client connected
		client = server.available();
	}

	// Send network information if serial data is received
	if(Serial.available()) {

		Serial.println(WiFi.localIP());
		Serial.println(WiFi.subnetMask());
		Serial.println(WiFi.gatewayIP());
		Serial.println(WiFi.SSID());

		while(Serial.available()) {
			Serial.read();
		}

	}

	delay(DELAY_BETWEEN_TRANSFERS);
}

The application contains mainly of a if-else statement that checks whether or not there is a client connected to the development board. If there is a client present the application starts to discard data not yet read by the client by executing the client.flush() method. Then the software takes control of the RGB LED and configures the color that will be used to signal if there is a client connected to the development board. The imu.readAccel() method updates the internal data structures containing the latest acceleration data fetched from the shield. This method has to be called before calling the method imu.calcAccel() in order to get the latest sampled acceleration. The next step is to define a buffer that we fill with a JSON-object containing the latest acceleration data. Before the buffer is transmitted to the client by calling the server.write() method the RGB LED on the development board is flashed.

If there is not client connected to the Photon the application ensures that the LED is turned on and the control is released. Then the application reviews if there is a client connected.

If the application receives any data from the serial connection, it transmits the network information from the current network connection. This includes the local IP, subnet, gateway and the name of the network. As a last action the implementation reads and discards all data received from the serial connection.

As a last step, compile and flash the software onto your development board and open a serial connection to the board, send any data to the board and note the network information. Note the IP of the board, we will use that later in the mobile application.

Step 3 – Mobile application

Source Code

The complete source code described below can be found on the Github repository.

Fetch the pacman-canvas commit 3c24c7a995b1e329c5849ac24421eedfc1d70474 from GitHub. Unzip the application and open the files index.htm and pacman-canvas.js in an editor of your choice. These are the only two files that we have to edit in order to achieve our goals. Let’s begin with index.htm.

The first thing that we will do is to remove some code snippets that we don’t need. Remove the code from the following sections:

  • Google Analytics
  • Google Adsense
  • Highscore

The reason that we remove the Highscore section is that it is implemented using a PHP. A technology that is not supported when executing the application in a mobile context. Locate the <div> with the id menu-button and delete the following code:

<li class="button" id="highscore">Highscore</li>

The next step is to add the code snippet below just before the </head> tag.

<script>
	// Redirect console.log to Evothings Workbench.
	if (window.hyper && window.hyper.log) { 
		console.log = hyper.log
	}
</script>

<script src="cordova.js"></script>

This code redirects every console.log() calls to Evothings Workbench making it possible to debug applications remotely. The rest of the snipped includes a required Cordova specific library.

The last step is to add two buttons to the main menu. One that we will use to connect to the Photon and one that will take us back to the start screen. Locate the <div> with the id menu-control and add the following code:

<li class="button" id="photon-control">Connect Photon</li>
<li class="button" id="evothings-back">Back</li>

There is nothing left to edit in the index.htm at this point so we are moving on to editing the pacman-canvas.js file. As a first step we are going to create an object that will represent the Photon. At the top of the file, right after the declaration of the global variables add the following code snippet, don’t forget to add the IP of the Photon:

// Photon handler
function Photon(){

  // IPAddress and port of the Photon
  var IPAddress = 'YOUR IP HERE';
  var port = 23;

  var socketId;
  var previousDirection = {};

  this.connect = function() {

    // Return immediately if there is a connection present
    if(socketId){
      return;
    }

    chrome.sockets.tcp.create(function(createInfo) {

      socketId = createInfo.socketId;

      chrome.sockets.tcp.connect(
        socketId,
        IPAddress,
        port,
        connectedCallback)
      });
    };

  this.disconnect = function () {

    chrome.sockets.tcp.close(socketId, function() {
      
      socketId = 0;

      // Reset graphics and callbacks associated 
      // with the button in the main menu

      $('#' + previousDirection.name).css('background-color', '');

      $('#photon-control').text('Connect Photon');

      $(document).off('click','.button#photon-control')
                 .on('click','.button#photon-control',
                 function(event) {
                  photon.connect()
                });
              })
  };

  function connectedCallback(result){

    if (result === 0) {

      // Update graphics and callbacks associated with the
      // button in the main menu 
      $('#photon-control').text('Disconnect');

      $(document).off('click','.button#photon-control')
                 .on('click','.button#photon-control',
                 function(event) {
                  photon.disconnect();
                 });

      // Set callback to handle received data
      chrome.sockets.tcp.onReceive.addListener(receiveData);

    }
    else {

      alert('Failed to connect to Photon! Try again!', function() {});
    }
  };

  // Function to handle received data. 
  function receiveData(info) {

    // Convert buffer to string containing the sent JSON-object
    var jsonString = String.fromCharCode.apply(null, new Uint8Array(info.data));

    // Try to convert the string to an actual JavaScript object 
    try {
      var jsonObject = JSON.parse(jsonString);
    }
    catch(e) {
      return; 
    }

    var ax = jsonObject['ax'];
    var ay = jsonObject['ay'];
    var az = jsonObject['az'];

    // Adjust pacman direction depending on received acceleration 
    if(Math.abs(ax) > Math.abs(ay)) {

      if(ax < 0) {
        adjustPacmanDirection(down);
      }
      else {
        adjustPacmanDirection(up);
      }

    }
    else if (Math.abs(ay) > Math.abs(ax)) {

      if(ay < 0) {

        adjustPacmanDirection(left);
      }
      else {

        adjustPacmanDirection(right);
      }
    } 
  };

  function adjustPacmanDirection(direction){

    pacman.directionWatcher.set(direction);

    $('#' + direction.name).css('background-color', '#008000');
      
    if(!direction.equals(previousDirection)) {
      $('#' + previousDirection.name).css('background-color', '');
      previousDirection = direction;
    }
  };
};

Note that you have to update the code snippet above with the ipaddress of your Photon.

The method connect() tries to connect to the development board using the defined IPAddress and port. This method is basically a wrapper for the chrome.sockets.tcp.create() that ensures that the application only is connected to one development board. The callback connectedCallback() is executed asynchronously when there is a result of the connection attempt. The callback evaluates if connection succeeded or not. If there is an established connection (result === 0) the main menu is updated to handle a disconnect and a callback to handle (receiveData()) is configured. If the connection failed for some reason a alert dialog is showed on the screen and the socketId is set to zero to enable new connection attempts.

The receiveData() function is executed each time there is new data received. The first step is to convert the received data to a JavaScript object from which data can be extracted. The second step is to adjust the direction of the pacman depending on the values of the acceleration.

The function adjustPacmanDirection() changes the direction of the pacman figure in the game as well as updates the arrows to show which direction the figure is heading at the moment.

Declare a variable named photon among the global variables by adding the following code on the line below the declaration of the variable mapConfig.

var photon;

The next step is to define the variable photon. This is done by adding the following code just below the definition of the game variable (game = new Game()).

photon = new Photon(); 

The next and final step is to add initial callbacks to the buttons we added to index.html earlier in this tutorial. This has to be done in the $(document).ready() method since the DOM has to be fully loaded before we can add the callbacks. Locate the method and add the following code where the other buttons callbacks are defined.

$(document).on('click','.button#photon-control',function(event) {
  photon.connect()
});

$(document).on('click','.button#evothings-back',function(event) {
  history.back()
});

If you followed each and every step of the tutorial you should now be able to connect to your Photon and use it as a game controller for the game.

pacman_2

Summary

It was a pure pleasure to work with the Particle Photon. The team has truly succeeded in developing an great cloud based offer that is simple to use. I will definitely use the development board as a base in future projects. Using Evothings Studio I managed to develop the application in no time.

Time for you to start exploring the exciting world of IoT applications. Download the Evothings Studio today and start developing your own IoT applications!

download