Interactive Canvas from Google Actions Cheat Guide (If You’re Just Getting Started.)

In this piece, I will share some of the tricks I’ve learned while developing an action using Interactive Canvas.

Ok, so you want to start developing on the brand and shinny Google Actions and Interactive Canvas. I’ve developed an action, while it was time-consuming and complex I learned a few tricks up my sleeve on the way.

Last weekend I participated in a hackathon called ‘ Woman in Voice Mx’. Where me and my team, won first place with our product called ‘Pixie en el Internet’ or Pixie on the Web.

Pixi en el Internet is a Spanish Google Action based on the Google’s Be Awesome Internet for safe browsing. 1 of 3 user online are minors and 80% of them learnt how to browse the internet by themselves or through friend or a older sibling. This puts all the minors (and let’s be honest some adults) in risk of scams, cyber crimes and more.

Pixie en el Internet is a game about Pixie, a 12 year old latina and her robot BIM. Pixie doesn’t know how to browse safely on the internet so with the user’s help and BIM’s help they will learn together.

Meet Pixie (fun fact she is animated by me :))

It’s such a fun project and unique project since Pixie can empathize with the end user while the all knowing robot helps them out. Pixie is voiced by a real Mexican girl and BIM is the google assistant voice. It consists of trivia questions to teach the ciber security curriculum and a mini game to test their new obtained knowledge.

We are currently on development and are actively looking for partners to make Pixie a reality to every latino kid who owns a Google Nest in their homes and eventually support it for every language available.

Explore and READ the official documentation:

Hear me out, it’s been just a couple of months since Google changed their documentation to Google Actions. If you google your way into the documentation the probability that you see legacy documentation is pretty high, although it is similar you can get confused easily.

So my tip is to go through all new Google Action documentation, go through all the pages, explore the endless resources you have. Even if you don’t need it right away it’s a great tip to see everything it can offer you.

Personally, I found many design guidelines, how to change your project to another language (and how those folders need to be managed) Many of the things I applied to my final action I found them before.

Google documentation is pretty good in general. This is not the exception but you do need to give it a little bit of reading time.

Snow Pal Code Lab Tricks

So if you’re like me, you’ll probably be on the lookout for some tutorials to get you going right away. (Oh, hi there!) But eventually, you’ll end up running the Snowpal Code Lab teaching you the basics because it’s the first step you should be doing.

This Code Lab is great and get’s updated every now and then. However, in the Hackathon, I participated.. few got it running. I ran into some issues but nothing an hour or three of browsing Stack Overflow questions couldn’t solve.

So here is my breakdown of some of the issues I had and other people had:

Gactions Library

Ok, so this is a library I couldn’t just download by npm. You might be able to do it eventually though… But for now, you have to go to the official gactions page and follow the instructions. Official Documentation :)

1- Download the gactions file for your OS system

2- Extract in your path /usr/local/bin

After that, I couldn’t get it working so... I ended up placing the folder on my code right here →

What my sdk folder looks like.

When I call it I just add a ‘./’ to access the file. So if the codelab says something like this:

$ gactions login

I would do the following:

$ ./gactions login

Firebase + Node JS version

Ok, let me just get my Node JS environment for you real quick.

To be honest, I haven’t coded in javascript in a while, I just love the simplicity of python too much and for CS studies purposes it makes it a great tool.

Check your node version and check that is compatible with your firebase tools. And yes, check if it did install it or upgrade it correctly. You might get a lot of warning messages and in between those you didn’t upgrade it correctly.

To check your node and firebase tool version.

$ node -v
$ firebase-tools -v

Install firebase tools:

$ npm install -g firebase-tools

Upgrade to the latest version of both on your machine.

$ npm update -g firebase-tools

Now… if is the first time you are doing this, follow the documentation.

GCS Project

Google Cloud Services is pretty straight forward and like any google product, it has its handful of documentation on how to integrate different pieces to a project. You can make it as complex as you desire, but keep it really simple. There are no hidden costs, and it’s… *chef kiss*

So what I did is create a project in Google Actions which automatically creates a project on your Google Cloud. You’ll be using a firebase webhook with your action so, the quick fix is going to the Firebase Console and change your plan for Blaze one. This means it will cost you to run your webhook but cents. It is actually not that expensive but I would be careful.

After this, what I did is do simple troubleshooting. I don’t recall any special messages. But, when you upload your code and launch it, it will probably give you warning messages, telling you to allow certain things or that your missing something. Again, GCS is pretty straightforward with their error messages so read what it’s actually asking you to do. I often google the error or the permission I have to allow and the solution appears in my search results.

If you have any issues with this, let me know in the comments and I’ll help you.

Make a game using Phaser:

Kudos to the engineers who made the codelab with Phaser. Never used Phaser before but once you get it, it’s pretty straight forward. If you ever made a game in no code or any type of canvas game it’s basically all you need. However it’s a bit different from the original Phaser configuration.

I started with the snowpal template and stripped it down so I could have the basic phaser configuration.

What you end up is with a script that has this elements:

• preload() → load all the images or sprites on your webapp.
• create() → create all the components or
• {game functions to show, hide, change and update things.}
• Class Objects → If you want to separate a specific part of the game in a class.

As you can see, there is no update() function as a normal phaser would have, so you can work around this using animations with sprites in loop.

To make it easier I would keep all the game logic or your webapp logic in the webhook because it’s faster. You can import JSON files in a blink of an eye and have the results so your assistant can say them while keeping your game data intact.

How to use scenes effectively.

To be completely honest at first I didn’t get why and what was the objective of using scenes until I found how really of a game changer they are.

I may be using them wrong, but for me it allowed me to divide my action into pieces and reuse many of the intents I made ( a YES and NO intent for example).

This way you can actually reduce the number of intents you make because you can reuse them. Also you can automatize a little bit more and put an intent or a webhook call to something once you enter the scene or have conditions met and then change the scene to another one.

One of the tricky things about the scenes is that the configuration in YAML doesn’t export to a 100% accuracy so you might want to go and change somethings manually like connecting your webhooks.

Data in Interactive Canvas

So now that you have the snowpal codelab up and running, you might want to reuse some of the code to make your game. So for me was two things, sending data to my game and collecting data from it. This is explained thoughtfully on the documentation but it took me a bit to figure it out completely.

You should have the simple actions.js, scene.js and index.js webhook file available in the documentation.

How to get data from your web app:

To get data on your canvas you will need basically two things, the first one is access it in your index.js file like so.

$ const example = conv.context.canvas.state.example

And also this function in your scene.js, this function should be called everytime you update the scene in your webapp or in your phaser game. Be careful on how you write the variables so you don’t have any errors. The white spaces are important.

updateCanvasState(){
    window.interactiveCanvas.setCanvasState({
       {global variable name} : {variable in your scene.js},
       example : this.example,
});

How to send data to your web app:

To send data to your web app you need to do a couple of things first let’s start with your webhook index.js file. You will need to load a scene to your webapp:

conv.add(new Canvas({
    data: {
        // This is the scene in Action.js you are calling
        command : 'START_EXAMPLE',
        // This is the variable you are sending to your webapp
        variable : 'this is an example'
    },
}));

Then you need to add it to your Action.js. This script is what basically ties everything up together. You need to connect it to your scene.js file

constructor(scene) {
   this.canvas = window.interactiveCanvas;
   this.gameScene = scene;
   this.commands = {
        START_EXAMPLE: (params) => 
           this.gameScene.start_example(params);
        }
   };
}

And also change the Callback by adding your variable.

setCallbacks() {
  // Declare the Interactive Canvas action callbacks.
  const callbacks = {
    onUpdate: (data) => {
      try {
        const dataEntry = data[0];
        const command = dataEntry.command ? dataEntry.command      dataEntry.google ? dataEntry.google.intent.name : null;   
         // RIGHT HERE --> 
         const params = dataEntry.variable ? dataEntry.variable : null;
         this.commands[command.toUpperCase()](params);
       } catch (e) { 
         // do nothing, when no command is sent or found
       }
    }
 };
 this.canvas.ready(callbacks);
}

And finally on your scene function your are calling, you are going to name the variable you’re using when you declare the function.

start_example(value_from_webhook){
    this.text.setText(value_from_webhook);
    updateCanvasState();
}

That’s it :)

Obviously this list of tips and tricks are very general and specific to my experience, I have a lot to explore in Google Actions and their interactive canvas. But overall I hope this list helps you if you’re just geting started.

If you have any questions or had a I would love to hear it on the comments or twitter @sogalanbat