Build fulfillment with the Actions on Google Node.js client library (Dialogflow)

The Actions on Google Node.js client library is the recommended way for accessing and interacting with the Actions on Google platform if you are creating a fulfillment webhook in JavaScript.

Introduction

The Node.js client library is a fulfillment library for Actions on Google that provides these features:

• Supports all Actions on Google features, including text and rich multimedia responses, account sign-in, data storage, transactions, and more.
• Provides an idiomatic layer of abstraction in JavaScript that wraps the conversation HTTP/JSON webhook API.
• Handles the low-level details of communication between your fulfillment and the Actions on Google platform.
• Can be installed using familiar package management tools, such as npm or yarn.
• Lets you easily deploy your fulfillment webhook on serverless computing platforms such as Cloud Functions for Firebase or AWS Lambda. You can also host your fulfillment webhook on a cloud service provider or on a self-hosted and self-managed environment.
• Is compatible with Node.js v6.0.0 and higher.

You can use the client library in conjunction with the Dialogflow integration for Actions on Google or with the Actions SDK.

To see full code samples for using the client library, you can visit the samples page.

View the API reference

The API reference is hosted on the Actions on Google Node.js client library GitHub page.

You can also generate a local copy of the reference by running the following command from the directory where you downloaded the client library code:

yarn docs

The generated docs will be available in the docs folder of the directory where you downloaded the client library code.

Understand how it works

Before you use the client library, it is helpful to understand how your fulfillment webhook uses the client library to process user requests that Actions on Google sends to your fulfillment.

When you create a fulfillment webhook in JavaScript, you can deploy and host your code on a serverless computing environment like Google's Cloud Functions for Firebase or AWS Lambda. You can also host the code yourself without additional work using the Express web framework.

Note: If you want to run your fulfillment webhook in some other environment, you can use the generic StandardHandler function. You can call this function using app.handler which takes a JSON object and headers as inputs and outputs a JSON object and a status code in a Promise.

Within the runtime environment, the fulfillment webhook can call functions in the client library to process user requests and send responses back to Actions on Google for rendering into user output.

The key tasks that your fulfillment webhook handles with the aid of the client library are briefly summarized below:

Figure 1. High-level architecture of the Node.js client library

1. Receiving user requests: When a user makes a query to the Google Assistant, the Actions on Google platform sends an HTTP request to your fulfillment webhook; the request includes a JSON payload that contains the intent and other data such as the raw text of the user input, and the surface capabilities of the user's device. For more examples of the JSON payload content, see the Dialogflow webhook format and conversation webhook format guides.
2. Framework call format detection: For supported frameworks, the client library auto-detects the framework's call format (for example, if the request came from the Express web framework or from AWS Lambda) and knows how to seamlessly handle communication with the Actions on Google platform.
3. Service handler processing: The client library represents the conversation HTTP/JSON webhook API for Dialogflow and Actions SDK as a service function. Your fulfillment webhook uses the appropriate service to create a global app instance. The app instance acts as a handler for HTTP requests and understands the service's specific protocol.
4. Conversation processing: The client library represents per-conversation information as a Conversation object that's attached to the app instance. Your fulfillment webhook can use the Conversation object to retrieve cross-conversational stored data or state information, send responses to users, or close the mic.
5. Middleware processing: The client library lets you create your own conversation services middleware, which consists of one or more functions you define that the client library automatically runs before calling the intent handler. Your fulfillment webhook can use your middleware to add properties or helper classes to the Conversation object.
6. Intent handler processing: The client library lets you define handlers for intents your fulfillment webhook understands. For Dialogflow, the client library routes the request to the correct intent handler by mapping to the exact string of the intent name defined in the Dialogflow console. For Actions SDK, it's routed based on the intent property sent from Actions on Google.
7. Sending responses to users: To construct responses, your fulfillment webhook calls the Conversation#ask() function. The ask() function can be called multiple times to incrementally build the response. The client library serializes the response into an HTTP request with a JSON payload and sends it to Actions on Google. The close() function has a similar behavior as ask() but closes the conversation.

Note: Be aware that the Actions on Google platform has certain restrictions on the responses that the developer sends back, like the number of SimpleResponse objects sent via the ask() function. Because this is independent of the client library, you should test with the Actions simulator to check for validation errors on your responses.

Set up your local development environment

Before you implement your fulfillment webhook, make sure to first install the client library.

Install the client library

The easiest way to install the client library to your local development environment is to use a package manager, such as npm or yarn.

To install, run one of these commands from the terminal:

• If using npm: npm install actions-on-google
• If using yarn: yarn add actions-on-google

Set up your project folders

Depending on where you plan to deploy the fulfillment webhook (Google's Cloud Functions for Firebase, AWS Lambda, or self-hosted Express), you may need to create a specific project folder structure to save your files.

For example, if you are using Cloud Functions for Firebase, you can set up required project folders by performing the steps described in Set up Node.js and the Firebase CLI and Initialize Firebase for Cloud Functions. For Cloud Functions for Firebase, you typically write your fulfillment webhook in the /functions/index.js file.

Build an app instance

Actions on Google uses specific messaging formats for exchanging requests and responses with your fulfillment webhook, depending on whether you are building a conversational Action using Dialogflow or the Actions SDK or building a smart home Action.

To represent these different request and response protocols, the client library provides three service functions:

• dialogflow
• actionssdk
• smarthome

The conversation webhook protocol is used by both conversational services (Dialogflow and Actions SDK), but each service wraps messages differently.

You use a service to create an app instance. The app instance encapsulates the global state and fulfillment logic for your webhook and handles communication between Actions on Google and your fulfillment using the service-specific protocol.

You can configure the properties of the app instance and call its methods to direct the fulfillment webhook's behavior. You can also easily plug the app instance into a serverless computing environment, such as Cloud Functions for Firebase, which accepts JavaScript functions as handlers for HTTP requests.

To build an app instance in your fulfillment webhook, follow these steps:

1. Call the require() function to import the 'actions-on-google' module and load the service you want. For example, the following snippet shows how you might load the dialogflow service and some elements used to build responses, and assign it to a constant named dialogflow:

   // Import the service function and various response classes
   const {
     dialogflow,
     actionssdk,
     Image,
     Table,
     Carousel,
   } = require('actions-on-google');

   Here, actions-on-google refers to a dependency that's specified in a package.json file in your project folder (you can refer to this example package.json file for an example).

   When obtaining an app instance, you can optionally specify classes representing rich responses, helper intents, and other Actions on Google functionality that you want to use. For the full list of valid classes you can load, see the reference documentation for the conversation response and helper intent modules.

   Note: The client library is frequently updated. Please see the latest release notes for any API changes.

2. Create an app instance by calling the service you loaded. For example:

   const app = dialogflow();

3. To configure the app instance at initialization, you can provide an options object as the first argument when you call the service. (See DialogflowOptions for more details.) For example, the following snippet shows how to log the raw JSON payload from the user request or response by setting the { debug: true } flag:

const app = dialogflow({
  debug: true
});

Set handlers for events

To process Actions on Google-related events created by the client library during the lifecycle of the user interaction with your Action, you'll use the client library to build handlers to process user requests and send back responses.

You can create functions that act as handlers for these main types of events that the client library recognizes:

• Intent events: Intents are unique identifiers that Actions on Google sends to your fulfillment whenever a user requests some specific functionality. If you are using Dialogflow, this corresponds to Dialogflow matching a user query to an intent in your Dialogflow agent.
• Error events: When a JavaScript or client library error occurs, you can use the app instance's catch function to process the error exception appropriately. You should implement a single catch function to handle all errors that your fulfillment cares about.
• Fallback events: A fallback event occurs when the user sends a query that Actions on Google is unable to recognize. You can use the app instance's fallback function to register a generic fallback handler which will be triggered if no intent handler has been matched for the incoming fulfilment request. You should implement a single fallback function to handle all fallback events. If you are using Dialogflow, Dialogflow can trigger a specific fallback intent when no other intent is matched. You should create a corresponding intent handler for that fallback intent.

Whenever the user sends a request to your Action, the app instance creates a Conversation object that represents that conversation session. This object is accessed via the conv variable name passed in the intent handler function as the first function argument. You'll typically use the conv object in your handlers to send a response to the user.

User queries can also include parameters that your Action can extract and use to refine responses.

• If you are using Actions SDK, you define parameters in the Action package. To see an example of how you can extract parameters from intents, see the Eliza code sample.
• If you are using Dialogflow, you can access the parameter values via the params variable. To see examples of handling intents with parameters in Dialogflow, see Access parameters and contexts.

Set handlers for intents

To set the handler for an intent, call the intent() function of your app instance. For example, if you're using Dialogflow, this is the DialogflowApp#intent() function. In the arguments, specify the intent name and provide a handler function.

If you are using Dialogflow, you don't need to set handlers for every intent in your agent. Instead, you can take advantage of Dialogflow's built-in response handler to automatically handle intents without implementing your own handler functions. For example, the default welcome intent can be delegated to Dialogflow in this way.

The following example shows intent handlers for the 'greeting' and 'bye' intents. Their anonymous handler functions take a conv argument and send back a simple string response to the user via the conv.ask()function:

app.intent('Default Welcome Intent', (conv) => {
  conv.ask('How are you?');
});

app.intent('bye', (conv) => {
  conv.close('See you later!');
});

Note that the close() function is similar to ask() except that it closes the mic and the conversation is over.

To learn more about how to build handlers for intents, see Build your intent handler.

Set handlers for error events

To set the handlers for errors, call the catch() function of your app instance. (For example, if you're using Dialogflow, this is the DialogflowApp#catch() function.)

The following example shows a simple catch error handler that sends the error to console output and sends back a simple string response to prompt the user via the conv.ask()function:

app.catch((conv, error) => {
  console.error(error);
  conv.ask('I encountered a glitch. Can you say that again?');
});

Set handlers for fallback events

To set a generic fallback handler when no intent is matched for the incoming request for fulfillment, call the fallback() function of your app instance. (For example, if you're using Dialogflow, this is the DialogflowApp#fallback() function.)

The following example shows a simple fallback handler that sends back a simple string response to prompt the user via the conv.ask()function:

app.fallback((conv) => {
  conv.ask(`I couldn't understand. Can you say that again?`);
});

Build your intent handler

This section covers some common use cases when you implement intent handlers with the client library. To see how the client library matches the intent, refer to the 'Intent handler processing' section in Understand how it works.

Access parameters and contexts

If you are using Dialogflow, you can define parameters and contexts in your Dialogflow agent to maintain state information and control the conversation flow.

Parameters are useful for capturing important words, phrases, or values in user queries. Dialogflow extracts the corresponding parameters from user queries at runtime, and you can process these parameter values in your fulfillment webhook to determine how to respond to users.

Whenever the user sends a request to your Action, the DialogflowApp instance creates a parameters object that represents the parameter values that Dialogflow extracted from that request. This object is accessed via the params variable name.

The following snippet shows how you can access the name property from the params object when the user sends a request:

app.intent('Default Welcome Intent', (conv, params) => {
  conv.ask(`How are you, ${params.name}?`);
});

Here's an alternative snippet that does the same thing. The curly braces ({}) perform JavaScript destructuring to take the name property from the parameters object and use it as a local variable:

app.intent('Default Welcome Intent', (conv, {name}) => {
  conv.ask(`How are you, ${name}?`);
});

In the following snippet, the parameter name is full-name but is destructured and assigned to a local variable called name:

app.intent('Default Welcome Intent', (conv, {'full-name': name}) => {
  conv.ask(`How are you, ${name}?`);
});

Contexts are an advanced feature of Dialogflow. You can use contexts to manage conversation state, flow, and branching. The client library provides access to a context via the DialogflowConversation#contexts object. The following snippet shows how you can set a context programmatically in your fulfillment webhook and how to retrieve the context object:

app.intent('intent1', (conv) => {
  const lifespan = 5;
  const contextParameters = {
    color: 'red',
  };
  conv.contexts.set('context1', lifespan, contextParameters);
  // ...
  conv.ask('...');
});

app.intent('intent2', (conv) => {
  const context1 = conv.contexts.get('context1');
  const contextParameters = context1.parameters;
  // ...
  conv.ask('...');
});

app.intent('intent3', (conv) => {
  conv.contexts.delete('context1');
  // ...
  conv.ask('...');
});

Access helper intent results

For convenience, the client library provides helper intent classes that wrap common types of user data that Actions frequently request. These include classes representing the results for the various Actions on Google helper intents. You use helper intents when you want the Google Assistant to handle parts of the conversation where the user must provide input to continue the conversation.

Example: Confirmation helper results

The confirmation helper intent lets you ask for a yes/no confirmation from the user and get the resulting answer. The following snippet shows how your webhook can customize its response based on the results returned by the confirmation helper intent. For a more complete example, see the Confirmation class reference documentation.

// Create Dialogflow intent with `actions_intent_CONFIRMATION` event
app.intent('get_confirmation', (conv, input, confirmation) => {
  if (confirmation) {
    conv.close(`Great! I'm glad you want to do it!`);
  } else {
    conv.close(`That's okay. Let's not do it now.`);
  }
});

Example: Carousel results

The following snippet shows how your fulfillment webhook can customize its response based on the user's input for a carousel. The carousel component lets your Action present a selection of options for users to pick. For a more complete example, see the Carousel class reference documentation.

app.intent('carousel', (conv) => {
  conv.ask('Which of these looks good?');
  conv.ask(new Carousel({
    items: {
      car: {
        title: 'Car',
        description: 'A four wheel vehicle',
        synonyms: ['automobile', 'vehicle'],
      },
      plane: {
        title: 'Plane',
        description: 'A flying machine',
        synonyms: ['aeroplane', 'jet'],
      }
    }
  }));
});

// Create Dialogflow intent with `actions_intent_OPTION` event
app.intent('get_carousel_option', (conv, input, option) => {
  if (option === 'one') {
    conv.close(`Number one is a great choice!`);
  } else {
    conv.close(`Number ${option} is a great choice!`);
  }
});

Configure conversation response objects

The client library provides conversation response classes that represent rich responses or multimedia elements that your Action can send. You typically send these responses or elements when users do not need to give any input to continue the conversation.

Example: Image

The following snippet shows how your fulfillment webhook can send an Image in a response which will be attached to a BasicCard response automatically by the library:

app.intent('Default Welcome Intent', (conv) => {
  conv.ask('Hi, how is it going?');
  conv.ask(`Here's a picture of a cat`);
  conv.ask(new Image({
    url: '/web/fundamentals/accessibility/semantics-builtin/imgs/160204193356-01-cat-500.jpg',
    alt: 'A cat',
  }));
});

Make asynchronous function calls

The Actions on Google Node.js client library is designed for asynchronous programming. Your intent handler can return a promise that resolves when your fulfillment webhook is done generating a response.

The following snippet shows how you can make an asynchronous function call to return a promise object and then respond with a message if your fulfillment webhook receives the 'greeting' intent. In this snippet, the promise ensures that your fulfillment webhook returns a conversational response only after the promise for the external API call has resolved.

In this example, we're using a fake API for getting the weather data.

/**
 * Make an external API call to get weather data.
 * @return {Promise<string>}
 */
const forecast = () => {
  // ...
};

app.intent('Default Welcome Intent', (conv) => {
  return forecast().then((weather) => {
    conv.ask('How are you?');
    conv.ask(`Today's weather is ${weather}.`);
  });
});

The following streamlined code snippet has the same effect, but uses the async await feature introduced in ECMA 2017 (Node.js version 8). To use this code with Cloud Functions for Firebase, make sure you are using the correct version of firebase-tools and have the correct configuration.

app.intent('Default Welcome Intent', async (conv) => {
  const weather = await forecast();
  conv.ask('How are you?');
  conv.ask(`Today's weather is ${weather}.`);
});

Note: When using Cloud Functions for Firebase, global variables may be cached across function executions. For more information, read about function instance lifespans in the Google Cloud documentation.

Store conversational data

The client library allows your fulfillment webhook to save data in conversations for future use. The key objects you can use for data storage include:

• DialogflowConversation#data or ActionsSdkConversation#data: Saves data in JSON format for the duration of a single conversation session between the user and your Action.
• Conversation#user.storage: Saves data in JSON format across multiple conversation sessions.

The following snippet shows how your fulfillment webhook can store data in an arbitrary property you defined (someProperty) and attach it to the Conversation#user.storage object. For a more complete example, see the Conversation#user.storage class reference documentation.

app.intent('Default Welcome Intent', (conv) => {
  conv.user.storage.someProperty = 'someValue';
  conv.ask('...');
});

You can use the Conversation#user object to obtain information about the user, including a string identifier and personal information. Certain fields like conv.user.name.display and conv.user.email require requesting conv.ask(new Permission) for NAME and conv.ask(new SignIn) for Google Sign-in, respectively.

const {Permission} = require('actions-on-google');
app.intent('Default Welcome Intent', (conv) => {
  if (conv.user.last.seen) {
    conv.ask('Welcome back! How are you?');
  } else {
    conv.ask('Nice to meet you! How are you doing?');
  }
});

app.intent('permission', (conv) => {
  conv.ask(new Permission({
    context: 'To greet you personally',
    permissions: 'NAME',
  }));
});

// Create Dialogflow intent with `actions_intent_PERMISSION` event
app.intent('get_permission', (conv, input, granted) => {
  if (granted) {
    conv.close(`Hi ${conv.user.name.display}!`);
  } else {
    // User did not grant permission
    conv.close(`Hello!`);
  }
});

Scaling with middleware

You can extend the client library via middleware.

The middleware layer consists of one or more functions you define, which the client library automatically runs before calling the intent handler. Using a middleware layer lets you modify the Conversation instance and add additional functionality.

The Dialogflow and Actions SDK services expose an app.middleware() function that allows you to add properties or helper classes to the Conversation instance.

The following snippet shows an example of how you might use middleware:

class Helper {
  constructor(conv) {
    this.conv = conv;
  }

  func1() {
    this.conv.ask(`What's up?`);
  }
}

app.middleware((conv) => {
  conv.helper = new Helper(conv);
});

app.intent('Default Welcome Intent', (conv) => {
  conv.helper.func1();
});

Export your app

To expose your fulfillment webhook for a web framework or serverless computing platform, you must export the app object as a publicly accessible webhook. The client library supports deployment to a number of environments out of the box.

The following snippets show how you can export app within different runtimes:

Example: Cloud Functions for Firebase

const functions = require('firebase-functions');
// ... app code here
exports.fulfillment = functions.https.onRequest(app);

Example: Dialogflow inline editor

const functions = require('firebase-functions');

// ... app code here

// Exported function name must be 'dialogflowFirebaseFulfillment'
exports.dialogflowFirebaseFulfillment = functions.https.onRequest(app);

Example: Self-hosted Express server (simple)

const express = require('express');
const bodyParser = require('body-parser');  

// ... app code here

express().use(bodyParser.json(), app).listen(3000);

Example: Self-hosted Express server (multiple routes)

const express = require('express');
const bodyParser = require('body-parser');

// ... app code here

const expressApp = express().use(bodyParser.json());

expressApp.post('/fulfillment', app);

expressApp.listen(3000);

Example: AWS Lambda API gateway

// ... app code here

exports.fulfillment = app;