Read Time: 5 min

Google’s New Smart Reply Feature for Inbox: Here’s How it Works


This week, Google announced it will be rolling out a nifty new feature to its Inbox app called Smart Reply.

The idea behind it is simple: after you receive an email to your inbox, Google will suggest—or perhaps more appropriately, predict—phrases to use when replying. Think of it like predictive text, for all of your email replies.

This shouldn’t come as a surprise, as when Google launched Inbox initially, it did so by positioning the app as your assistant. Well, here we are. Now you can outsource your email replies to…your phone.

What a time to be alive.

But, is this a cool, useful new feature? Or yet another “yay, technology” moment that’ll never generate any widespread adoption? Like..say, Google Glass?

Before you decide, let’s first take a look at how the technology behind Smart Reply actually works. It’s pretty fascinating.

The backstory

Earlier this year, Bálint Miklós—a software engineer at Google working to make Gmail smarter—approached Greg Corrado, senior research analyst at Google, about the possibility of automating email replies on mobile.

Why? Because replying to email on your phone is a rather tenuous experience. Small screens, small buttons, and fat thumbs (raises hand) makes the whole process a real test of your patience.

Frustrations notwithstanding, was automating replies a feasible aspiration?

“I said it sounded too much like passing the Turing Test to get our hopes up,” wrote Corrado in this tell all. “But having collaborated before on machine learning improvements to spam detection and email categorization, we thought we’d give it a try.”

And in the coming days, you’ll have that opportunity, as well.

How it works

The Smart Reply system connects a pair of recurrent neural networks, one used to encode an incoming email, the other to predict responses, to form a direct cycle.

Explains Corrado:

“The encoding network consumes the words of the incoming email one at a time, and produces a vector (a list of numbers). This vector captures the gist of what is being said without getting hung up on diction — for example, the vector for “Are you free tomorrow?” should be similar to the vector for “Does tomorrow work for you?” The second network starts from this thought vector and synthesizes a grammatically correct reply one word at a time, like it’s typing it out. Amazingly, the detailed operation of each network is entirely learned, just by training the model to predict likely responses.”

Impressive, sure. But not all emails are as short and simple as, “Are you free tomorrow?” Some are dozens of sentences, or even hundreds of words long. Here’s where the recurrent neural networks come into play.

Using a variant of a long-term-short-term-memory (LSTM) network, Corrado’s team was able to “preserve long-term dependencies, and home in on the part of an incoming email that is most useful in predicting a response, without being distracted by less relevant sentences before and after.”

Remember the predictive text analogy I made earlier? Well, the Smart Reply feature is actually far more complex in its ability to not only consume incoming content, but to also discern the context in which it was written without the distractions of irrelevant words and sentences.

The trickiest part in developing Smart Reply? Preserving your privacy. Because Smart Reply adheres to the same rigorous privacy standards Google has long held, engineers had to get machine learning to work on a data set that they themselves could not read. This ensures that no humans are reading your emails in order to offer predictive responses.

Which, according to Corrado, is “like trying to solve a puzzle while blindfolded—but a challenge makes it more interesting!”

What does this mean for marketing automation?

Much of the technology Google pioneers eventually makes its way into mainstream software, so to address the elephant in the room, what will this mean for marketing automation in say, five years?

Twenty years ago triggered drip campaigns weren’t even a thing. Ten years ago it was inconceivable to think you could segment based on the plethora of devices someone was using. Five years ago it seemed crazy that you could manage all of your inboxes in one application. Just last year, it may haven seemed crazy that you could instantly preview how your email would look across all the popular inboxes and devices.

Five years from now, will customer service representatives even have to give thought to their response? (Sure, many organizations use templated scripts for their reps, but human thought and both qualitative and quantitative analytics still had to go into their creation. Will a machine now take care of this?)

Software is getting smarter. Google is getting…Googl-ier? Predictive text is getting so [d]ucking smart, that now it can understand context?

Maybe it’s the cynical consumer in me, but I can see this type of technology having a negative impact in customer service and sales, two areas where more human interaction is so desperately needed on the enterprise level. But who knows, maybe I’m taking it all out of context. I’ll leave it up to software of the future to both understand and remedy that.


So what do you think? The Smart Reply feature is rolling out to iOS and Android this week. Will you be trying it?

Let’s stay in touch

All our latest content for email marketing pros delivered to your inbox a few times a month.

John Bonini

John Bonini

John Bonini was the Growth Director at Litmus