27
.
10
.
2016

Building a Generic Questionnaire Bot for Telegram Messenger

Can't be that hard, can it?

Jumping On the Bandwagon

The vast growth of messenger apps in the last years could not have been overlooked. Their usage has surpassed classical social networks and one fourth of the 30 most widely used apps in the US are made up of messengers. Although messaging tools are widely used by a majority of internet users as a crucial part of private communication, there are still emerging possibilities hidden in the seemingly trivial messenger tools. What started as simple text messaging evolved into rich communication, including images, videos, sketches, voice etc. Using those messenger chats for interaction bears the opportunities for a better user experience, not only for private users but also for businesses.

For example, the dutch airline KLM has swapped out part of their flight documentation and customer service channel to Facebook’s Messenger App. What previously had to be managed via e-mail and website forms - or worse: telephone calls to the customer service - is now integrated into the user’s messaging environment. This enables the user to communicate with the service as if it was just another chat, more personal than email, more convenient, flexible and user-friendly than using the browser. So why not use a messaging tool to let the user fill out questionnaires?

We wanted to explore this possibility by writing a bot for the Telegram Messenger App, as Telegram offers to add free bots that behave like normal chat users. They have a username by which you can add them and then chat with them. Therefore, the bots process the messages the user sends them and can reply with text and custom keyboards that facilitate answering to questions. We used this functionality to build a bot, that lets the user choose a questionnaire, answer the questions with yes or no and then receive a result based on the given answers.

Let’s Get Started

To implement our idea, we built a Spring Boot application and used the TelegramBots package written by Ruben Bermudez to communicate with the Telegram Bots API. Ruben Bermudez also provides a repository with several examples to illustrate the usage of his bot package, which is very helpful to get started.

Before we start writing code to control the bot’s interaction with the user, we first have to create a bot account. The easiest way to do it, is to install the Telegram App on your phone and start a chat with @BotFather. By following this bot’s instructions, you can create a bot and add a title, description and commands. After creating your bot, you get a token you need to later communicate with the Telegram Bot API. Be aware that the token acts like the login data to your bot, so make sure it’s kept secret and not pushed to any public repository.

After creating our bot @QuestomatBot, we can already chat with it, even though we haven’t written a single line of code yet. So the first screen looks like this:

Now we can start to code our bot. Therefore, we add the following code to our main method:

github:54ffb096e70a8a293a40bd7f499fe238

Going Into Detail

The telegram API provides two kinds of communication mechanisms: Webhooks and Long Polling. We decided for the latter because for our example the drawbacks of polling seemed reasonably small. The package we used has a base-bot for each kind of mechanism. The custom handler we wrote simply extends the basic bot for long polling and overrides three methods: getters for each of the bot’s username and token as well as one crucial method (onUpdateReceived) where all the magic takes place:

github:e14ba44534b105eb4c7350939913efb7

Receive and send messages

The onUpdateReceived method is called whenever a user sends a message to our bot. The exact message, that was sent to our bot can be found in the Update object. Now that we can read the message the user has sent us, we can distinguish different cases. For our scenario, there are two possible cases:

  1. The user starts the conversation with the bot, which is usually done with messaging “/start” to the bot.
  2. The user has already started a questionnaire. We identify this case by checking if the update contains callback data. More on this later.

All other cases are ignored.

github:40f67d8dd6f36388a09d520d61cdb74c

Where to store the data

After the basic setup, we encountered one big question: Where can we store the data and especially the state? Alias how do we know which questionnaire the user has chosen and what questions the user has already answered? As the only thing we can work with is the chat id and the current message, we needed to find a way to keep track of the current state. Most examples therefore use a database on the server side to store the user’s input and retrieve it for every request. This might be a quite reasonable idea for larger bots, but as we only ask the user some questions, we wanted to keep all data on the client side.

Therefore we looked for a way to enhance each request by adding additional information. As normal buttons only send the text they show, we decided to use Telegram’s inline keyboard markup. It consists of a list of inline keyboard buttons. The buttons get a json object as parameter which holds the button text as well as optional callback data that can be processed by the handler when a button was pressed. In our case, the callback data holds not only the value of the current button but additionally an ordered list of all previous given answers. A drawback of the callback data is its limit to 64 bytes, which forced us to store the data in a very compressed way. As we only use yes/no answers in our questionnaire, we decided to map them to 1 and 0. How this works in detail is discussed in the following.

Handling the initial message

In order to react to the user’s wish to start a conversation with our bot, we use the SendMessage object. First we have to set the chatId to the one that is held in the previously received message object. Then we set the text to whatever we want to reply. In the third step, we set the replyMarkup. This means, that we not simply let the user answer with a regular message, but have the possibility of providing a custom keyboard containing predefined values, the user can answer with.

github:a4185cecf222e7f055f2f6a0f16a2956

Handling the Questionnaire

As in the initial message handling, we create a SendMessage object and set the chatId. In the next step, we use the callbackData from the message to read the index of the chosen questionnaire. We use it to retrieve the right questionnaire and get the next question by the amount of already answered questions. If all answers were given, the final result of the questionnaire as well as a restart text are sent to the user.

github:02d2ccd27b4de0fbd8d774c21c7182df

Using Custom Keyboards

We use two different keyboards: One for the choice of the questionnaire and one for the answering of the specific question. The questionnaire keyboard contains a button for each available questionnaire. Each button holds the questionnaire’s index and a separator in the callback data. Later on, we use the same data to concatenate the user’s answers.

github:a75f497255195964675eb37b0081040a
Telegram questomat bot - choose questionnaire
Telegram questomat bot - choose questionnaire

The second keyboard is used for simple yes/no answers. It uses the callbackData from the previous message and concatenates a 1 for yes and a 0 for no.

github:64b2d8ddff49b10353728cbd76298f45
Telegram questomat bot - answer questionnaire
Telegram questomat bot - answer questionnaire

Testing Our Bot

As a side-effect of the Long Polling, we can start our application locally and nevertheless communicate with the users online. After gaining the first users, you can also create a second bot just for testing. This nice opportunity can be widely used to test the bot’s functionality. But with a growing variety of offered commands and interactions, it’s good to add unit and integration tests to make sure the app works as expected. The TelegramBots package doesn’t yet support bot testing, nevertheless we can write normal unit tests for our classes and a kind of integration test which tests the onUpdateReceived(Update update) method of our bot handler. For larger bots it would be recommendable to also write some testing support, so we could easily generate user interactions and use them to test our implementation.

github:41ff2a63326121444786a0cd41a7aedd

Uploading to a Store

Although there is no official store, you can upload your bot to storebot.me so other Telegram users can find it. It also used a bot for the registration, so open the Add Bot page on your mobile device with Telegram installed and follow the instructions. The final approval will take 2-3 days or sometimes a bit longer. Nevertheless, your bot can already be tested and tried out by directly writing a message to its Telegram account name, in our case: @QuestomatBot.

Lessons Learned

Building our first messaging bot, our expectation of usefulness for a flexible and user-friendly interface got confirmed. Even though we went through the questionnaires many, many times for testing, it was still fun to do, as it really felt like a conversation with a human person. Of course, the variety of possibilities was very limited, but even getting different meal suggestions was fun and offered a preview of what might be possible by adding more artificial intelligence then just a random selection of different lists of dishes.

In our small use case with very clear options, we didn’t encounter any problems with hard to validate user input, as we just used yes and no buttons. But even looking at a simple weather app, we can already get a feeling of how difficult it could get to validate user input and guess the right meaning. In this perspective, bot interaction is very similar to voice control: if the system can’t understand what the user wants, the user can easily get stuck and abort the interaction. Especially, as the user will also have less overview of the possibilities the system offers. This might give users also a feeling of not being in control, which can get frightening and decrease satisfaction dramatically.

A weakness we encountered during development is related to privacy and security issues. As the token we get from the BotFather is the absolute access token, anyone who gets hold of it can access the chat streams and evaluate all not yet processed messages. For next versions we hope Telegram will at least offer some kind of restricting domains, so our token can’t be misused that easily. As most bot examples we saw saved their data in some kind of server-side database, this might also lead to vulnerability. Adding some kind of user hint to which server messages will be sent, might also increase privacy and enable the user to determine possible risks.

There are also other Telegram Bot API implementations available, for example JavaTelegramBot-API written by Zack Pollard, offering four example bots using it. Also the TelegramBots4Java by Pieter looks very promising, as it uses annotations to delegate the user’s interaction to the right method.

TL;DR

  • Messaging bots will be part of our future, so let’s be part of it
  • Setting up a Telegram bot can be done in just a few hours, thanks to different Telegram Bot API implementations and examples online
  • Sending additional data with each interaction is limited to 64 bytes and only possible with inline keyboards
  • Saving data and state sever-side is very common
  • Testing is not yet supported in most API implementations, but basic testing is possible
  • Bots can be uploaded to storebot.me, even though it’s not an official store
  • A good user experience doesn’t come for free, especially creating a feeling of control for the user can be really hard
  • Think about privacy and security issues

Learning more

For further information, have a look at Telegram’s ‘Bots: An introduction for developers’ article. If you would like to explore some existing bots, have a look at storebot.me.

If you have any questions or suggestions, feel free to comment or contact us: Jeannette Schwarz or Elisabeth Engel