[OBSERVE: Use this field to inform the user (in your language) that the Join us page is only available in english]
Artificial intelligence analyses open text responses to a customer survey
Many customer satisfaction surveys ask for numerical reviews and provide an opportunity to clarify responses in open text fields. With more than 10,000 responses accumulated to the customer, the volume was too large to be analysed manually. Cybercom built an artificial intelligence tool for analysing responses, which will enable it to quickly analyse responses and address grievances also in the future.
Cybercom built an artificial intelligence solution to analyse a client's customer satisfaction survey. Most of the survey questions were formed with a Likert scale and an open text field to clarify responses. The most important task was to analyse text comments and look for reasons why the customers of a particular service were more dissatisfied than others.
More than 10,000 responses had already been collected, so it was no longer possible to go through it manually, even though it had been tried and some findings had even been made. However, manual analysis of text data is very time consuming and also prone to human error. In addition, a similar survey is repeated annually. Therefore, the aim was to build the solution at once so that it would also serve future inquiries and save a significant amount of working hours in the future.
In addition to interpreting the answers, data analysis will help to formulate questions in the best possible way in the future. Above all, it will guide to ask just the right things. Once the groundwork has been done, the results can be analysed more quickly in the future and with it, we can focus on those issues that are important to the users of the service.
Erkki Ruskio, Customer Service Manager, Cybercom
NLP technology was used to look for recurring factors
The analysis sought to examine whether there are explanations for weaker numerical grades in open text fields. Artificial intelligence was used to answer questions such as the following:
- From the last question (Your other greetings), compile the ten most frequently mentioned or most important factors for users.
- Which factors mentioned in the free text comments influence the formation of good (> 3) grades?
- Which factors mentioned in the free text comments influence the formation of negative (<3) grades?
To investigate the first question, a baseline was formed using the traditional ngram method, which wanted to know if there are significant factors in the text field in the first place. It found e.g. common word pairs, so it was possible to divide the text into subject areas. FinBERT, based on neural networks and modeled using Finnish texts, was chosen for the distribution of data. It allows each text to be projected into a space of about 700 dimensions, allowing similar subjects to be grouped using traditional classification methods. As the text included comments in Finnish as well as in Swedish and English, a similar multilingual LaBSE method was also tried. It was discussed with the client how many topics were to be considered.
"Finnish is an agglutinative language, i.e. many different parts can be added to the word body. Therefore, to teach ideal models would hypothetically require all wordings in all different contexts. Because the material used for teaching is always limited, the performance of the models is also limited", says Petri Puustinen of Cybercom, who worked as a data scientist in the project.
The LaBSE method managed to isolate sufficiently good topics, because it also brought English and Swedish topics into the same areas as the Finnish texts. With regard to follow-up questions, the challenge was to link the formed topics to negative and positive grades.
It was decided to use NLP or Natural Language Processing technology to analyze the answers. Machine learning and natural language processing make it possible to analyze large amounts of information quickly and cost-effectively.
The technologies seldom know Finnish
NLP technologies have developed significantly in recent years and can be used to make processes considerably more efficient. Although ready-made solutions are directly available for larger languages, the Finnish language skills of NLP solutions are still very limited. In this implementation, another challenge was that the answers were given in three languages and some of the vocabulary used was internal to the client.
With current machine learning / artificial intelligence methods, it is not possible to fully “understand” the text at all. One challenge was to find a suitable clustering method for this data.
The result was a clear presentation of the correlations
As a result of the analysis, it was found that the answers and grades were indeed distributed according to certain conditions. The end product was a clear presentation in which the findings were presented. Data analysis was found to be useful and will be continued with future surveys.
"In addition to interpreting the answers, data analysis will help to formulate questions in the best possible way in the future. Above all, it will guide to ask just the right things. Once the groundwork has been done, the results can be analysed more quickly in the future and with it, we can focus on those issues that are important to the users of the service", says Cybercom's project manager Erkki Ruskio.
The analysis helps to catch up more quickly on the problems behind the answers. Making changes accordingly will increase users' motivation to continue responding to surveys.