TL; DR
When people can search using their voice, they tend to ask questions conversationally, like how they would ask a human. They also then ask for more subjective criteria. This is important because users might make a voice query that cannot be answered with the available objective data.
What did they do?
The researchers built a voice prototype that lets users search for movie recommendations on Movielens. They asked 95 Movielens users (to make sure they knew how the site worked) to try using their voice search tool. Users could see their question’s transcription and fix it if there were errors. The researchers compared voice searches to typed searches from other participants in this study.
What did they find?
Voice questions were longer than typed questions (19 characters vs 12.5 characters). 40% of the questions asked using voice were in a conversational format (ex. “I’m looking for a hard sci-fi movie”) while only 4% of typed questions were conversational. They also found that participants took more time to ask their questions when they used voice compared to typing (39 seconds vs 17.5 seconds). This slower response was probably because participants could review how the program transcribed their speech, so they spent time fixing the transcription.
They found that when users searched using voice, they would ask for more “objective deep features”; things like plots twists. This was a problem because these types of movie features were not in the database. They also found that users searching using voice using more “subjective movie-based features”; things like “I’m looking for a movie like Top Gun“.
So what?
These results suggest that when users are given an open prompt and can use their voice, they’ll ask questions the same way they would ask a person. This is a problem because users might ask a question that cannot be answered with the database’s information (ex. movies with plot holes). Voice designers should keep this in mind and make sure to guide their users so that they are more likely to ask questions that can be answered.
Article citation
Kang, J., Condiff, K., Chang, S., Konstan, J. A., Terveen, L., & Harper, F. M. (2017, August). Understanding how people use natural language to ask for recommendations. In Proceedings of the Eleventh ACM Conference on Recommender Systems (pp. 229-237). Full article.
