Dual Cat
How do you implement user research from scratch in a small mobile game studio?
Sept. 2021 → Jan. 2023 (1 y and 3 months)
Games User Researcher - Employee
Context
Dual Cat is a development studio that publishes mobile video games in the Hypercasual category: very simple and short games whose economic model is based primarily on advertising (this position has since evolved).
I was hired to fill a completely new position. After the increase in revenue triggered by the first successes (see examples below), a recruitment strategy was put in place and user research was one of the investment areas.
User Research had been the subject of some effort in the past, but no one was exclusively positioned in this role. Nevertheless, this showed a certain interest in this philosophy and its potential added value.
Long Neck Run
Bubble Tea
My missions
Develop a fast and robust user research process to follow the development process of each game.
Develop player knowledge through the development of personas. These personas should allow for better acquisition through more relevant advertising content.
Evangelize the applications of UX and psychology in the context of video game development.
“I had the pleasure of working with Matthieu for 18 months at Dual Cat, where he was in charge of several UX projects. During this time, I was constantly impressed by his ability to work independently and manage partners effectively. Matthieu played a key role in industrializing our testing process by bringing a solid theoretical foundation to our work, with a particular focus on pedagogy. He is passionate about both video games and psychology, which makes him particularly well qualified for user research in this field. In short, if you are looking for a user researcher who can propel your product to success, I highly recommend hiring Matthieu. He is a talented and dedicated professional who is sure to have a positive impact on any team.”
Alexandre Aminot, Co-founder, Dual Cat
“Matthieu’s research with PlaytestCloud demonstrates innovation and best practice within Games User Research. He has also provided valuable feedback that our product team have incorporated into new features. From discussing research goals, planning projects, negotiating subscriptions; Matthieu has consistently operated with excellent communication and professionalism.”
Peter Joyes, Senior Business Development Manager, PlaytestCloud
Methodology
Outcome
In a short time I was able to demonstrate what user research could bring to the development of mobile games. I made significant progress in each of my missions:
A fruitful partnership was sealed with an external partner: PlaytestCloud. It allowed the studio to test more than 50 games including chart topping titles totalling more than 300 Million downloads.
A user testing process that met expectations was established and went through several iterations. Time spent on each game was reduced by more than 50% compared to an average project of the same type (evaluative research like user tests).
Two iterations were performed on the personae. In total, 15 were made.
A new in-game survey tool was developed and tested. It performed well above average with a 55% response rate.
Intelligent methods of evangelization have been found and applied. 3 impactful workshops were performed on the topic of Psychology in Game Design to dev teams.
— More details below! —
Finding the best solution for user testing
Tests needs
The testing requirements were as follows: for each new game idea to be developed two user tests had to be performed, a concept test and a playtest. If some games were significantly successful then additional playtests could be scheduled.
The problem
Thanks to my experience in research, both academic and applied, I was able to anticipate that recruiting the necessary users to meet these needs would be impossible without outside help.
An external partner
Unlike bigger studios Dual Cat didn’t have open channels to access potential users directly. Building them up would take a significant amount of time. In order to solve this time constraint I quickly recognized the need for an external partner. I quickly came up with the following plan of action:
Conduct a comparative study of these potential partners
Build relationships with each
Negotiate proposals
In my study I considered Userlytics, Testapic, Tandemz & PlaytestCloud.
PlaytestCloud had a key advantage: it is a structure specialized in video games and therefore has relevant and practical options such as the possibility to directly target the desired player profile with many readily available variables without having to go through a pre-selection questionnaire. It was also a company that had a steady growth and were updating their tools on a frequent basis with options that could prove useful on a longer term.
Which modality for the tests?
Doing a first iteration of a new process with unmoderated and remote tests seemed to me to be a good basis. The objective is to be able to iterate on these methods to master the process and then add moderated tests.
This approach allowed me to quickly experiment:
Different test protocols
Different ways of writing test reports
Different ways of presenting and articulating the information to be transmitted
Urgency of first personas
The problem
The marketing team quickly expressed the need to have a first version. This need was catalyzed by the loss of advertising identifiers (IDFA for Apple / AAID for Google) that no longer allowed for performance verification of this type of content, which disrupted user acquisition.
The potential contribution of personae
A proposed strategy to adapt to this major change is to study player motivations specific to the identified player base (not just behavior) and transform those into relevant ads.
Source : Brain Labs Digital
A first version
With the time and resources available, I could only hope to provide "proto-personas" (according to NNg's classification), that is, basic personas compiled from two sources of information immediately available:
feedback from experienced team members and leaders
data published in the various reports of the relevant structures
By conducting this quick phase of desk research I was able to define 5 personae in a few weeks (example below).
Helen, the fun matriach
How to go further?
After being used for a few months, I looked to build a second, more complete version.
To achieve this, I took advantage of the possibility in PlaytestCloud to include in each test (whatever the chosen method) a series of 10 questions. A part of these questions was reserved for what happened during the test but the rest could be used to collect data on the players' motivations. As the tests were very regular, I was able to quickly build up a large database for analysis.
The testing process was therefore optimized to serve both the specific needs of each game and the more global user research needs.
By skimming more and more reports and cross-referencing them with my database, I have built 10 new and more specific personas. I was also able to develop a more accurate method of tracking their use.
This new version has been presented to the whole studio.
Arlo, the aspiring student
Another initiative
In-game survey (IGS)
To be able to extract feedback from the experiences of players present on Dual Cat games, the options were limited. Some larger and older studios can leverage their visibility to establish a direct dialogue via forums or Discord servers. These studios have a community and it is an excellent source of data (even if it should not be the only one).
However, an option remains : interviewing players directly in the game via a questionnaire that appears after a level has been completed.
I was able to conduct this project by prototyping the tool (see examples below). A first successful test took place: the response rate was 55%, which is an excellent result compared to the usual numbers (33% or 13% in the questionnaires posted in apps in general).
Low-fidelity two-page mock-up that was tested.
Relevant evangelization
Knowledge with impact
One of the most important lessons in psychology is that learning is easier if it is done in context.
A good way to transmit the most relevant knowledge for game design was therefore to highlight it in the games played by the developers.
In order to obtain a selection of the most appropriate knowledge, some books represent a good selection of the information to be transmitted (see examples below).
Gamer’s brain (2017) - Celia Hodent
Bottlenecks (2017) - David C. Evans
I was able to do about ten mobile game analyses, each time illustrating 2 or 3 notions of UX, psychology or behavioral economics, the latter being extremely important for monetization issues.
I was also able to provide more in-depth analysis on hybrid games (halfway between hypercasual and casual games) in order to best support the studio's transition towards the development of this type of game.
Key lessons
Arriving in a brand new position is an important challenge: almost everything had to be built from scratch and it was an opportunity to deal with a series of constraints. Time and budget constraints but also more theoretical ones with new research questions with little available data (hypercasual is a new and unique segment). Does the classical knowledge in psychology apply in this ecosystem in the same way as it applies in the rest of mobile games?
I was able to successfully confront the creation of personae, a difficult exercise with which I was not necessarily comfortable before.
I was able to have a very strong impact on the developers thanks to two elements. First of all, a playful and efficient evangelization of the knowledge on UX and psychology for game design. Secondly, the iterations on the test reports allowed me to find a clear and striking formula with carefully selected illustrations.
I was also able to progress on Notion and on how to use such a tool to facilitate the organization, sharing and centralization of knowledge, which is a key issue in many structures. This was the opportunity to integrate a atomic research approach through the implementation of a research repository.
Thanks to this experience, I was able to understand which areas to work on to improve my user research practice. The constitution of a better research repository and a better measurement of the impact of this research are two major points.
Finally, thanks to this experience I was able to demonstrate my ability to work in autonomy and stay motivated. However, and this is undoubtedly the only negative point of this experience, I would have appreciated working in a team so that I could more easily have an outside perspective on my work and exchange ideas.
Conclusion
Unfortunately, my adventure at Dual Cat ended due to an unfavorable global situation, the studio having to restructure like many others. Nevertheless, I managed to make significant progress on all my missions in a time that was rich in lessons and memorable people.