Mobile Game Doctor Logo

How to Run Effective A/B Tests: 12 Steps to Success

Leona Jasa • Jun 28, 2023

How to Run Effective A/B Tests: 12 Steps to Success

Leona Jasa • Jun 28, 2023

A team developing a pizza-themed game is currently facing a dilemma. They are considering whether providing new and unique toppings in a real-money starter pack would improve player conversion, engagement, and satisfaction. Some members of the team believe that this is the best direction to take and suggest increasing the price of the pack to ensure that the player perceives the value of the pack to be high. They believe that this change will not have any negative impact.



However, other members of the team disagree. They argue that monetizing pizza toppings in this manner could potentially turn players away. They suggest that separating the changes would be more effective in ensuring that each modification yields positive and measurable results.

Game and product teams are often faced with dilemmas like these. Improving KPIs is always a big focus, but so is keeping the game fun and making it enjoyable. In this instance, the challenge is to determine the optimal pace of change: whether to implement a significant change quickly and “get on with it” or proceed slowly and cautiously. The best way to resolve this issue is through A/B testing.


What is A/B testing?

A/B testing has been the standard practice for making changes to gaming products for a long time. Although it was originally associated with mobile, today it applies to various types of games on almost all platforms, except for those whose publishing policies make it inherently difficult. The primary goal of A/B testing is to generate positive trends in KPIs. By testing different versions of games against two or more audience cohorts, developers can objectively measure how changes impact their KPIs and determine the best course of action.

However, while the purpose of A/B testing is straightforward, implementing it as a product management methodology for games can be highly complex. Game studios that are new to A/B testing frequently make numerous errors, from overcomplicating their test conditions to improperly applying statistical analysis to evaluate results. Effective A/B testing requires significant rigor, patience, and a willingness to interpret test results honestly, rather than rationalizing negative outcomes. It also requires a high level of clarity.


In our experience, the most effective approach to A/B testing is to have a clear understanding of the questions you’re trying to answer, and for the team to agree on the objectives of those questions. Once you have a clear understanding of these two things, A/B testing becomes a systematic process in which game studios meticulously vary specific factors within their game and analyze the outcomes. Typically, studios only modify one factor while keeping all other internal and external factors constant to determine the impact of a singular change.

Of course, there are many additional considerations to keep in mind when implementing A/B testing. For instance, it’s crucial to have knowledge of the KPI roadmap, understand the gravity of the impact of a specific change, and be aware of the available remote configuration variables that enable the testing of particular aspects of a feature. Some other important considerations throughout the A/B testing process include:


Building a testing plan

To develop an effective A/B test plan, it is essential to craft clear, focused questions that have measurable answers. These questions will guide the A/B testing process and enable the team to evaluate the results objectively. Good questions to ask could include whether changing the price of an item in the game would lead to an increase in visits to the market. On the other hand, a question that cannot be definitively answered would be whether players like the change. By being clear about the questions being asked and the goals of the test, the team can design an effective test that yields meaningful insights.


Cross-team cooperation

If there are features required for your A/B test, it’s important to loop in the engineering team early to give them sufficient time to prepare for any necessary features or changes that may be required for the A/B testing process.


Tracking documents

Create a central tracking document to chart out your plan. This document should include a list of your A/B tests, ordered by priority, along with the start and end dates, platforms, regions, the hypothesis being tested, KPIs, variants, results, and any relevant links. It’s important to establish this document early on in the process so you can keep track of all the necessary information and easily update it as needed. To maintain consistency, it’s recommended that you use a naming convention that includes both the date and the topic, such as “economy1b_2022620”.


Run one A/B test at a time

If you’re just beginning to use A/B testing, run one at a time and see how the process works before trying multiple tests.

When you do run multiple tests, always ensure that the tests do not touch the same parts of your game. Do not, for example, run two separate tests on grain pricing simultaneously. If you do, the results will corrupt each other and leave you with no useful information. Good separations include things like testing changes on entirely separate levels. Poor separations include things like testing product price and pop up offers at the same time. 


User group variants

To ensure accurate results, it’s important that the user groups in A/B testing do not overlap in any way. For instance, if some players in a multiplayer game experience a change in matchmaking while others do not, it is important to avoid matching them together in battles. Doing so would compromise the integrity of the results and make them unreliable. Therefore, it is necessary to carefully design the user groups and ensure that they are completely separate from each other during testing.


Limiting the number of variants

To obtain quick results, it’s advisable to restrict the number of variants you test simultaneously. The reason is that the more variants you test, the more users you need to generate reliable results. Therefore, if time is a concern, testing fewer variants will likely produce more conclusive outcomes.


Testing new users

To ensure you have enough new users for your test, it’s important to loop in the marketing/user acquisition (UA) team early. This is especially true if you are testing first-time user experience (FTUE) completion rates or changes aimed at improving early retention. In such cases, you’ll need a user acquisition plan that can provide you with qualified users in sufficient quantities when you run the test.


Simultaneous tests

To ensure clear and accurate results, avoid starting new tests while others are still running. Always make sure to stop all current tests before starting new ones, as this can lead to muddied or unclear results for both sets of A/B tests.


Split tests by platform

To ensure accurate results, it is important to split tests and test results by platform, such as Android versus iOS. There can be significant differences in how users from each platform interact with a game due to various factors, such as culture, demographic, and economic status. When tests are mixed between platforms, the results may become corrupted, leading to inaccurate conclusions.


Don’t end your tests early

It is important to not stop an A/B test before the results are available. For example, if you are testing the impact of a change on day-14 retention, you should run the test for the full 14 days after attaining a sufficient number of users. If the test is stopped prematurely, the users who experienced the change may be returned to the general population, altering their behavior compared to existing users and invalidating the day-14 test data.


Involve your community

To prevent player dissatisfaction and avoid derailing player strategy, it’s important to inform your community when changes are being made. Even if you end up choosing the control variant, it’s still important to message players in the variant and take the necessary steps to ensure that they don’t have any issues shifting back.


Adjusting the A/B test

Finally, it is important to remember that not all A/B tests are created equal, and therefore each of the factors mentioned above should be considered and weighed for each individual test, rather than assuming that they will all work in every situation.

In conclusion, A/B testing can be a powerful tool for game developers looking to optimize their game and improve player engagement. However, it’s important to keep in mind the various factors that can impact the validity of A/B test results, including user group overlap, test duration, variant limits, and platform differences.



By following the tips and best practices outlined in this guide, you can ensure that your A/B testing process is as effective as possible. Remember to start with a clear hypothesis, create a tracking document, limit the number of variants, loop in marketing/UA, and test each variant by platform, among other key steps.


While it’s true that not all A/B tests are the same, by weighing these factors for each test and carefully analyzing the results, you can make data-driven decisions to enhance your game and create a better player experience.



By Johnny Liu 07 Mar, 2024
Dive into the world of mobile gaming monetization and engagement systems with our Product Management Consultant, Johnny Liu! In this insightful video, Johnny presents a short case study comparing the monetization and engagement strategies of two popular titles: Honkai Star Rail and Genshin Impact. Gain valuable insights into how these differences impact user experience and revenue generation.  Watch now to uncover key learnings for optimizing your own game's strategies:
By Mike Kanarek 22 Feb, 2024
I spend a lot of time working with small and medium developers who are trying to get started on UA, or who have started but have hit roadblocks and face uncertainty about how, where, and how much to spend. Larger developers make performance marketing look complicated and challenging, because at large scale where fractions of a percent can mean whole annual salaries in revenue won or lost, it is For small and medium developers establishing a UA program, it is a lot simpler just to get started. There are major questions that need to be answered before you build enough scale to start obsessing over every marginal gain. So - what channels should you advertise on, and what geos should you focus on? The answer depends upon many factors, so seeing where others in your genre (your competitors) have succeeded allows you to take advantage of some of the learning they’ve already paid for. Below, I describe some of the tools and processes that can help clients evaluate the right channels and geos for their marketing efforts. Who are these tools for? These will be especially useful to anyone who’s got a limited budget, or is looking for a place to start. Larger developers, living in a world where a few percentage shift in CPI could generate tens of thousands in additional revenue at scale, will likely have additional resources to devote to extensive competitor research. But even in that scenario, the kind of research I describe below can help a team save a lot of time by giving some starting points. These will also be useful if you’ve run through the most common big name ad networks (Meta, Google, etc.) and are trying to choose between the dizzying array of other networks to test on next. RpD: Benchmarking and geo analysis Look at revenue over the last 30 days and downloads over the last 30 days. This will tell you how much revenue per download your competitors are getting. This is a useful benchmark because it can help set expectations about average CPIs across networks. If your competitors are making $5 per download, they’re probably not afraid of a $4 CPI. - This gives you an idea of the general health of your competitor by comparing current RpD to all time RpD. If there’s been a slowdown in growth your competitor may be struggling, look at different time periods to see which networks they’re sticking with and which ones they spin up and scale down in a hurry. Those networks that your competitors abandoned are probably less likely to find your game appealing. - If you can see other parameters in your tool of choice, try to break it down further. What’s RpD by country? You’re probably most interested in T1 geos. In the example below for Coinmaster note that RpD is highly distorted by a huge volume of installs in Vietnam, which is less than 2% of their revenue. RpD for US is closer to $42, but the overall average RpD appears to be about $12. Typically, you’d want to find networks that have a higher RpD, although variability in CPI from network to network still needs to be factored in to get a complete picture.
By Dave Rohrl 01 Feb, 2024
In a recent conversation, someone asked me a question I had never heard before - “What is your favorite failure?” Over the course of a long career in game development, I’ve made a lot of successful games. But I’ve also made a number of unsuccessful ones, and even some that never saw the light of day. So having nearly 30 years of experience to think about gave me a lot to digest. After reflecting on the question for a few days and reviewing the various failed projects I’ve worked on, there was one in particular that stood out - both for the reasons for its failure and the lessons I learned from those going forward. Back in 2003, I had founded and produced and designed the first few games for Pogo To Go, Pogo.com’s downloadable PC game channel. Some of those games were modest successes, some were hits, and some were misses. There was enough going on in the business to justify a couple of new hires so we could release more games, so I expanded the team with a pair of producer/designers. One of the new hires had come out of the edutainment industry and didn't have a strong background in pure gaming. Around that time, we had been talking about making a version of Puzzloop , a simple puzzle game where you shoot colored balls into a spiral to make matches. We talked to a number of developers, and settled on a new team that I hadn’t worked with before, but who had done a number of games for my previous employer, The Learning Company. Before we kicked off the project, we wanted to give it an innovative twist. Around the office, we had been playing an emulated version of an obscure Japanese arcade game called Money Idle Exchange . This was a more complicated puzzle game that involved pulling coins down from a grid then tossing them back up to merge them into coins of higher denominations. It was a bit of a niche game, but fun and different. We decided to combine both of these mechanics into a single game with a charming cartoon pirate theme. The game’s original working title was Pirate Booty, but one of our advisors convinced us to change the name after a quick Google search. The game’s final name was Swashbucks , and we put it into production with the new developer under the new producer’s supervision. What could go wrong?
By Mike Kanarek 27 Dec, 2023
Dave Rohrl The biggest challenges in mobile games in 2023 were the continued impact of Apple’s privacy changes and a sour macroeconomic environment. Apple’s policies made it far more difficult for marketers to precisely target high value users on iOS, which tends to be a better monetizing platform than Android. And concerns about the macroeconomic environment have led to both a dramatic decrease in venture capital funding (especially in games) and a significant increase in commercial interest rates. These changes have made it much more difficult for developers to afford to spend money to acquire new users whose costs they may not recoup for 180 days or more, especially if they need to fund that user acquisition through debt rather than cash in hand.  In an environment where user acquisition at scale is difficult to execute, designers and product managers need to have a strong focus on retaining users and keeping them consistently engaged. When new users are too expensive to acquire efficiently, mobile game businesses perform best by keeping the players they already have playing and - as a natural result of playing free-to-play games over time - paying. This means making sure that your content teams are generating a steady stream of content on time and at high quality and that your development teams are creating and perfecting critical features that will keep entrenched players coming back for more. When every churned player has an unacceptable replacement cost, long-term retention becomes by far and away the most important aspect of your product.
By Mike Kanarek 20 Dec, 2023
In the first part of our learnings from 2023, this blog takes a reflective look back at the key learnings and insights gained by the Mobile Game Doctor team in 2023. It's been a year of significant growth, challenges, and innovations in the mobile gaming industry, and our experts have distilled their experiences into valuable lessons for the future. From navigating new trends in game development to adapting to shifts in market dynamics and player engagement, our team shares their comprehensive knowledge and expertise. These insights are not just reflections of the past year but also guiding lights for the strategies and approaches we'll adopt in the coming year. 
By Dave Rohrl 08 Dec, 2023
As any developer worth their salt knows, figuring out how to monetize a game is one of the most important skills in free-to-play game development. At any conference (or in any blog or podcast series, etc. etc.), you will find a lot of discussion about the mechanics of how to best monetize players - what the mix of IAP vs. ads should be, whether battle passes are hot or not, how to optimize your ad network stack, and so on. These are all important and useful discussions, and it’s important for any successful team to think about them. But it’s also important to realize that the answers to these questions depend very heavily on what type of game you’re building and for what audience. Hypercasual games need to lean heavily into their ad monetization while midcore or casual games will tend to lean much more heavily into IAP. And this optimization can even change from day to day as CPMs go up and down across the ad market. From a design perspective, it’s much more valuable and interesting to understand why players monetize rather than how to monetize them. And as a bonus, these core underlying player monetization motivations remain steady over time and across game genres and categories. Building a strong understanding of these core motivations to pay will help you craft successful free-to-play games while honing your player empathy. It’s a classic win-win! I use a simple heuristic to think about the core types of player spending motivation: CAPS$ (pronounced “caps money”). This acronym lists players’ most important reasons for spending money in your game, and conveniently it does so in increasing order of spend per player (generally ARPDAU, but certainly ARPPU). Let’s take a look at each of these core motivations in succession. (No, not the TV show about the spoiled rich family…)
social systems in monopoly go by patricia pizer
By Patricia Pizer 26 Sep, 2023
Player retention and LTV are perennial drivers of game development. Even with sufficient initial splash to gain the critical mass of players, keeping those numbers requires planning and building your games accordingly. The ubiquitous, ever more powerful smart device, connecting players 24/7, brings mobile games into the spotlight in a way that requires examination of how to improve those key metrics. A host of methods for increasing retention and LTV exist. Daily activities or rewards help encourage players to login every day and create a habit. Rewards that build over time help cement that daily activity until it has become ritualistic. Infrequently discussed, however, are Social Systems which are among the most powerful methods. Simply put, a Social System is an underlying game system that supports or encourages social connections and interactions. Social Systems were suddenly critical as multiplayer spaces became omnipresent, creating opportunities and problems. Over time, social connections become many players’ primary reason to continue playing. To maximize the impact and utility of Social Systems, they need to be built into the underlying game, requiring investment in development up front, but the ROI can be enormous if it doubles or triples your LTV. Once those systems are in place, however, leveraging them is very low cost. If you are currently paying any attention to games, it’s nearly impossible to be unaware of Monopoly Go. So, let’s look at how Monopoly Goplayers are connecting. 1. Create a Daily Habit or Ritual Monopoly Go has heavily leveraged this particular method of player retention with numerous mechanics. The first one of these systems would be, as the name would indicate, “Daily Treats.”
By David Nixon 20 Sep, 2023
Mobile Game Doctor and AppMagic’s partnership goes back to 2016 and we have been extremely fortunate to have access to the depth of market intelligence data that AppMagic provides for the Mobile App/Game market during that time. So, I was certainly thrilled when they asked me to review their most recent Casual Mobile Gaming whitepaper for H1 2023 and provide additional insights. The first thing I would point out for context on Casual Mobile Gaming whitepaper for H1 2023 is that is a historical analysis. In 1948, Winston Churchill paraphrased philosopher George Santayana when he said - “Those that fail to learn from history are doomed to repeat it. –Winston Churchill” I point this out because while there are certainly lessons learned from history, past success or failure is not a particularly accurate predictor of future success or failure. This article needs to be consumed with the perspective of learning from the past to try to avoid mistakes that were made and not as a template for future market behavior. I have often given advice that if you want to break into the mobile game top-10 top-grossing, the first thing you need to do is deeply study everything that currently holds a place there, and then do something completely different than everything else. What follows below is a somewhat organized list of thoughts I have related to the information presented in AppMagic's Whitepaper informed by the practical context of consulting on 20 or 30 games that were struggling in one way or another during this timeframe and the 6 months leading up to it. Observation #1 - Casual Mobile Game Marketing/Distribution is Very Difficult Success in the recent/current mobile casual game market is primarily driven by well executed marketing strategy. A fun game that is accessible, compelling, well-designed, with a strong live operations execution is “table stakes” for entry into the market, but many, many games with all those attributes have failed predominantly because the publisher (the entity driving the game business, which might be a separate company or might be the actual game developer), failed to execute a viable marketing plan. Here is an example conversation I’ve had with a number of developer/publishers over the past few years: Developer - “Hi, Mobile Game Doctor? We need better monetization in our game.” Me - “Ok. How do you know?” Developer - “What do you mean? We don’t make enough money!” Me - “Enough money for what?” Developer - “Enough money to market the game profitably so we can grow!” Me - “Ok. What is your ARPDAU?” Developer - “$0.60” Me - “What is your D1, D7, and D30 retention? Developer - “52%, 21%, 9%” Me - “Those numbers are pretty good, actually…why do you think you have a monetization problem?” Developer - Because we can’t get a CPI under $25!” Me - “...sounds like a marketing problem.” The above is a bit of a dramatization, but the point is that I’ve worked on a number of projects where the developer is under the impression that they have a systemic product-design-related problem, when in reality the issue is that they have a flawed marketing/distribution plan. Today’s game performance marketing ecosystem is: Complex Crowded Competitive Overpriced Under-optimized (and difficult to optimize) Designed to maximize margin for the marketplaces Mostly misunderstood Couple these factors with a relatively low barrier to entry into the market, and you end up with a pretty voracious mix of factors that effectively make it IMPOSSIBLE for any new game to succeed in the market without some kind of true marketing competitive advantage. “The developers are still launching new projects at an insane rate: 258 in the past six months (compared to 80 in the Match-3 market). So, naturally, we see success rates plummeting from 3% in 2022 down to 0.8% in H1 2023; the market simply cannot cope with the sheer volume of games. Out of the 258 Survival titles launched, only the two above can be called successful. – Casual Mobile Gaming whitepaper for H1 2023 ” Many of the “failed” products in the market did not fail to build a compelling product - they failed to either: Foresee the amount of time/cost necessary to attain market-viable KPI Discover a genuinely market-viable USP Aside: In free-to-play games, product development and marketing aren’t really two different things. They are two sides of the same ecosystem. This is a very important point and the single most misunderstood factor of free-to-play games, especially by folks with a strong background in “premium” game development Observation #2 - The time/cost necessary to attain market-viable KPI There was a time, long ago, where you could tell, within a week or two of a “soft” launch if a mobile game MVP (“Minimum Viable Product”) had success potential or not. THOSE DAYS ARE OVER Here’s why: Most game genres and sub-genres are well explored. Many have at least one, if not more than one, established and entrenched competitors, as well as a number of less successful offerings. It is very, very, very unlikely that a brand-new entry into the genre is likely to perform at the level of entrenched competitors with mature and optimized game systems and marketing engines. The functional economics of performance advertising practically make this a law of the universe. Image source Casual Mobile Gaming H1 2023 , annotations added
By Paul Rybicki 07 Sep, 2023
Start with existing UX & UI models in XR For designers new to creating novel User Interfaces and novel User Interactions in a spatial environment like the current round of battery-powered VR headsets, you can leverage two areas of your prior experience as a conceptual bridge to this design space. We will get into those in detail in this article and you will leave with a conceptual foundation to operate from when designing interactions and interfaces in spatial environments. We need to start with two aspects of human centered design: Memory and Attention. Your attention please… Attention is an aspect of information processing that impacts how our users gather information from a sequence of screens over time (spatio-temporal environments) and process that information using Working Memory into larger constructs which they then attach to the goals they have formed while playing your game. This applies to a phone, a laptop or a headset. There are three forms which impact a user’s information processing and task completion. Selective attention, sustained attention and divided attention. Selective Attention is the act of choosing which stimuli to focus on and to process. Usually our emphasis is on visual stimuli and information input from a screen, but auditory stimuli (sound, especially spatial sound), tactile stimuli (vibration and haptic feedback) also use selective attention. These stimuli interact and can influence each other. Sustained or focused attention is the effort it takes to maintain processing of stimuli or information elements while ignoring other stimuli, or ignoring other information from the same stimuli. Divided attention is the ability to process or focus on more than one event or stimulus at the same time. When designing interfaces and interactions we often combine stimuli in a single experience, or substitute one for another, as in the case where a user taps on an on-screen button and the handset vibrates to indicate input received, but the button does not change color or size. Types of memory Recall dips into our Long-Term memory, our built-up knowledge over time. To recall a piece of information from long-term memory we need to see/hear/feel enough pattern fragments to connect those fragments with the memory and its associated information. Users should not need to recall information from screen to screen. When using an interface, Recognition is the primary system at play and uses Working, or Short-Term memory. Short-term memory is highly influenced by our attention resources, so we often use pictures or visual representations (icons) in our interfaces as a shorthand to aid in recognition and lower the burden on our attention. Knowing our place within the screens or the flow of a game we play regularly is very similar to our ability to recall the layout and arrangement of physical spaces, known as Spatial Memory and is drawn from our long-term memory. Combining our spatial memory and our present “location” within a flow or virtual space results in Situational Awareness and is how we know which way to go, or which UI element will take us to which feature or system within the game. Using an interface is a form of spatial navigation in short-term memory based on our processing of stimuli or an event intentionally constructed to carry information. How we use the space around us to work (the Promise) Recalling ideas or concepts stored in long-term memory requires enough memory traces to be reactivated through recognition (pattern completion) of any one of the stored patterns from a fragment (Smith & Kosslyn 2013). Therefore our ability to recall what functionality or feature sits behind the click of a button or which information is in what file relies on cues and on context of the user’s immediate spatio-temporal environment. It’s highly associative. Looking around your office, people are constantly using the walls for work spaces. Using the wall for a brainstorming session with Post-Its? Perhaps there’s a mood board of images on another wall of your workspace. That rolling dry-erase board makes a fantastic post-meeting reminder of discussions recently or even long past. Why is that? And how do we design digital interfaces and products now that have this recall capability and support multiple users just like people used to do in an office? An author once described a shared, networked visual workspace as “Unthinkable complexity … ranged in the nonspace of the mind, clusters and constellations of data.” The “shared” portion of that quote has been realized by wi-fi, fiber and cellular networks. The non-space of the mind can be thought of as our cognition, and our situational awareness.  Any interface is a reference point for the user to their long-term and short-term memory. A reminder of the cluster of data and the constellation (connection to other information or decisions) in which it resides. Existing 2D and 3D GUI’s use spatial layout and information hierarchy as well as temporal sequencing and progressive refinement to give users a way to segment and compartmentalize ideas, concepts & memories. For example: you need a flashlight. Where is your backpack? Tap an icon or reach with an arm & finger gesture towards your shoulder. A visual representation of the backpack opens and items are displayed, you open a side pocket to reveal more items and the view of the backpack visually shifts to focus on the pocket and it’s contents, following your attention and focus.This is the same thing as clicking your way through folders to get to a file. In terms of UI, the Treeview model can be employed in spatial and volume-centric environments as progressive refinement, but it’s still all progressive refinement & disclosure in the end. 2D UX to a 3D space Current head mounted display products (battery powered headsets using Quallcomm’s chips) already allow for gaze-based interactions as well as hand and finger gesture recognition. Think of a user's gaze and head movement like a mouse pointer. This gives you mouseOver / mouseOut events where the scene can react to your gaze. The mouse interaction model that items scale up or expand in size when you look at them, is incredibly powerful in VR, where it appears to move slightly towards you. When using pointing or arm gestures for interaction, thinking of your interfaces and the user’s hand as magnetic objects which can attract or repel each other will yield very natural feeling results as an interface moves slightly toward or away from a use’s hand. While this interaction is simply using the 2d interaction model of scaling up a UI element onMouseOver / onMouseOut it has a fundamentally different feeling when employed in 3d space. Finally, immersion. Polygon count and textures of the uncanny valley are not the only key to User immersion. They are one way to get there but there is a far better method which works well on both the current and previous generation of user-purchasable hardware: Interaction & reaction. Imagine you’re in a cartoony outdoor nature scene. Everywhere you look, birds flutter away when you move your arm suddenly or make a noise. A squirrel wanders towards you then runs away when you shift your weight form one leg to the other, moving your whole upper body. Environmental sounds come from all around you, and you hear a cartoon dog sniffing at your shoes from behind you. There are literally dozens of small, medium and even large ways you can interact with the scene around you. Spend 30 seconds in that environment? Focused attention. Situational awareness. Immersion due to the sheer amount of interactivity of the digital environment you are in. This approach to immersion is buildable now and when displays improve, adding more detailed textures or more 3d audio events to this immersive foundation will pay enormous dividends. Mobile-centric decision chunking FTW Despite the seemingly infinite canvas of an immersive headset experience, displaying limited information to your user in a sequence of chunks, as employed by mobile games, actually works much better.Keeping in mind that Attention is generally both selective and focused, let’s consider the following scenario. A player is refilling their backpack from a storage locker. The backpack stores items in a “tool roll” style interface allowing for 2d UI projection in world space. They run out of room in the pack - how do you allow them to sort or discard items? You can magically ‘unfold’ the tool roll to expand it larger and larger than is possible on a laptop screen, sure. But do you also allow them to see how many of each item they have in storage elsewhere? And how much they could get for selling each item later? Think: what would we do on Mobile? We would show the UI needed for the sorting task the user’s attention is focused on. We might give them a filter view icon to see the same items with different detail information. Mobile design keeps user-actions chucked by context and is far more effective at task competition. Countless design patterns exist for small-screen devices and work well when applied to sequenced tasks in immersive environments. Because letters. Text is still hard to read on the displays in current gen hardware so we use larger fonts. The next-gen hardware with their improved displays will change this however hardware limitations in reading text or UI up close to the user still exist. For the screen door effect to be resolved, a requirement for the long-use sessions of work-related VR, a combination of varifocal and high density displays will likely be needed. Our eyes naturally rotate to bring items in view into correct stereoscopic focus at different distances, but since headsets have a fixed view plane, our natural focusing muscles work against the visual stimuli when focusing ‘up close’ in the headset leading to eye strain and headaches. So for now, experiences designed using distant (2 meter plus) Interfaces with large fonts to make them readable at said distance are best. Foveated rendering is a rendering technique which concentrates resources in the area of the display where the user is looking. This allows for battery-friendly rendering with a noticeable visual improvement. Fixed versions of this concept have existed for some time and render the periphery of a user's vision at lower resolution. Designers can rely on auditory stimuli and use 3d sound to compensate for a lower fidelity visual system. And don’t forget the other senses! Designers can rely on auditory stimuli and use 3d sound to compensate for a lower fidelity visual system. This can be especially useful in providing immersion and directional cues outside the User’s FOV. How to plan for the future (the potential) Shared VR spaces have the potential to engage groups in a shared immersive experience. There can be a temptation to use the technique we know from console or desktop and show high poly and visual detail, and it’s totally beautiful! But visual detail, while it provides an initial reaction in our users, only gets us so far. Interaction is the true key to deeper immersion, where things react to your user’s focused attention or casual action. Building from a foundation of deep and subtle interaction means as the hardware generations get more powerful, you can add layers of additional visual detail to your game that will be both beautiful and performant. All the steps Mobile games use to get performance and battery life from a handset should be employed. The current IDE/engines will do many things for you out of the box: combined draw calls, backface & occlusion culling, dynamic level of detail (billboards) baked lighting, dynamic resizing of UI for consistent size at varying distances, and more! It’s up to us as designers to avoid surfaces with high color variances which create visual hierarchy problems, avoid translucency on current devices due to material or layer compositing overhead & subsequent hit to battery life, avoid exhausting our user’s attention with too much simultaneous information or choice; and to create spaces with low line of sight and multiple object occlusions for a perception of depth and distance for users new to headsets. In the past people have sought to use the promise of VR to provide virtual spaces in a virtual office that embrace real world workspaces and extend them. But in practice this likely results in information display that overwhelms the user’s attention & recognition. Use VR for what it’s good for right now: targeted immersive experiences of limited session time with minimal use of text that employs known interaction patterns and recognizable gestures for input. Don’t try to make it a general purpose laptop solution (yet).
More Posts
Share by: