Debunking 3 Myths About A/B Testing for Mobile Apps

We do it for websites. We do it for mobile advertising. It’s time to start A/B testing mobile apps as well.

However, even as the practice becomes ever more common, several myths persist about A/B testing mobile apps in the publishing world.

1. Myth #1: A/B testing mobile apps is difficult

The truth is that mobile app testing used to be difficult. Waiting to see if your new layout performed better than your old layout based on app store reviews was not only time-consuming, but inefficient and often ineffective. Reviews don’t tell the whole story. Users will let you know if there’s a feature they don’t like, but they can’t tell you whether a “buy” button in red or yellow makes them more likely to complete a purchase.

The good news is that optimization and A/B testing mobile apps isn’t hard anymore. You can drag and drop screen elements, push out A/B tests instantly to different audience segments, and revise your app layout based on user response.

Leave the more difficult app functionality changes to your developers. Testing and optimizing your user interface? That’s easy.

Myth #2: A/B testing mobile apps only applies if you didn’t do it right the first time

You have no way to know how different audiences will react the first time you release an app into the wild. For the most part, there’s no inherent right and wrong when it comes to layout design, and even though a design might “look right” aesthetically, it doesn’t guarantee business results. The only real measurement of value is how users interact with your app, and whether the user interface has a positive impact on conversion rates, or a negative one.

Testing an app doesn’t mean it’s been badly designed. It means you’re committed to finding what resonates best with your audience.

One of Artisan’s customers found that simply by moving a button to the right on one screen, it was able to increase conversion rates by forty percent. There is no way the company could have uncovered that information without A/B testing.

Myth #3: A/B testing mobile apps only applies to your mobile strategy

Data on user behavior is valuable no matter where it comes from. While information from A/B testing mobile apps is particularly useful in the app environment, it can also be applied to other channels.

What products pair well together? How do different user segments respond differently to different promotions? Are there particular content elements that make users more likely to respond favorably to your brand?

Consumers are spending more and more time on their mobile devices, and that means the mobile channel is a critical window into likes and dislikes, purchasing triggers, and behavioral trends. More than ever, app testing is an important source of data on the audiences you want to reach… on mobile devices, and everywhere else.

Sign up for our blog digest emails.