04 Feb 2014

Behind Conversions Testing is a Real Person (Do You Know What They Want?)

Putting two images of page layouts in front of an audience to ask them to choose which one converted best is both entertaining and frustrating.

I was reminded of this again at a recent PubCon , where images for testing were put up and the audience was asked to guess which one converted better. I loathe these moments because as a usability analyst, I know that what I think might convert best is not the point. In addition, my magical powers don’t make me miraculously know what targeted users might choose.

You may think it’s easy to guess for pages that compare one with a big call to action button and one with a big text link call to action. No-brainer right? A big “Buy Now” or “Sign Up Today” button begs to be used and is easy to find, but if you are a special needs individual using assistive technology to interact with web sites, the text links are just fine. If the big juicy button shares space with other strong calls to action, all bets are off for the simple reason that visual distractions are an issue for many people.

At the conference I guessed correctly on the first one, wrong on the second one and abstained from the third one because I was done playing the game. The one I got “wrong” was two images of a shopping cart process page. One page layout provided additional information for confidence building in a box on the right side of the items placed into the cart, such as shipping information, warranties, accepted payment methods. The other page for comparison had just the items and no additional information before leading to the next step in the cart, which I assumed was to start the shipping address and payment sections.

The page that provided no additional information got more conversions. I chose the page that offered additional information because I know helping customers feel confident during the purchase path helps to keep them on track.

Older couple with laptopWhat We Don’t Know

The information we were not provided before being asked to choose which page converted better includes:

  • Who is the target market?
  • Are they younger, older, male or female, professional or consumer, regular or new customers?
  • Why are they here?
  • Did they arrive after receiving a coupon code in an email? Are they return customers who already know about the shipping deals and what payment methods are accepted? Was the process we were shown from a logged in user or non-logged in?
  • What information was provided earlier in the sales funnel?

Since the page that did not include any user instructions or additional supportive content was the “winner” in their conversions test, I wanted to know if the information they removed was offered earlier. For all we knew, the product pages and homepage, as well as the footer, had the information needed to feel confident about making purchases so that in the step we were shown, the simple “let’s do this now” layout is perfect.

Generic User Testing

As a testing tool, many companies pay for user testing that gives them 5 random people who are asked to review pages. For free, the data collected gives a basic pass or fail overview. The pages either sucked or not.

For free and some imagination, you can also go out in search of your existing customers, online or in person, and get feedback on a redesign. For free, using imagination, and seeing an opportunity to illustrate your exceptional customer service and to increase brand reputation, you could set up a kiosk in a mall and get feedback from new prospects, existing customers, network with curious people and call in a local TV station to showcase how you care so much about your customers’ online experience, you literally went out and looked for them.

Okay. That idea might not be free but what’s a kiosk rental worth to you?

User testing should be performed by the people you are designing your website for. Your grandparents may get lost using it, or your cousin Sam may not know what a “cloud” is. You need as much information on your target site visitors as possible to perform valid user testing so that you can choose tests that fit your site’s specific requirements.

For example, if you know your web site provides information to a group of people who have very little time for research, are easily distracted and wear reading or prescription glasses, you can be prepared with valid testing plans that meet their specific needs.

Tasks Testing

The shopping cart page choices we were shown at the conference did not include where they appeared during the task. To me, this is the most critical part of any conversions testing.

Had we been asked to truly get involved with split testing the two shopping cart pages we were shown, I would have looked at how users arrive to this point and inspect what they are shown next. I had so many questions about the example we were shown that I was angry and had trouble focusing on the rest of the talk. I was likely the only usability analyst in the audience however.

Device Testing

One of the questions I had was how the two pages rendered in mobile devices. Good mobile design removes distractions. Tasks are simplified. The reasons why someone would place an order from their mobile device may be different than those sitting at their desktop computer. Many people do just about everything with their cell phones and tablets. The environments are different. A small mobile device used on a fast moving train packed with noisy people and the passenger in the middle bumping into you is far different than casually browsing for products from a quiet office.

Product pageThe Whole Picture

I have the same qualms about Which Test Won. The examples are great for providing new ideas but the mistake is believing that what worked for one company’s conversions may not provide the same lift for your web site. In fact, the same shopping cart page test shown to us at the conference was performed on another website and the results were the total opposite. In that case, the page with extra support information increased conversions.

Web site reviews are a much needed and sadly overlooked part of site ownership. At that same conference an audience member during my talk argued that underlined links are old school and ugly. The page I was using to illustrate a point about calls to action showed products listed on a products page that showed no way of knowing what to click on to get to the actual product item page.

Later, the conference attendee and I continued the conversation and at best I think I got him to see that there are so many types of people using websites that we must pay attention to who we are targeting. Everyone recognizes underlined text as a link. Everyone recognizes beveled edges to be something they can push, like a button. Flat design removed the buttons and creative folks removed the underlines. So now all web users must learn where designers are hiding click paths and sales funnels are growing into a true mystery.

Conversions for Who

The underlying theme of all the talks at Pubcon Austin was conversions. From the marketing perspective, the uber-critical component to making nice with search engines is by providing quality content to your site visitors. To do this, in a “keywords not provided” [Google] world means looking for new ways of understanding what your site visitors want, and how they found your site.

Yet, even if you select the very best keywords, make landing pages for them and choose navigation link labels that incorporate action words with keywords, you still face the usability factor. Until you know more about who uses your web site, who they refer to it, who is not using your site, and how to increase traffic by making it accessible to more people, your conversions will just be lazily floating on a raft in a cool, blue swimming pool with a drink in one hand and reading a good book in the other.


  1. Stephen Anderson February 5, 2014 at 2:04 AM

    It’s funny, you just never know which is going to convert better. The only thing you really can do is try something and see how it works, measure and try another version.

    It’s not too hard to do, but takes some time to come up with alternate versions, but its well worth the time spent doing it.

Leave a Reply