Very often I hear people say "Perfect! We have a data scientist. Let's optimize everything!"

And to that I say: maybe. The truth is, the qualitative side of research is often overlooked, especially on teams that are powered by data science. This can really hurt an AB test or a product decision as sometimes, the problem isn't thoroughly understood through the user's perspective.

Here's my traditional flow when it comes to an AB Test:

• 1. Fall in love with the problem.

• 2. Synthesize qualitative research.

• 3. Mind-map creative ways forward.

• 4. Test your idea quantitatively.

• 5. Iterate, Iterate, Iterate

1. Fall in love with the problem

"The best way to get to know a problem is by observation."

Schedule some interviews; watch some user behavior; really get to know what the root of the problem is. Be conscious to listen for the anxieties that people have and the reasons they have those anxieties. Do they not want to share content because they have never shared something on social media before? Do they drop off on the signup process because they feel it is not trustworthy?

Although this is just a single-user's perspective and you cannot generalize this user's experience with everyone, the depth that comes from these one-on-one observations fuels the possibility of creative solutions. Maybe the problem isn't that the button doesn't look clickable but rather that the button shouldn't even be on the page. By really empathizing and seeing the user's experience firsthand, you are able to gather ideas about what the true problem is.

2. Synthesize qualitative research

This part is my favorite. Time to take all that messy chicken scratch (that probably resembles the scene out a detective movie where they are assembling a murder board) and get to work!

At this point, I like to distill all my notes down to singular post-it-notes encapsulating one idea. Taking all your notes and putting them to post-it-notes is one form of synthesis and the added benefit is that the post-it-notes are now mobile. You can start to group like ideas together and voila, themes start to appear. You get to dig deeper into the actual problem at hand rather than the surface level problem that you started out with.

3. Mind-map creative ways forward

I lied, this part is actually my favorite. I really learned everything I know about mind-mapping from one of my all-time favorite books, Creative Confidence. I highly recommend you check it out!

The main idea here is you start with one of the problems you found above. You don't want this problem to be too broad or too narrow (it takes practice to get it right so just take a stab at it and you will get there). The goal of mind-mapping is to get as many ideas out there in a way that is fast-paced and divergent.

At the center of the page I recommend framing the question as:

"How might we solve for (insert problem here)?"

Then, set a timer and make a goal to come up with 50 different solutions over the next 3 minutes. I know it may sound daunting, but the timer will help give you the pressure you need and the freedom you need to put down what you might consider to be the world's stupidest solutions. But you never know, those solutions could turn into something really magical.

Do a quick check about what data you have to support some of your favorite solutions; see if it aligns.

4. Test your idea quantitatively

Once you have a solution that you actually like, start to think about what success of this idea being implemented looks like. Does success look like:

• Spending longer on the page?

• Engaging more with the widget?

• Clicking the call to action?

Quantitatively testing your idea allows you to see if your creative solution worked. If you have never run an AB test before, I recommend getting started with a free solution like Google Optimize. Reach out to me if you have any questions here.

5. Iterate, Iterate, Iterate

The biggest mistake is stopping when you get to a positive quantitative result in Step 4. This is how companies and projects get stuck in local optima instead of global optima. Use what you learned to fuel new questions to research. It will only strengthen your understanding of the problem.

This is a never-ending cycle. Some days you get caught in being discouraged by the problems you see users facing with a product you love. Other days, you ride the highs of success for an idea you implemented. No matter what though, you are making a positive impact. Here's to making the world around you better designed and more creative.