Blogs
Myth: Father Knows Best. It’s Time to A/B Test.
Front-End Web Developer
Aug 7, 2014
Father Knows Best was an American comedy that ran on radio and then television from 1949 to 1960. The character of Jim (Father) has strong influence over his family with the best of intentions. I mean, just look at this guy. He radiates confidence and authority. With a character like Jim, it’s easy to forget the most important, invisible star that made this show a long running smash hit: The Audience.
Likewise, businesses with talented, well-meaning employees may also forget the unseen star of the company’s online success: The User.
Quite unintentionally, our managers, peers and expertise can become the stars of the website. We may even forget to question how the changes we make can impact our audience. Let’s face it. We can all be like Jim, when we think we know what’s best for the company website without really knowing. For example: A company might think adding a section of their employees may make the business appear too small for people to be likely to engage. On the other hand, a user may appreciate this transparency, feel more connected, trusting and be more willing to engage. There is a way to uncover which scenario works better with quantitative data to back it up. It’s called A/B testing.
So, what is A/B testing?
An A/B test simply measures the impact of a change (big or small) to your current website.
To A/B test a web page, create a copy of the original page (control) and add the change you want to test to the new page (variation). The change may be a different content heading, a layout adjustment, an alternate image, CTA button, font, color, etc.
A specified portion of users will see the control and the rest will see the variation without being aware that there are two versions. One version may influence a user’s behavior resulting in a measurable outcome such as a significant revenue gain, increased form submissions, etc.
Google Experiments or other A/B testing software can be used to set up the experiment goals and collect data on both versions for a period of time long enough to determine a winner or learn that there is none. And most importantly, it brings us valuable insight into what our users prefer. Over time and testing, we can get to know our users better and make informed decisions when it comes to change.
It’s good practice to limit the number of changes to one per test. That way, you can be certain about what made the impact.
Test Before Change
The larger your company, the more departments it has. People in different departments have different perspectives and priorities. And nearly everyone has opinions about the company website. Just ask. Input and suggestions for site changes can vary greatly or even conflict with other requests and points of view. The more requests for change, the more challenging it can get to know what to prioritize, implement or say no to. Your site may become muddled over time if you say yes to all requests without vetting them, especially if they’re from vastly different sources. Each person has unique value and expertise. However, with A/B testing, we’re able to find out how these requests resonate with users before implementing them across your site.
When you hear ideas about changing the site, it’s the perfect opportunity to suggest A/B testing and have the requester fill out an A/B test request form. All ideas to be potentially tested should be well thought out with a good hypothesis. Otherwise it’s just throwing spaghetti against the wall to see what sticks.
Sample A/B Test Request Form
Who is the test targeted towards?
Segment examples: All users, mobile users, new users, returning users, German users, etc.What specifically do you want to test?
Experiment examples: an image, a CTA, content, a color, page layout, etc. – one change per test please!What do you hope your test achieves using a specific test metric to measure the results?
Goal examples: Increase revenue by 5%, decrease bounce rate by 10%, increase form submissions by 15%, etc.What is the page you want to test?
The specific URL goes here. Testers, make sure this test doesn’t interfere with any other A/B test, promotion, holiday, campaign, etc. on your site.Why do you want to test this?
Is your test idea based on a successful case study? Does it come from a knowledgeable resource? Or does it arise from a known issue or is it a gut feeling? Please be specific.Start an A/B Testing Culture
So how do you begin to spread awareness within your company about the importance of the user’s viewpoint and A/B testing for your site?
- Bring up A/B testing in meetings as a means to test new ideas or resolve internal conflicts.
- Create an outlet for employees to share suggestions with you for future A/B tests, using tools such a the “A/B Test Request Form” (detailed above).
- Share any interesting A/B test results (both wins and losses) with your company.
- Share A/B testing case studies and resources with your colleagues like this:whichtestwon.com
The folks at WhichTestWon post weekly, A/B tests results from companies, right after you see both versions and guess yourself. I guessed wrong so many times, it dispelled some false beliefs I had about what works on a website. I knew best only about 50% of the time according to the results. Lesson learned: If you’ve never tested it before, don’t assume how your users will respond.
Challenge
Take your best guess at Which Test Won for 10 weeks. WhichTestWon presents a different A/B test every week. Check out both versions and make your best guess before looking at the results. Keep track of how many times you guessed right and wrong. It may surprise you, annoy you, or delight you. But best of all, it will make you aware that sometimes your instincts are wrong and you may not know what’s best for the user despite your opinion or experience!
A/B See
I recently attended and highly recommend this conference for anyone who wants to begin or refine A/B testing: WhichTestWon: The Live Event – the most inspirational and useful conference on A/B and multivariate testing.
When it comes to change on a website, it’s really all about the users. Popular opinions, good intentions, years of experience, raw talent, position on the food chain and great ideas don’t trump data. A/B testing provides solid proof of what does & doesn’t work on your website. When you A/B test you move beyond hoping and guessing, into data-driven, informed design. Let the data dictate change. Remember, Dad makes mistakes just like everyone else!
-
Examining Dyn’s Culture And How We’re Different
Front-End Web Developer
Jan 30, 2012
In the business world, culture is defined as “a shared set of values, attitudes, practices and goals that make a company unique”. In essence, it is the character of the organization. Every company creates its own culture whether consciously aware of it or not.
Whether you are an employer or an employee, culture is very important and it impacts individuals as well as the company as a whole. For employers, culture affects morale, communications, productivity, growth, the bottom line and can ultimately make or break the company. For employees, a positive culture can motivate, inspire creativity, loyalty, cooperation and a desire to go above and beyond. On the other hand, it can erode trust, security and connectedness while increasing stress, indifference and job searches.
Dyn is definitely in the minority of companies ahead of the culture curve. Receiving regional recognition from Inc. Magazine as a winner in the 2011 Top Small Company Workplaces competition and being named one of the Top Small Businesses To Work For by Business NH Magazine for four years isn’t by happy accident. Dyn’s overall success and rapid growth is greatly attributed to the following three major culture factors:
- Dyn’s People: Priority One
- Dyn’s Practices: Transparency, Trust, Team Building, Timely Reviews
- Dyn’s Perks: Traditional and Fringe Benefits
Our culture is as truly unique as it is amazing. It’s refreshingly anti-corporate and better experienced than talked about. You can see its first impression reflected on the faces of visitors taking a tour of our office or new employees coming aboard for their first day. You can sense its influence in the positive attitudes, passed down from founders to managers to individual contributors. You can feel it in the air: a laid back but highly energized atmosphere. Sound dreamy and intangible? It is and it isn’t. Here’s a few cultural examples on how we got there.
People Are Priority One
People are Dyn’s most valuable asset and they are treated accordingly.
To quote our CEO Jeremy: “People are a company’s best asset. A company might have a great idea, customers or product, but it’s still the team in place that’s responsible for executing. Without them, you have nothing.”
Dyn focuses on employee strengths — not weaknesses.
Illustration Credit: Matt Groening, Courtesy of Random House, Inc.
New employees take the StrengthsFinder test and are encouraged to specialize in what they’re good at and what they enjoy doing. For our engineering team, “Offroading Days” take place twice a month, where employees come up with their own ideas for projects and bring them to life.
These opportunities inspire creativity and exploration which can result in new company technologies or the discovery of new strengths and interests for the individual. It’s not uncommon for a Dyner to wind up in a completely different department than where they started.
Caring about the safety and well being of employees is the Dyn way.
This translates to stay home sick when you’re sick, work from home when it’s unsafe to travel in bad weather, get a lesson before jumping on a Segway, keep fit with a free membership to Gold’s Gym and de-stress with a nice game of foosball or drag racing.
Being valued as a human being is a breath of fresh air if you’ve ever experienced the “macho management mentality” where employers expect employees to come to work ill, injured or during a state of emergency. Sadly, this attitude still exists in some of today’s work places.
Practices: Transparency, Trust, Team Building, Timely Reviews
There are a lot more practices that make Dyn culture great, but I wanted to focus on a select few:
Transparency
Dyn practices honest transparency with its employees, competitors and customers.
Kept in The Loop
There are weekly one-on-one meetings between managers and team members to discuss projects, ask questions, receive feedback or express concerns. Monthly company meetings bring employees up to date with the latest company news and future plans. Quarterly open book financial meetings are held to share detailed financial reports and projections.Communication Between Competitors
Dyn was proud to host and facilitate our first Email Deliverability Summit last December, bringing together some of the biggest names in email delivery and the deliverability experts that fuel their success.The Good, The Bad and The Ugly
Dyn believes that transparency is critical in keeping customers informed, even when it’s not popular or pretty.Trust
Transparency leads to trust and trust is a must for good culture. It’s an absolute pleasure to work in an environment where employees are trusted to do what they specialize in instead of being micromanaged. Dyn has pride in its people and isn’t afraid to relinquish control to let them get the job done. Another example of trust is allowing employees to work from home on occasion and trusting they will put in an honest day’s work. Imagine that!
Team Building
Photo Credit: Lynda Straffin
Team building activities at Dyn happen often, from Geek Week to company outings to company events with live performers and bands.
Dyn fosters a team environment where people become so connected at work they often choose to hang out or do activities together after work. Different departments work closely together, maintaining mutual respect and communication because one department can greatly impact another despite having different perspectives and goals.
There’s no room for an “Us vs. Them” attitude. Sales and techies living in harmony? Yep, it happens here!
Timely Reviews
Dyn makes the time for regularly scheduled employee reviews. There’s nothing worse than working hard for a company and going years without a review. It can leave a person feeling unappreciated, unnoticed, unimportant and maybe asking the question: “Why do I work here?”
Not to mention these reviews eliminate missed opportunities for employers to provide valuable feedback that can help the individual and the company on many different levels. Dyn also offers employees the rare opportunity to review their managers anonymously. It’s a valuable tool that provides a more accurate inventory, builds trust, earns respect and puts everyone on a more even and fair playing field.
Perkier Perks
Why do people want to work, play and stay at Dyn? It’s a combination of the nontraditional management style, the relaxed, upbeat atmosphere, the unparalled traditional and fringe benefits and the awe-inspiring physical surroundings. Dyn does not skimp on culture or make it a vague, last place priority. It puts culture at the forefront, spares no expense and reaps the rewards of being defined best of the best and a power of example to the rest.
Just a few of Dyn’s perkier perks:
- Have you ever stood in line for breakfast at work while the CEO cooks your omelette made to order?
- Can you imagine working from a comfy couch, beanbag or sunny deck?
- Are you able to step away from your desk and play ping pong, skee ball or video games? (By the way, this relieves stress, enhances problem solving skills and increases productivity!)
Want to know more about why culture matters to your business and learn strategies on how to improve it? Dyn is hosting our first-ever Culture-Con on January 31st at our Manchester, NH headquarters. Find out about signing up your company culture representative today!
