Surveys

Description:

Surveys are used to obtain the Voice of the Customer.

Objective: 

Create a clear, unbiased, and complete survey to get relevant feedback without being difficult for the respondent to complete and return. The proper choice of words and linear scale settings are important characteristics. 

---

Qualitative Survey

The key to understanding qualitative analysis is that the survey question responses will be in the form of words instead of numbers.

These types of surveys may contain questions that are:

Open-Ended Questions

Binary Questions

Categorical Preference Questions

OPEN-ENDED

Questions that offer the most freedom to respond have the benefit of obtaining a higher level of detail but also take the most time to decipher and analyze. These also run the risk of needing to follow-up (if possible) or possible misinterpretation. 

Sample questions are:

Why did you choose this product/service?

What improvements would you like to see in this product?

What do you like about the product?

What do you dislike about the product? 

Review the first question. There could be the following responses:

1) Great price
2) Referral from a friend/family
3) Already own one, good reputation and trust
4) Good warranty
5) Higher quality and fair price compared to competition
6) Nothing else on the market

These responses would be reviewed by the team and broken down into categories such as Quality, Price, Brand, Delivery, and Customer Service. 

Then organize the responses into the categories and it is possible that one response may get multiple categories (such as question #5) which is the benefit of open-ended questions.

The following is a methodical way to break down the responses to open-ended questions. 

1) Have the team review the responses and try group consensus to categorize the response. 

2) Create possible categories for answers and there will likely be some responses that don't seem to fit, or can be labeled as miscellaneous. If there are too many of these, it is possible the question needed to be more specific.

3) Determine the number (count) of responses in each category.

4) Display counts in each category and now the BB/GB can provide the team results in terms of percentages, mode, rankings, or just the count of each category. 

BINARY QUESTIONS

Questions that offer two choices to respond are binary questions. You may offer the responses and have the respondent select one, or it may be obvious to the two choices and they can write in their response.

Sample questions are:

Do you prefer this product HOT or COLD?

What is your gender?

Do you prefer the glass half-full or half-empty?

Would you refer this product to a friend/family? YES or NO

These are simply categorized and then analyzed. 

CATEGORICAL PREFERENCE "WORDED" QUESTIONS

Many surveys use numbers to represent categories or classifications and the difference between each number has meaning.

This is ORDINAL data and is one level higher than the lowest level of data, known as NOMINAL data.

1 = POOR
2 = FAIR
3 = AVERAGE
4 = GOOD
5 = EXCELLENT

Other questions are:

What is most important to you? Price, Quality, Warranty, or Service (Nominal Data)

What color do you most prefer in the product? Blue, Red, or Green (Nominal Data) 

When you provide the possible answers it is easier to categorize and count but sometimes the responses may not be exactly the respondents best choice or ideal answer.

They may feel obligated to pick one and this can lead to wrong decisions from the statistical analysis. This is non-metric or qualitative data and requires the use of non-parametric tests to evaluate statistically. 

Mann Whitney: Test the differences between categories

Chi-Square: Test the independence between categories

Spearmans Rho: Test the relationship between categories

---

Quantitative Surveys

Summary

Qualitative feedback is more challenging to break down into similar categories. Tools such as the Affinity Diagram can help the team break down questions and responses into preference categories for analysis.

This often is more time consuming and difficult to perform than quantitative analysis but usually provides more information and detailed responses than just a number.

Quantitative analysis is usually less expensive to conduct and are easier to conduct with automation and electronic mediums.

It is also easier to mine the data to investigate trends and statistics but lacks some of the details a worded response or open-ended response will provide.

---

Survey Sample Size

Determing the Power and Sample Size for a survey is required to properly communicate the results. Futhermore, you can run "what-if" scenarios to understand the fine balance of the costs and practical ability of acheiving a certain level of Power through increasing or decreasing your sample size within a survey.

Obtaining samples cost time and money. Of course, it's ideal to get 100% of a population but rarely is that possibe in reality and therefore you need to decide what amount of sample can practically be obtained and what risks that leaves within your data.

The closer you get to obtaining 100% of the population, the stronger your statistical results will be and less error is likely.....but that comes at a cost which varies by survey and project. And how you collect the samples is critical too, click here to learn about various sampling strategies.

Key Factors

The following are four key factors in determing the sample size for your survey if using continuous data. 

1) What size of detectable difference are you looking for?

2) What magnitude of alpha-risk are you willing to accept (or level of significance)? Typically this is set to a Level of Significance of 95% which is an alpha-risk of 5% (0.05).

3) Power - which is the chance of detecting the true difference in 1). Typically the Power is selected to be 80% or 90%.

4) The standard deviation (SD) of your data (data can be normal or non-normal as SD tends to be larger for non-normal data)

The video below helps explain these terms and their relevance to properly understand the risks in your final statistical analysis of the survey results. 

---

Return to the DEFINE phase

Return to the MEASURE phase

Templates and Calculators

Return to the Six-Sigma-Material Home Page