7 tips (and a bonus one) for making a great online survey

So you want (or need) to create an online survey for a project? You have an idea -a hypothesis – and want to test it by letting a bunch of people answers a couple of questions. Most of us would start straight away by selecting an online tool and start to enter questions. Please don’t. First do some research. Submitting the questions is just the last step. Or as Abraham Lincoln put it:

“If I had eight hours to chop down a tree, I’d spend six sharpening my ax. – Abraham Lincoln

Okay, it’s is not really a Lincoln quote, it’s fake. But the message is clear. Preparation is everything. In this post you will read 7 simple tips for creating a successful online survey.

Survey tip 1 – Write down your project goals

What do you wish to learn from your respondents? And just as important, what don’t you need to learn. This is the most important part of the project. Fail to have the correct goals and you’ll end up with results that won’t be useful.

If you have any doubts, check your goals with your supervisor, professor, manager of anybody who you know is not afraid to criticize you. Try to avoid friends and family that don’t like to give you honest feedback.

Survey tip 2 – Determine your target audience

Who is your target audience? And just as important; how do you find them online? If you want to check your business idea for a new product for say female gaming enthusiasts, be sure you can reach sufficient of them online. Do the research and check for the following facts:

• Total number of target audience (in your city, country, worldwide)
• How can you reach them online?

Nothing can be more frustrating then creating the best survey ever and realizing you have no way to reach your target audience. Use social media, respondent databases and specific targeted websites such as online fora as potentional sources.

Survey tip 3 – Determine the minimal number of required respondents

When do you have enough respondents? There are a lot of factors hanging on this one. If you take and scientifically approach you can use the 95% standard deviation. To put it very simplistic it states that if you have an unknown population (size) the sample size should be 384. 

There is also a practical approach which is based on things as available budget or network size. If your project is not for a PHD keep in mind that there is a difference between respondents answering a questions and those same respondents acting in real life. If you asked them if they would buy project A based on given information such a price and they state they would, you still don’t have any guarantee they would in real life really do so.

There are many debates about this and for sure many will disagree with this statement. Online respondents should be used to get a rough prediction on a outcome of a hypothesis. To test the hypothesis measure results in real life. For instance with sales results.

For most projects 200 respondents are a minimum. It will give you a good -nonscientific- idea of the results of your hypothesis. Based on these results you can always do more research.

The minimal required number of respondents can be part of your project goals. Ask your professor, supervisor or a field expert if possible.

Survey tip 4 – Ask the proper questions

Dived the questions in two parts. Part 1 contains the questions for the hypothesis. Start with the basics and go ask to more details questions in the end. Use the invitation e-mail and the introduction text of the survey to triggers the curiosity of the respondents. This will make them complete the survey.

Part 2 is to get the required demographic data. These consist of age, sex, educational level, location, etc. Prevent asking demographic date that are difficult to analyses. For instance, ask for year of birth and not for age. Peoples age chances every year, perhaps next week already. Their year of birth always stays the same.

Think about which question type you use per question. The most common is the multiple choice question with one answer to select. It is answered easy by respondents and the data is analyzed efficiently.

Open questions are far more difficult to analyze for instance. All results have to be manually read and analyzed, which is a lot of works with 200 respondents or more. Try to prevent this when possible.

Survey tip 5 – Keep it relevant (routing / skipping feature)

Routing of skipping is the feature to lead respondents to curtain questions based on previously given answers. For instance:

Question 1 – Have you even heard of Product B?

1. Yes
2. No

Question 2 – Where have first you heard about Product B for the first time?

1. Through friends
2. Through family
3. Through online advertisement (websites)
4. Through offline advertisement (tv, radio, newspaper)
5. Through outdoor advertisement
6. I don’t remember

If the respondents answered “B – no” at question 1, don’t let them answer question 2 and lead them to the first next question which is relevant. It also prevents respondents to answer more questions than necessary.

Survey tip 6 – get the correct answers

Just as important to ask the proper questions, the respondents need to be able to select the correct answers. If a respondent can’t select the correct answer, the results will be ruined. An simple example:

Question: please select your job situation

• Student / still in school
• Entrepreneur / self employed
• Small company (less than 100 employees)
• Big company (more than 100 employees)

If the respondent doesn’t have a job? In this case either an “Other, please fill in” option can be added of a 5^th answer option can be added. The “Other, please fill in” option is hard to analyze though, so if possible prevent using it.

It is important to make sure the respondent can select a correct answer in the answer options. Test your survey questions before sending them to your target audience.

In a separate blog post badly formulated questions such as leading questions, double barreled  questions and loading / steering questions will be handled.

Survey tip 7 – Keep the completing time below 5 minutes

Most respondents answers about 5 simple multiple choice questions per minute. More complicate question types such as thesis questions take longer. Try to keep the total time needed to complete your survey to 5 minutes. Test it on a small (about 20 respondents) group if possible. The more time it takes the complete the survey, the more respondents will back out during and the longer it takes to get the required number of respondents. The quality of the given answers can also recline during long survey.

To simply put it: try to prevent respondents getting bored.

Survey bonus tip  – Explain why you invite somebody

So your survey is ready for respondents. Keep in mind that respondent have 168 hours in a week, most of it is spend on sleeping, eating, working / studying, social activities and hobbies. They have to exchange a couple of minutes of these activities to complete your survey. Make sure respondent have the feeling they aren’t wasting their precious time. When creating the invitation and/or survey introduction keep the following in mind.

1. Explain why you invite the respondent
2. In necessary explain you got their contact details
3. Introduce yourself a few words
4. State the project goal
5. Give a indication of the completion time of the survey
6. After completing the survey thank the respondent for their time

Most people are willing to fill your survey if they sympathize with either you or your project. Keep survey tip 7 about completing time in mind though.

Now start building your project.