Creating an effective online survey
Creating an effective online survey can seem like a daunting task at first, but we’re here to help. This guide will take you through the whole process of running a survey project so that the end result will be the best that it can be. From start to finish, you’ll find tips and advice to help you throughout your project. Whether you’re an absolute beginner or a seasoned professional, there’s something for everyone here.
Let’s get started and look at how to make our online surveys effective, engaging, and useful.
To do this, we need to think of the survey not just as questions in a form on a website, but as a project with a beginning, middle, and end, broken into key steps for each of these phases. We’ll walk you through these phases step-by-step. For some surveys, some steps may seem trivial, but it’s important to run through them all, just as a mental checklist, to
make sure that your survey is of the highest quality
With that in mind, let’s look at these steps. They can be thought of as follows:
Define your goal – great online surveys start with clear goals
Having a clearly defined goal is important for any project, and online surveys are no different. Even so, it’s common to find that an organization will start a survey project just because they feel they should, especially around the customer satisfaction space.
“We don’t really know what we need to find out”, is something we sometimes hear and this can lead to organisations creating bloated, meandering surveys that have poor engagement as the organization tries to ask every question it can think of as part of a single survey.
So, before you get stuck in, it’s important to define this larger goal. What’s the “big question” that you’re trying to get an answer for? It could be “Are my customers happy?”, “Should I bring this product to market?”.
Defining this early on will make the following steps easier. This goal should be kept in mind throughout the project and should inform every decision that you make. It might not be easy to do this, as the temptation to add extra goals can be strong, both from yourself and sometimes from other parts of your organization. But your survey will be better, and the results more useful, if you keep true to your goal.
TOP TIP: It’s always important to remember that online surveys should be approached as an ongoing process. Thinking in terms of a survey program, rather than a single survey, will help get this across. If you’ve identified several of these big questions, then start thinking of them as separate surveys, and plan them out accordingly.
Define your audience – targetting the right people is crucial for good engagement
This usually goes hand in hand with the first step, as in a lot of cases, the goal defines the audience in question – your customers (or a subsection of them), maybe people who live in a given area, or of a certain demographic profile.
In general, you’ll see better response rates when you target your survey towards people who are the most relevant respondents, rather than going for as wide a distribution as possible.
A lot of this, of course, relates to the fact that outside specific workplace surveys, respondents are spending their own time to complete the survey, so unless they are interested in the subject, or see some clear benefit to taking part then there’s a high risk of non-participation.
That “clear benefit to participation” is often thought to mean a specific incentive, such as the chance to win vouchers or some other reward, but it doesn’t have to be this.
Offering cash (or equivalent) incentives can raise the risk of collecting bad data is something most people can understand, but that’s not to say that it can’t be a valid strategy. There are measures you can take within the survey to “weed out” the respondents who are “straight-lining” or “speedrunning” – simply clicking through the survey quickly to get the reward. We’ll cover those when we talk about creating the survey itself. Outside of monetary incentives, a “clear benefit to participation” can be simply that the respondent can have confidence that their feedback will be taken seriously and used to inform future decision-making. Being able to refer back to previous surveys and identify points of feedback and associated courses of action taken can be highly encouraging to potential respondents.
You said, we did
Often people can feel that many of the companies and other organizations they deal with are faceless and opaque, and so surveys (and consultations) become one of the few avenues where they feel they might have their voices heard. This can sometimes manifest as a fairly cynical attitude that “it doesn’t matter, nothing will happen” so being able to illustrate a responsive attitude for the organization, by means of a “you said, we did” feature or other publicity can help with this.
Once you’ve defined your target audience then you’re ready to move on. Except — what do you do when you’ve defined an audience, but have no idea how to reach them?
In these circumstances, there are companies who can sell you targeted responses for your surveys. This is a huge boon for organizations who are launching new products or services and don’t yet have an existing user or client list that they could contact. We’ll talk more about this when we discuss survey distribution, but for now, if the above situation describes you or your organization, then don’t worry, there is a solution for you.
Writing your online survey — great questions are key to success
We know what we’re trying to find out, and we know who we’re trying to find it out from. Now we need to think about the questions themselves. In general, it’s best to try and keep your survey as short as possible while still achieving the goals you’ve set out. It can be tempting to keep adding extra questions here and there and ending up with something unwieldy, unfocused, and worst of all, boring for the respondent, leading to non-completions. Many organizations fall into this trap, and it’s really a symptom of treating online research as unconnected single-hit projects, rather than an ongoing process.
TOP TIP: As you’re reading this guide, we’ll assume you’re starting out, so as a baseline aim for ten to fifteen questions (though if your goal doesn’t need this many, great!)
For each question, it’s good to think about what you would honestly expect (rather than want) the result to be. As you start with the actual question writing, you need to keep in mind the intended audience for the survey and use this to maintain the right language and tone of voice in the text for each question. The level of formality or specialist language you use should match that which you’d use in other communications while maintaining clarity.
Sticking to conventions
Other things to keep in mind for question writing is that there are certain unofficial conventions that are used across most surveys, and when survey authors go against these, it can make the survey confusing for the respondents.
Scales and ratings
One of the first of these is that we’re used to being asked to rate things on the basis of how “good” they are. So, this means that we tend to associate high scores with a “positive” response. This also applies to how questions are displayed. If people are scoring on a number scale, then you’d expect that if the scale is going left-to-right that the highest number would be at the right-hand edge. Simple enough. But this can also apply for scales that don’t use numbers. People subconsciously expect “good” to be on the right. As in a lot of things, there are always exceptions. The most important thing of all is to be internally consistent, and not swap the conventions you’re using back and forth in the same survey. The specific layout of questions doesn’t really become important until the stage of actually creating the survey, but we mention it now so that when you’re writing, you maintain the same conventions in your question-wording.Beware! False Precision
This also feeds into a related point, which is to beware of false precision. Lots of surveys may ask people to rate things on a 1-10 scale or maybe even more detail, but do we really know that everyone is using the same internal calibration? For some people, the difference between an eight and a nine out of ten might be huge, and for others, it could be trivial. Taking things down the basics of “was something good or bad” can make your survey easier to respond to, pushing engagement. This isn’t always the case and there can be solid cases for taking the opposite approach. Every survey is different and the audiences that take part in each can vary wildly. That said, the general trend over the last few years is towards simpler interactions. In most situations, survey respondents are doing it in their own time, which means that your survey has to compete against the vast array of distractions and other things that compete for the attention of anyone who’s sitting in front of a computer or has a smartphone or tablet in their hands.Avoiding leading responses
Other things to pay close attention to when writing questions is making sure that the questions don’t lead the respondent into giving particular responses. We’ve probably all seen or completed surveys that were set up to supply some predetermined result or other, but for the purposes of this guide, we’re in the business of collecting data to find out something true about the subject.The right answer
As well as questions, it’s likely you will be writing a lot of answers too, particularly when using multiple choice question types. Again these should be kept as short and simple to understand as possible. Other points of good practice for answers are to avoid offering respondents a chance to sit on the fence or give a non-answer. Asking people to rate things on a 4-point scale can be better than asking on a five-point scale as it removes the opportunity to answer with the mid-point. There are probably very few things that your respondents are genuinely indifferent about and making people come down on one side or the other can help you a lot.Making the best use of time
One way to make the best use of respondents’ time is to use skip logic or branching and If this is something that you think will be appropriate for your the survey then it’s best to have a good think about it at this stage, instead of waiting until you’ve started building the survey.
The usual solution to this is to create a flowchart, which is an old-school solution but still effective. Having the user flow marked out like this will make it far easier to set up your rules.
If you’re not usually involved in writing public-facing copy for your organization, it can be a good idea to have someone proofread your questions for clarity before moving on to the next step.
TOP TIP: When you’re thinking about your logic and flow, something to remember is that you should make sure that each page has no more than one question on it that will trigger skip logic rules to avoid conflicts.
Making online survey creation easy
Now comes the easy part – SmartSurvey makes it nice and simple to create your survey using our online builder. We won’t get down right into every click and keystroke as once you start doing that, this becomes more of a technical manual than a guide so instead, we’re going to talk about things in a more general sense.
The whole guide up to this point has been designed around making this part as easy as possible. All you should need to do is choose the appropriate question type from the menu, copy, paste, rinse and repeat.
TOP TIP: One of the best pieces of advice we can give you is to make use of “Descriptive text” questions. These questions act as content blocks where you can add non-interactive content such as text or images. They’re very useful as survey or page introductions and give you a wealth of options for styling and formatting the text.
You should add all your questions and pagination as the first “pass” on the survey and then go back to add any logic functions once this first pass is complete. Once this work is all done, then the next step is the first stage of testing. Test your survey using the preview function for correct functionality, typos, and (for want of a better word) flow.
Flow really just means how the survey feels while you’re completing it. Ideally, a multipage survey should have each page take a similar amount of time and be neither too short or too long. If everything feels good with the survey, then the next step is making it look pretty
Getting the right look & feel
SmartSurvey includes a huge library of themes for choosing the look and feel of your survey. Our themes are responsive and will look good both on traditional PCs as well as on tablets or smartphones. If none of the themes is absolutely perfect for you, an editor is provided that will let you customize a theme with colors and font selections. For those with the most demanding requirements, it’s possible to add custom HTML and CSS to the theme, meaning that you can create surveys that match perfectly with whatever branding they need to.
TOP TIP: You can add graphical elements, like a company or product logo and include images.
The sky is very much the limit with this sort of thing, though it’s worth a reminder that clarity and legibility should always be borne in mind when creating survey themes. Before we move on, we mentioned earlier about measures to counteract straight-lining. You may yourself have noticed these in surveys. What researchers do is put trick questions into the survey with questions like “Answer this question with 1” to check that the respondent is paying attention. You can then use filtering at the reporting stage to exclude responses where the respondent didn’t do this. You can find detailed guides for using specific features on our knowledgebase, a great article to get you started is the Creating Surveys guide. help.smartsurvey.co.uk/article/how-to-create-a-survey
Distribute your online survey — choosing the right channel
With the survey built, we just need to send it to our respondents. There are various options within SmartSurvey for doing this, which each offer different ways of getting your survey to the hands of your respondents. Which one of these you choose is going to be driven by the Goal and Audience you defined at the start of the project
Tracking links
The basis of survey distribution is the “Tracking Link”. Nearly all the distribution methods use a tracking link as part of how they work, and each tracking link can be customised to have its own attributes such as thank you pages, closing time, response limits and more.
Web Links
The simplest kind of tracking link is called a “web link”. These are customisable links to bring users to your survey, which you can distribute however you like. They can be used as part of social media posts, used simply as a link in a web page or email, or printed on paper and handed out (though this particular method is unlikely to yield great results).
Web Embed Codes
Web embed codes allow you to insert your survey directly into a web page, removing the need to navigate to a different site to fill in a survey. Combined with the theming capabilities on offer, your survey can be seamlessly integrated into your website.
QR Codes
If you do need to make it easy to access your survey from the “real world”, then you can use our tool to create a QR code for your link.This instantly makes it far easier for the respondent to get from printedmaterial to the survey as their smartphone can translate the QR code into a URL and take them straight to the survey.
Pop-Ups
Similar to web embed are pop-ups, also known as web intercepts. This option will give you code to incorporate into your website to show a survey to a respondent during a visit. This is a valuable tool for website feedback and comes with a hugely flexible set of options to control the frequency with which it pops up to visitors.
Facebook Posts
The “Facebook” option allows you to create a pre-made Facebook post for the survey
This leaves us to talk about three more options, offline, SMS & email
With the survey built, we just need to send it to our respondents. There are various options within SmartSurvey for doing this, which each offer different ways of getting your survey to the hands of your respondents. Which one of these you choose is going to be driven by the Goal and Audience you defined at the start of the project
Offline surveys
Offline allows you to create a link that, when visited, will allow a simple survey to collect data without access to the internet. The survey is loaded onto a device via an internet connection. It can then display the survey and collect responses anywhere, regardless of connectivity. Once the device is connected to the internet again, the responses can be uploaded, ready for reporting. This is a useful feature for field researchers, whether that’s market research, academic, or even in disaster response zones.
SMS
SMS sending works in much the same way as the email tool, via the upload of contact lists and the creation of invitations. The format of the invitation is of course restricted by the restrictions of SMS as a platform but even so, many clients have seen benefits in response rates by using it as a distribution method. Respondents seem to place higher priority on respondent to SMS alerts than email alerts and their respective inboxes are usually less crowded. There is an extra cost to using SMS, but the increased response rates it can bring make it a sound investment.
The email tool is ideal for targeted communications where you already have contact details for your potential respondents. These can be uploaded as a contact list which you can then use to send an email invitation. These invitations can be styled and branded, which can raise engagement, though it’s important to note that, if your organisation sends sales emails you should differentiate your invitation from these. Once an invitation’s been sent out, you can set up automated reminders to chase the members of the contact list that haven’t yet completed the survey.
If you’re intending to collect data via a variety of sources, it can be a good idea to create new tracking links for each one. This will give you extra options when filtering your data, should you want to. As in many things it’s easier to set this up and then not end up using it than to not do it beforehand and then find yourself trying to answer a question that could have been answered using source link filtering.
If you’re still unsure about which method to choose, check out knowledge base article for more details: help.smartsurvey.co.uk/categories/send-and-collect
Gather data – from action to insight
With the Survey open, and your chosen distribution method selected, it’s time to put the kettle on, make a well-deserved cup of your favourite beverage and wait for the responses to come in. Depending on the number of respondents you sent your survey to, you may need to wait a few hours, or overnight, before moving on to the next stage. If you’re not getting the number of responses
If you’re not getting the number of responses you were hoping, you can wait, try adding new links in new places, reminders, and many other avenues to drum up interest. Another option that’s open is to buy.
Buying responses
SmartSurvey have partnered with leading technology platforms to enable you to access opinions of real people in just a few clicks. Reach over 7 million people in 80+ countries. This can be done with a high level of targeting and the costs can be very reasonable. We’ve found this to be of particular interest to companies looking to launch new products, where they don’t have an internal customer base to draw on, but there are many more applications than this. To do this, get in touch via our online form and we’ll contact you with a quote. One of the best things about this is speed, in getting the results you need quickly instead of having to wait for a couple of days or more to gather data. Often, results can be had within hours. www.smartsurvey.co.uk/services/consumer-panels You’re always going to want to collect as much data, in the form of responses, as you can. There’s a whole field of statistical mathematics based around how much is enough, or (looking at things from the other direction) how accurate your data is likely to be based on the number of responses you have. We have an online page that will calculate these for you, to take the hassle out of it.Generate reports — start making #smartdecisions
You should now have collected a nice chunk of data with which you can go about answering the big questions that you set out back at the start. The summary report page for the survey is the best place to start as it contains the aggregated data for all your respondents. From here, you can directly generate the reports easily to allow you to analyse and digest the information you’ve gathered with your survey. Or, you can head to the “export” tab and create downloadable exports for your data, so you can process it externally. We don’t need to go deep into the statistical analysis side of things in this guide, but here are the general principles for what to bear in mind when looking at your results. These principles should also be the things you focus on if you’re creating reports for distribution through your organisation.
TOP TIP: When creating reports, much like writing the survey itself, it’s important to keep your audience in mind. It rarely pays to make claims for the data that the results can’t support, so be clear in your conclusions and be prepared to justify them based on the data.
The most important things that you’ll end up thinking about, or putting in the reports are the following:
Do the results match the expectations?
If the answer to a given question matches your expectation for it, then that’s great! You now have some solid data to back up what was previously a hunch. If it doesn’t, then that’s also great! You now have a much better idea of the truth, and can act accordingly.What new questions does this data raise?
When the results don’t match your expectations, the most common new question this raises is “why?” That’s a great place to start, as mentioned earlier in this article, research is best when it’s approached as an ongoing, iterative process. If you used a distribution method that allows user tracking, you can see which respondents gave a particular answer to a given question and use this information to follow up with a new survey.Your next survey – keep your finger on the pulse
Even if your survey confirmed everything you already thought, then it’s always good to start thinking about your next survey. Data ages and what was true last week might not be so in the future. Over time, you’ll build up a much better picture of your respondents, however they were acquired, and as you become more proficient and confident using online surveys, then you’ll find that many decisions that were difficult or stressful become much simpler when you’ve got the answer to that “Big Question” on hand.
Right. That’s it! We hope this has been of use for you and we’ve helped give you the confidence to go out there and start doing surveys for yourself. As a final point, it’s not often mentioned just how fascinating the process can be. It’s always the unexpected results that are the most interesting and useful, and the only way to get those is to get out there and start asking questions — so what are you waiting for?