Review Of Three Powerful Online Survey Tools

 |  Posted: by

Online Survey Tools

Updated October 23, 2013

After posting the comparison of three popular online survey tools’ free versions, many have requested a similar analysis of paid online tools. It seems there is a new tool coming up every day and comparing them all would be a rather difficult task. I decided to compare three online survey tools I have used most recently for different projects.

I have been using SurveyGizmo for a while now. As I said in my previous post, it is a great value given the number of features it offers at a very affordable price. The tool has been improving with time so it was a natural choice for comparison purposes.

Not long ago, I saw a demo of Beacon by Decipher and was pleasantly surprised by the advanced features if offered and the business model that supports it. Most online survey tools require either a monthly or annual subscription, but Beacon does not. You only need to pay for completed surveys. I decided to give it a shot and used it to program a pretty complex survey. I had the chance to become very familiar with the tool, and thought it was worth including it in this comparison.

Finally, since my above mentioned post about free online survey tools, I have had many requests asking me to review Qualtrics, which is also becoming quite popular. I happened to use it in two recent projects with very different requirements, which gave me the opportunity to test it in depth.

For a comparison between SurveyGizmo, Qualtrics, and SurveyMonkey check the post: Three Popular Online Survey Tools – What You Get For Your Money.

The list below includes some of the most often-used features in moderate to advanced survey programming and, as you can see below, all three online survey tools are pretty powerful and comparable in many aspects. That said, none of these online survey tool is perfect, and they all would benefit from continuous improvement in capabilities and usability. Most of the time they do the same thing in different ways that have an impact on  users’ learning curve.

 

 
General Overview SurveyGizmo Beacon Qualtrics
Number of surveys Unlimited Unlimited Unlimited
Number of questions Unlimited Unlimited Unlimited
Number of responses Unlimited Unlimited Depends on license
Number of email invitations Up to 500k Unlimited Unlimited
Multi-page surveys
Account users Up to 20 users Unlimited Depends on license
Question types 32 60+ 90+
Graphic survey templates 37

22

+ custom templates

55
Real-time reporting
Data exporting
Data deletion Never Never If account is closed
Supports all languages
Price $75/mo.
or $810/annual subscription

No subscription needed. Pay $0.75/complete. No minimum

From Free trial. $5,000/year
Question Types SurveyGizmo Beacon Qualtrics
Textbox – Short text
Textbox – Numeric Format Validation
Textbox – Email Format Validation
Essay – Long text
Multiple text fields
Table of text fields
Checkbox
Table of checkboxes
Radio button
Table of radio buttons
Bipolar, semantic differential  
Drop-down menu
Tables of dropdown menus
Ranking – Table format
Ranking – Drag & Drop
Ranking – Top X# of items
Ranking – Stars
Constant sum
Continuous sum
Constant sum table  
Descriptive Text/Image
Image choice (radio button/checkbox)
Video Free if using streaming service. Hosted for a fee
Contact information form
Date format  
Heatmap
Image/Text Highlighting  
MaxDiff format  
Card Sorting
Stack Sorting  
Question/Page timer  
File upload
Captcha    
Drill down    
Combination of different question types in one table
Survey Formatting SurveyGizmo Beacon Qualtrics
Customizable font family
Customizable font size
Customizable font color
Customizable background colors
Add/Edit Options in Bulk
Change orientation of labels and options
 
Change position of labels
Editable Next, Submit, and Close buttons text
Editable Next, Submit, and Close buttons graphic elements (color, size, borders)    
Optional question numbering
Fully Customizable XHTML/CSS templates
Can create own CSS templates Simple form
Mobile survey compatible templates
Auto-format as currency, percentage, or a whole number
Editable error messages
Change length, height, and width of textbox or table
Editable survey width
Editable survey after deployment
Min and Max options for numerical answers
Reorder Questions
Randomize the answers
Randomize columns in a table/matrix question
 
Randomize questions within a page
Randomize pages
Set how many checkboxes someone can check
Optional survey tool’s logo
Depends on license
Add your own logo /White label
Create max word size
Separate answer titles and reporting values
Question grouping in blocks
 
Logic SurveyGizmo Beacon Qualtrics
Question-based skip logic
Page-based skip logic
Skip logic based on hidden variables
Unconditional skip logic
Show and hide questions based on logic
Question trigger on the same page
 
Pipe answer titles or reporting values from a question into another
Repeat questions based on answers to previous questions
Repeat pages based on answers to previous questions
Add values from a URL variable as question options
Add custom script / Javascript
Quiz score with tally
 
Quiz score with pass/fail
 
Make questions required
Make the questions soft-required
 
Random assignment of respondents to different paths/cells/blocks
Quota-based logic
Loops
 
Report Creation Options SurveyGizmo Beacon Qualtrics
Real-time online report
Crosstabs Counts, Percentages Counts, Percentages, Means, Top 2 Box Counts, Percentages
Statistical Testing   T-Test, Test of Proportions Chi-Square, T-Test, Test of Proportions
Graphing – Use a bar graph, pie chart or spreadsheet to view answers
Export data to Excel, CSV
Export data to SPSS
Browse all responses individually
Can change titles and internal report values
Clear all responses
Share results with others via a link with or without a password
Share results with all respondents by including a link on the Thank You Page
Add a sharing/bookmarking button    
Export report to PDF
Export report to Word
Export report to Excel
Export report to E-mail
Export report to Power Point  
Survey Deployment SurveyGizmo Beacon Qualtrics
Generate test data
Send email using survey tool
Send reminders using survey tool
Set a cutoff date and time
Set maximum response count
Enable IP blocking
Redirect browser
Link to panel providers
Pass data through survey link
Add a unique survey identifier to each link
Embed surveys into a Webpage
Embed surveys using an iFrame
Collect data in offline mode    
Collect data in Kiosk mode
Convert link into QR code  
JavaScript option embed surveys using JavaScript
   
Question-based quotas
Close survey based on total quota
Remember respondents’ place in survey if completion interrupted
Other Survey Options SurveyGizmo Beacon Qualtrics
Recode values of answer options
Pipe in values from answered questions into later questions’ answers
Pipe in values from answered questions into later questions’ text
Upload images to internal library
Embed images and media via external link
Prevent search engines from indexing survey
Thank You page
Add a “Close” button to Thank You page  
Back Button Optional
Progress Bar Optional
Create hidden variables
Compliance with Section 508 guidelines of the Rehabilitation Act
 
Export survey to PDF
Export survey to Word
Import SurveyMonkey surveys    
Create mobile-compatible surveys
Import data from other data sources  
Polls SurveyGizmo Beacon Qualtrics
Poll templates 37 + Custom   Customizable CSS
Poll Types Single choice, Image Choice, Ranking   Single choice, Multiple Choice
Poll result sharing  
Vote protection By IP address, by cookies, or none   On/Off
Poll Widget sizing  
3rd Party Integration SurveyGizmo Beacon Qualtrics
Open API access   Available for certain accounts
Facebook integration  
Twitter integration  
MailChimp integration    
Salesforce.com integration  
iModerate integration  
ClickTale integration    
ExactTarget Surveys integration    
Freshbooks Surveys integration    
Jive Surveys integration    
Eventbrite integration    
GoDaddy integration    
Twilio SMS integration    
Cint Panel integration    
WhatUsersDo Testing integration    
Support SurveyGizmo Beacon Qualtrics
Support via email
Support via phone
Support – FAQ, Tutorials

Updated 10/23/2013

 

I find SurveyGizmo to be the most user-friendly and you can do pretty advanced programming with it. They have been adding new question types and integration partners. However, the data export to SPSS and result reporting feature need improvement. For some reason multiple choice questions get some random variable names that don’t match the variable name specified in the survey, when they get exported to SPSS. Variable names don’t show up in the online reports either. Also, I wished they included statistical testing in their crosstabs.

Qualtrics is the second most user-friendly survey tool of the three. Some things are not obvious. I like the large number of question types. Many are variations of standard questions with horizontal and vertical layouts, with and without images. Among the question types, you will find Constant sum tables, Heatmaps, Hotspots (The latter is pretty useful in concept testing) and MaxDiff (assuming you already have an experimental design). They recently added a new reporting tool, which is still in beta. I like that you can add banner points to the crosstabs and drill down through different question layers, unfortunately it is still not very intuitive when it comes to create different table types.

Beacon is a powerful tool, but the learning curve is a bit steep. Thankfully, they have a very responsive technical support, and I was able to get answers to my long list of questions and issues, very quickly. What I liked the most about Beacon is the flexibility it offers to create very complex skip patterns and validations. I also liked its crosstabs. They include counts, percentages, means and Top 2 Box with statistical testing. Initially I was mostly bothered by a somewhat cumbersome process for previewing and testing surveys, since I think these are basic features, but they have recently simplified the process and made previewing and testing much more easier and user friendly. A minor thing I don’t like, that is more about my own taste than anything else, is that all questions have a table format and there is no way to change that. The tool seems to have been developed by and for programmers since the environment is not very intuitive for neophytes. You can spend a lot of time trying to figure out how certain basic features work.

Regarding cost, each online survey tool uses a different model. SurveyGizmo’s subscription can be monthly or annual (the latter gives you a 10% discount). Beacon doesn’t require a subscription, but charges $0.75/complete. This may be a good deal compared to SurveyGizmo as long as your project’s sample or the total sample across different projects is 100 or less. If you go over 100 respondents, SurveyGizmo would be a more economical alternative with unlimited completed surveys.

Qualtrics is the most expensive of the three. The number of completed surveys will depend on license cost and you have to commit for a whole year, so consider your research volume before making such commitment. However, if you have need of some of their unique question types (e.g. Hotspot), it may be worth it.

As I said before, no online survey tool is perfect. All have quirks when you get into the details. The decision to use one or the other should be based on how well they meet the needs of the project you are working on and what the budget is.

Comments Comments

Valera Vlasyuk Posted: May 20, 2012

Hi Guys,
You’ve created the best survey feature comparison list I’ve even seen!
How can my online survey tool https://www.phpforms.net get reviewed here?

My best regards,
Valera

Michaela Mora Posted: June 13, 2012

Valera,
Thank you for your feedback. We review new tools from time to time as the need arises and time permits. I’d be interested in seeing a demo of your tool.

Persio Grullon Posted: June 15, 2012

Hi guys
Im in the middle of creating my own business. Need some reliable data from a market research before hand . Want to do right first time. My business is in the packacging industry , that is corrugate and conversion type of business. Great comparison, gut which one will you recomend for my typy of business. I live in Mexico
Thanks
PG

Colin Wheeler Posted: June 18, 2012

Hi,

I’m curious – have been using LimeSurvey (limesurvey.org) and am finding it extremely good.

As a standalone, open-source solution, it’s effectively free (or the cost of a server in the cloud) and with an amazing amount of flexibility. Already hosted versions are readily available and are low, low cost.

How would you rate this in comparison to other tools or have you not come across it before? If the latter, are there other full open-source solutions which you would recommend.

Regards, Colin.

Michelle Posted: September 14, 2012

The tools mentioned in the list are good but i think it still is incomplete without http://www.sogosurvey.com as it is ideal for users who may be new in the field for survey creation and very helpful to those who do researches with surveys. They even provide accounts to Non-Profit organizations.

It may not be the best but certainly knows what it’s upto.

Adi Bhat Posted: September 11, 2013

Another good pick for a online survey tool is http://www.questionpro.com

Its affordable and really easy to setup, it also has all the advanced features that any surveyor would require.

SamDaniel Posted: September 13, 2013

I don’t agree to this list. I would say in my list there would be:
https://www.sogosurvey.com/
https://fluidsurveys.com/
https://www.surveymonkey.com/

Fábio Posted: October 28, 2013

Hi,

I did a search on Qualtrics but I got over 250 responses and I can’t find a way to buy this more questions. I sent any emails but they don’t respond.. Anyone know how to buy all the answers?

Regards,

Romain Baczyk Posted: November 13, 2013

Dear Michaela,

This is an excellent review; I really enjoyed this article and immediately subscribed to your blog. I have been in the software for market research business for almost a decade and this is one of the best software comparisons I have ever seen. I represent Q-FI Solutions; we are Market Research technology company. We have just launched a new survey platform with unique capabilities. I would love to schedule a live demonstration at your convenience. Please advise of your interest.

Regards,
Romain

Romain Baczyk, PRC
EVP, Global Sales & Business Development
Q-FI Solutions

https://qfisolutions.com/

Bruno Posted: May 21, 2014

SamDaniel: the ones you mentioned offer many fewer options and although more accessible in price are not nearly as good if you’re willing to lay down the money.

TANG ANGEL Posted: June 11, 2014

Hi, such a wonderful review. I wonder what is the currency for the price of the subscribing the tools? Is it presented in USD?

Michaela Mora Posted: June 21, 2016

Yes, it is in USD.

Only logged in users can leave comments.