Updated October 23, 2013

After posting the comparison of three popular online survey tools’ free versions, many have requested a similar analysis of paid online tools. It seems there is a new tool coming up every day and comparing them all would be a rather difficult task. I decided to compare three online survey tools I have used most recently for different projects.

I have been using SurveyGizmo for a while now. As I said in my previous post, it is a great value given the number of features it offers at a very affordable price. The tool has been improving with time so it was a natural choice for comparison purposes.

Not long ago, I saw a demo of Beacon by Decipher and was pleasantly surprised by the advanced features if offered and the business model that supports it. Most online survey tools require either a monthly or annual subscription, but Beacon does not. You only need to pay for completed surveys. I decided to give it a shot and used it to program a pretty complex survey. I had the chance to become very familiar with the tool, and thought it was worth including it in this comparison.

Finally, since my above mentioned post about free online survey tools, I have had many requests asking me to review Qualtrics, which is also becoming quite popular. I happened to use it in two recent projects with very different requirements, which gave me the opportunity to test it in depth.

For a comparison between SurveyGizmo, Qualtrics, and SurveyMonkey check the post: Three Popular Online Survey Tools – What You Get For Your Money.

The list below includes some of the most often-used features in moderate to advanced survey programming and, as you can see below, all three online survey tools are pretty powerful and comparable in many aspects. That said, none of these online survey tool is perfect, and they all would benefit from continuous improvement in capabilities and usability. Most of the time they do the same thing in different ways that have an impact on  users’ learning curve.

General Overview
Number of surveys	Unlimited	Unlimited	Unlimited
Number of questions	Unlimited	Unlimited	Unlimited
Number of responses	Unlimited	Unlimited	Depends on license
Number of email invitations	Up to 500k	Unlimited	Unlimited
Multi-page surveys
Account users	Up to 20 users	Unlimited	Depends on license
Question types	32	60+	90+
Graphic survey templates	37	22 + custom templates	55
Real-time reporting
Data exporting
Data deletion	Never	Never	If account is closed
Supports all languages
Price	$75/mo. or $810/annual subscription	No subscription needed. Pay $0.75/complete. No minimum	From Free trial. $5,000/year
Question Types
Textbox – Short text
Textbox – Numeric Format Validation
Textbox – Email Format Validation
Essay – Long text
Multiple text fields
Table of text fields
Checkbox
Table of checkboxes
Radio button
Table of radio buttons
Bipolar, semantic differential
Drop-down menu
Tables of dropdown menus
Ranking – Table format
Ranking – Drag & Drop
Ranking – Top X# of items
Ranking – Stars
Constant sum
Continuous sum
Constant sum table
Descriptive Text/Image
Image choice (radio button/checkbox)
Video		Free if using streaming service. Hosted for a fee
Contact information form
Date format
Heatmap
Image/Text Highlighting
MaxDiff format
Card Sorting
Stack Sorting
Question/Page timer
File upload
Captcha
Drill down
Combination of different question types in one table
Survey Formatting
Customizable font family
Customizable font size
Customizable font color
Customizable background colors
Add/Edit Options in Bulk
Change orientation of labels and options
Change position of labels
Editable Next, Submit, and Close buttons text
Editable Next, Submit, and Close buttons graphic elements (color, size, borders)
Optional question numbering
Fully Customizable XHTML/CSS templates
Can create own CSS templates		Simple form
Mobile survey compatible templates
Auto-format as currency, percentage, or a whole number
Editable error messages
Change length, height, and width of textbox or table
Editable survey width
Editable survey after deployment
Min and Max options for numerical answers
Reorder Questions
Randomize the answers
Randomize columns in a table/matrix question
Randomize questions within a page
Randomize pages
Set how many checkboxes someone can check
Optional survey tool’s logo			Depends on license
Add your own logo /White label
Create max word size
Separate answer titles and reporting values
Question grouping in blocks
Logic
Question-based skip logic
Page-based skip logic
Skip logic based on hidden variables
Unconditional skip logic
Show and hide questions based on logic
Question trigger on the same page
Pipe answer titles or reporting values from a question into another
Repeat questions based on answers to previous questions
Repeat pages based on answers to previous questions
Add values from a URL variable as question options
Add custom script / Javascript
Quiz score with tally
Quiz score with pass/fail
Make questions required
Make the questions soft-required
Random assignment of respondents to different paths/cells/blocks
Quota-based logic
Loops
Report Creation Options
Real-time online report
Crosstabs	Counts, Percentages	Counts, Percentages, Means, Top 2 Box	Counts, Percentages
Statistical Testing		T-Test, Test of Proportions	Chi-Square, T-Test, Test of Proportions
Graphing – Use a bar graph, pie chart or spreadsheet to view answers
Export data to Excel, CSV
Export data to SPSS
Browse all responses individually
Can change titles and internal report values
Clear all responses
Share results with others via a link with or without a password
Share results with all respondents by including a link on the Thank You Page
Add a sharing/bookmarking button
Export report to PDF
Export report to Word
Export report to Excel
Export report to E-mail
Export report to Power Point
Survey Deployment
Generate test data
Send email using survey tool
Send reminders using survey tool
Set a cutoff date and time
Set maximum response count
Enable IP blocking
Redirect browser
Link to panel providers
Pass data through survey link
Add a unique survey identifier to each link
Embed surveys into a Webpage
Embed surveys using an iFrame
Collect data in offline mode
Collect data in Kiosk mode
Convert link into QR code
JavaScript option embed surveys using JavaScript
Question-based quotas
Close survey based on total quota
Remember respondents’ place in survey if completion interrupted
Other Survey Options
Recode values of answer options
Pipe in values from answered questions into later questions’ answers
Pipe in values from answered questions into later questions’ text
Upload images to internal library
Embed images and media via external link
Prevent search engines from indexing survey
Thank You page
Add a “Close” button to Thank You page
Back Button Optional
Progress Bar Optional
Create hidden variables
Compliance with Section 508 guidelines of the Rehabilitation Act
Export survey to PDF
Export survey to Word
Import SurveyMonkey surveys
Create mobile-compatible surveys
Import data from other data sources
Polls
Poll templates	37 + Custom		Customizable CSS
Poll Types	Single choice, Image Choice, Ranking		Single choice, Multiple Choice
Poll result sharing
Vote protection	By IP address, by cookies, or none		On/Off
Poll Widget sizing
3rd Party Integration
Open API access			Available for certain accounts
Facebook integration
Twitter integration
MailChimp integration
Salesforce.com integration
iModerate integration
ClickTale integration
ExactTarget Surveys integration
Freshbooks Surveys integration
Jive Surveys integration
Eventbrite integration
GoDaddy integration
Twilio SMS integration
Cint Panel integration
WhatUsersDo Testing integration
Support
Support via email
Support via phone
Support – FAQ, Tutorials

Updated 10/23/2013

I find SurveyGizmo to be the most user-friendly and you can do pretty advanced programming with it. They have been adding new question types and integration partners. However, the data export to SPSS and result reporting feature need improvement. For some reason multiple choice questions get some random variable names that don’t match the variable name specified in the survey, when they get exported to SPSS. Variable names don’t show up in the online reports either. Also, I wished they included statistical testing in their crosstabs.

Qualtrics is the second most user-friendly survey tool of the three. Some things are not obvious. I like the large number of question types. Many are variations of standard questions with horizontal and vertical layouts, with and without images. Among the question types, you will find Constant sum tables, Heatmaps, Hotspots (The latter is pretty useful in concept testing) and MaxDiff (assuming you already have an experimental design). They recently added a new reporting tool, which is still in beta. I like that you can add banner points to the crosstabs and drill down through different question layers, unfortunately it is still not very intuitive when it comes to create different table types.

Beacon is a powerful tool, but the learning curve is a bit steep. Thankfully, they have a very responsive technical support, and I was able to get answers to my long list of questions and issues, very quickly. What I liked the most about Beacon is the flexibility it offers to create very complex skip patterns and validations. I also liked its crosstabs. They include counts, percentages, means and Top 2 Box with statistical testing. Initially I was mostly bothered by a somewhat cumbersome process for previewing and testing surveys, since I think these are basic features, but they have recently simplified the process and made previewing and testing much more easier and user friendly. A minor thing I don’t like, that is more about my own taste than anything else, is that all questions have a table format and there is no way to change that. The tool seems to have been developed by and for programmers since the environment is not very intuitive for neophytes. You can spend a lot of time trying to figure out how certain basic features work.

Regarding cost, each online survey tool uses a different model. SurveyGizmo’s subscription can be monthly or annual (the latter gives you a 10% discount). Beacon doesn’t require a subscription, but charges $0.75/complete. This may be a good deal compared to SurveyGizmo as long as your project’s sample or the total sample across different projects is 100 or less. If you go over 100 respondents, SurveyGizmo would be a more economical alternative with unlimited completed surveys.

Qualtrics is the most expensive of the three. The number of completed surveys will depend on license cost and you have to commit for a whole year, so consider your research volume before making such commitment. However, if you have need of some of their unique question types (e.g. Hotspot), it may be worth it.

As I said before, no online survey tool is perfect. All have quirks when you get into the details. The decision to use one or the other should be based on how well they meet the needs of the project you are working on and what the budget is.