This is the fourth post in our Event Measurement Best Practices series.
In our first post, we looked at the need to move event measurement programs from quantifying what happened at the event, to analyzing the impact of the event on the business. Our second post explored why you should review the event market research that your company already has in place. In our third post, we offered up five tips for how to maximize your sample by ensuring that your surveys are user friendly.
In this post we are going to give you some ideas about choosing event survey software. It’s been top of mind for us, because after ten plus years with our old survey application, we are migrating to new software. Our business is growing and we need tools that leverage our time… along with the investment, support and security a bigger company provides.
To pick the right solution for your needs, you must make a fundamental assessment. Do you need a solution for a specific event, or do you need software that can satisfy a wide range of needs across your organization? Do you need something with bells and whistles, or do you just need something to get out the odd survey?
Here are seven of the criteria that we considered, which might impact your choice.
1/ Branching Logic
An online survey has one huge advantage over its paper equivalent, branching. Using branching (also called skip and progressive) logic, a survey can be programmed to adapt to the respondent as they complete the survey.
Keeping questions relevant to the respondent, is the key to maintaining engagement and getting the largest sample.
Here’s a common situation. You just introduced a new silver bullet gizmo, and you need to determine just how successful the launch event was. You need to ask the Partners and the Customers a lot of the same questions – using branching you can ask the Partners how many the think they are going to sell, and ask the Customers how many they think they are going to buy…
In our decision making process, we put a very high premium on picking a survey application that offered a full complement of logic options. For us this is the “gotta-have” feature that separates the serious tools from the lightweights. Can you think of ways you might apply branching to your surveys?
2/ Question Types
Can you present questions the way you want to, or are you limited to a few pre-formatted designs?
It used to be that if you had a couple of multiple-choice questions, a true/false and a write-in block, you were pretty well set. But people today are used to a more visual, more interactive communication style.
The ability to compare images, rate videos and interact with a specific image may be important to your company. Departments like Training, Customer Support and other groups in marketing may also be able to take advantage of these capabilities.
There are two distinct schools of thought about branding surveys. One school of thought is that a survey should be visually straightforward to communicate that it is an objective, third party instrument.
But there is also research that suggests that in some instances the response rate will improve if the survey is branded. Bottom line, you probably need a certain amount of flexibility to meet the needs of various projects.
Take a look at the graphic tool set. Does it require knowledge of HTML and CSS to get the most out of it? Can you easily incorporate logos, pick backgrounds and implement other details?
If you are looking to the future, you need a solution that is mobile device friendly… That automatically scales so your subject can respond on a smart phone, a tablet, a laptop or even on that old dinosaur, a desktop. After all, you want to make it as easy as possible for people to take your surveys when and where they want to.
5 / Reporting Dashboard
Our old software was great at spitting out numbers and tabular reports, but that was about it. The first thing that we had to do was create charts, usually in Excel. Not only was this time-consuming – it was slow.
Today, many systems offer real time reporting using pre-formatted dashboards of various kinds. It sounds impressive but let’s separate these two concepts.
We are not at all convinced that real time reporting is a good thing. At the end of the day, statistics is about sample size, and the worst possible case scenario is for people to start over-reacting based on what the TV networks call “the early returns”.
But a pre-formatted dashboard, now that is a thing of beauty and genuine value. Once again, how much flexibility and control you will need is a question of resources and application. There are a number of things worth considering.
The first is access. For instance you might want your vendors to be able to see some of the data, but not other parts. How much control do you have over access?
How easy is it to create the reports you want? Can you define how the data will be reported? Can you easily present the data in the look and feel that is expected?
Finally, beyond displaying charts and graphs, how powerful is the software? Does it support going beyond counting responses to doing analysis, such as segmentation?
Which leads us to another key criteria, integration.
In the case of survey software, integration is about two things. How easily software can add additional information from outside sources to individual records. And then, after the survey is complete, how easily it can share the information collected with other software.
One of the things that we routinely do is integrate survey data with registration data. That requires a unique identifier (a linchpin) to connect the individual registration file and the survey file. If you need or want to be able to combine data sets, make sure your solution has this capability.
The second is sharing the data that is collected. This is all about the wonderful world of API’s, Application Program Interfaces, which enable one program to talk to another. The obvious one is exporting a .csv file to Excel. Essential, but boring.
But there is much more that can be done. For instance, our new application has a salesforce.com API –it can port responses directly to a clients CRM system where the data becomes part of the customers record. That’s pretty slick – to say nothing of fast and accurate. It’s the kind of thing that is of tremendous value to the right customer. How can you improve the value of your data to your internal customers?
One of the nice things about the new generation of solutions is that many are cloud-based, so the software can be constantly updated and improved without the user having to deal with it. But all of the whiz-bang features in the world aren’t worth much if you can’t use them, or they don’t work they way that you expect them to, or it takes too long to get things done.
Here are three things we loosely lumped under support, the idea being that each makes it easier for us to get it right the first time.
First, libraries. Take a look at things like pre-built templates and pre-formatted question types which offer quick solutions to last minute projects.
Second, on-line tutorials. Whether you are just learning a system, or want to use a new feature, being able to go online to get a good tutorial is a real time-saver.
Finally, the tech support people. Start with – can you find the phone number on the website??? Talk about pet peeves!
A commitment to support is a direct manifestation of a company’s culture and philosophy. The more people in your organization are going to use the software, the more important support becomes.
So that’s it – seven areas that we looked at and you may wish to explore when you consider survey software applications. If you’ve read this far, you’re probably curious about which survey software solution we chose for Audience Metrix. For these reasons and others specific to our business and our clients, we went with Qualtrics.
Check out this sample survey, it’s an eye-opener!
Are you currently looking for new survey software? What would you recommend about your current survey application? What else would you like to know?