Surveyitis

The Symptoms

Are you beginning to feel that those telephone interviewers who call your house at dinnertime have taken over your company’s call center? Is your e-mail clogged with requests to help improve your workplace by responding to yet another online survey? If you divide your business mail into three piles (surveys, advertising, and work related memoranda), is the survey pile higher than the other two? If the answer to these questions is “yes,” your organization may be suffering from “surveyitis,” a common ailment of organizations that are moving toward fact-based decision making.

Now, fact-based decision making is indeed an important management tool. And employees are often the best resource for obtaining the necessary facts. However, the process of obtaining facts from employees involves skills that employees need to learn in order to be effective. Organizations often invest in consultants and marketing experts to conduct customer satisfaction surveys and even employee satisfaction surveys, but the same rigor is rarely applied to surveys conducted internally.

What are the consequences of poorly designed internal surveys? Reduced employee satisfaction, wasted resources, and poor decisions! If you are from one of the organizations with “surveyitis,” you have probably felt the “anger” of being required to fill out a survey that is wasting your time (because it is so poorly worded you can’t respond effectively, because you were an inappropriate person to ask in the first place, or because the results aren’t likely to be acted upon). Reducing employee satisfaction and wasting employees’ time aren’t the only consequences of poor survey techniques. Leaders often have to make decisions on incomplete data, but the data shouldn’t be incomplete because your survey didn’t ask the right questions! Poorly designed surveys, inappropriate data analysis, and nonrandom samples can misrepresent data, resulting in poor decisions!

Preventive Medicine

How can you avoid “surveyitis” without hiring survey firms to conduct all your surveys? Build internal expertise in basic survey techniques and follow the seven survey strategies provided below.

Strategy 1. Begin with the business question clearly in mind. The business question will help you determine what information is needed, by when and how the information will be used. This information helps in the design of a survey plan, including the outline of the survey contents, the schedule for survey design and implementation, the budget, and who will work on the project. Throughout the survey project, the business question will serve as an “anchor,” assuring that the survey doesn’t lose focus.

Strategy 2. Investigate the best way to obtain the needed information. Internet surveys are so easy and cheap that people often just assume that the Internet is the only way to go. Paper and pencil surveys are often the second choice for information gathering. Before you jump to conclusions, investigate whether the information already exists! Sometimes other groups have done surveys on almost the same topic. Even if the data isn’t exactly what you need, you might be able to use the earlier survey questions in your survey. Benchmark information on your topic may be found through a library search. Or perhaps you will find that there are better ways of obtaining information than by asking employees. Are there logs or data records that have the information you need? Suppose your team is concerned with the cleanliness of the corporate cafeteria. You could ask employees how they “perceive” the cleanliness of the cafeteria, or you could observe the cleanliness of the cafeteria at various times. Either or both data collection methods may be appropriate depending on the business question. The point is that data can be collected in multiple ways – employees’ time is valuable and surveys should be used only when appropriate. If you do decide to conduct a survey, remember that they may be conducted with face-to-face interviews and telephone interviews as well as with the more common paper and pencil and online media.

Strategy 3. Design your sampling plan carefully. Who has the information you need to answer your business question? Be precise. Suppose you are studying the user friendliness of an online tracking system used by one administrative assistant in each department. Do you want to survey all administrative assistants in your company? Of course not. Do you want to survey a random sample of administrative assistants in your company? No! You want to survey only those employees with direct experience with the online tracking system. Be sure your sampling plan includes strategies to avoid nonresponse. Effective communications about the importance of the survey, follow-up postcards or e-mails, and prizes for participants are some of the strategies used to encourage response.

Strategy 4. Survey questions must be clear and mean the same thing to all respondents. Writing good survey questions isn’t easy. Draft your survey, have the questions reviewed by multiple people, and finally, pilot your survey. Remember, keep it short! Employees’ time is valuable – don’t waste it looking for answers that would be “nice to know” but aren’t directly linked to the business question.

Strategy 5. Develop and implement an internal communication plan around your survey. How you conduct your survey sends strong signals to your employees – about how well you know the subject of the survey and about how you respect your colleagues. A good communication plan will encourage people to respond, make them understand the importance of the information they are providing, and set expectations for how the results will be used. The communication plan needs to consider all the employees – not just those who will be directly involved in the survey. For example, if you are going to conduct a half-day face-to-face interview with each of 20 people from a department, the department manager would want to know. If that department manager encourages employees to cooperate, the likely success rate of your survey increases! The communication plan also needs to include how you are going to assure confidentiality (or not) and how you will inform employees about the confidentiality arrangements. Be honest – trust is very hard to build once it is lost. Follow through on any promises. If you promised to provide respondents the results of the survey, then make the results available. Communicate how the results are being used. I know one company that posts employee survey results next to the bathrooms, along with information from the action teams who have been charged with taking action on the survey results. Employees who know their opinions are valued and used will be more apt to respond to the next survey.

Strategy 6. Take action on your results. Unless you are doing basic research and gathering information for the joy of learning, you are conducting a survey in order to inform a business question. If the results of your survey are not specific enough to make a decision, then your survey was poorly designed. You can avoid this possibility by adding a “simulation” step in the design of your survey. Once you have written your questions, “simulate” possible results and how you would analyze them. If you have no idea how to take action on your simulated data, real data will be just as difficult.

Strategy 7. Provide expertise on survey design to teams or employees who may need to conduct surveys. Some organizations have a central area responsible for conducting surveys. The employees in such areas are often trained professionals in survey design. These individuals can be given the responsibility to provide coaching as requested. Other organizations may provide a half-day or full-day course to individuals who are conducting surveys. Still other organizations hire consultants to help design high-profile surveys (perhaps of customers) and assign internal employees to work with the consultant and learn survey techniques from the consultant. However it is done, building the in-house expertise on survey design can strengthen an organization’s fact-finding effectiveness.

Cured!

The seven simple steps to avoid the dangerous side effects of poorly designed surveys sound like common sense, but they are often forgotten or ignored by organizations in their early implementation of fact-based decision models, including Six Sigma or Total Quality Management. In the enthusiasm for “gathering facts,” the process of fact gathering may be overlooked. Training programs emphasize the statistical tools, but may slide over the softer skills of “designing questions.” By conscientiously using these seven simple steps, surveyitis will become a thing of the past!

Cheryl Wild is the president of Wild & Associates, Inc., which helps organizations improve the reliability and validity of surveys, assessments, evaluations and focus groups. Her company specializes in improving internal expertise for ongoing performance improvement. She can be reached at 539 Main Street, Avon-by-the-Sea, NJ 07717, 732-774-5188


Connect with Cheryl on LinkedIn