PILOT TESTING OF THE QUESTIONNAIRE


Pilot testing also called pre-testing means small scale trial run of a particular component; here we are referring to pilot testing of the questionnaire.

research methods business mathematics statistics  PILOT TESTING OF THE QUESTIONNAIRE

Conventional wisdom suggests that pre-testing not only is an established practice for discovering errors but also is useful for extra training the research team. Ironically, professionals who have participated in scores of studies are more likely to pretest an instrument than is a beginning researcher hurrying to complete a project. Revising questions five or more times is not unusual. Yet inexperienced researchers often underestimate the need to follow the design-test-revise process.

It is important to pilot test the instrument to ensure that the questions are understood by the respondents and there are no problems with the wording or measurement. Pilot testing involves the use of a small number of respondents to test the appropriateness of the questions and their comprehension. Usually, the draft questionnaire is tried out on a group that is selected on a convenience and that is similar in makeup to the one that ultimately will be sampled. Making a mistake with 25 or so subjects can avert the disaster of administering an invalid questionnaire to several hundred individuals. Hence the main purpose of pilot testing is to identify potential problems with the methods, logistics, and the questionnaire.

Administering a questionnaire exactly as planned in the actual study often is not possible. For example, mailing out a questionnaire might require several weeks. Pre-testing a questionnaire in this manner might provide important information on response rate, but it may not point out why questions were skipped or why respondents found certain questions ambiguous or confusing. The ability of personal interviewer to record requests for additional explanation and to register comments indicating respondent’s difficulty with question sequence or other factors is the primary reason why interviewers are often used for pretest work.

What aspects to be evaluated during pilot testing?

1. Reactions of Respondents:

The reactions of the respondents can be looked at from different angles. The researcher may be familiar with the local culture; still getting the first hand experience is always useful. Going to the field, contacting the people, and their reactions to the different aspects of research may be a learning experience.

  • Availability of study population timing. In case we are doing interviewing then pre-testing might help to find out the most appropriate time when the respondent shall be available. The researcher can plan the interviewing accordingly.
  • Acceptability of the questions asked. An important purpose of pre-testing is to discover participants’ reaction to the questions. If the participants do not find the experience stimulating when an interviewer is physically present, how will they react on the phone, or in the self administered mode? Pre-testing should help to discover where repetitiveness or redundancy is bothersome or what topics were not covered that the participant expected. An alert interviewer will look for questions or even sections that the participant perceives to be sensitive or threatening or topics about which the participant knows nothing.
  • Pre-testing will also provide the opportunity to see the acceptability of the wording of the questions in the local cultural context. Some of the issues may be discussed openly while for others people use a disguised language. If people consider the use of certain phrases as offensive, then it is high time to change the wording.
  • Willingness of the respondents to co-operate. Field testing of the questionnaire will give the idea about the level of cooperation the research team is likely to get from the respondents, particularly if they have to interview them.

2. Discovering errors in the instrument:

  • Do the tools provide you the information? Reliability. Suitability for analysis. Tabulation of the results /of a pretest helps determine whether the questionnaire will meet the objectives of the research. A preliminary analysis often illustrates that although respondents can easily comprehend and answer a given question, it is an inappropriate question because it does not help solving the issue. The information may not be suitable for analysis.
  • Time taken/needed to interview/conduct the observation. Pre-testing can indicate the time taken for interview or to conduct the observation. Too long questionnaires may not be recommended and, therefore, need modification. It can also help in estimating average time being taken to collect information form a respondent. Such an exercise can help in budget estimations.
  • If there is any need to revise the format of the tool. Question arrangement can play a significant role in the success of the instrument. May be we should start with stimulating questions and place sensitive questions last. Such a situation might be handled through pretesting. Therefore, pre-testing may help in putting questions in proper sequence, using acceptable wording, doing appropriate translation, question spacing, structuring of answers, coding system, and needing instructions for interviewers (probing).

3. Sampling procedure can be checked:

  • The extent to which instructions given are followed. Field functionaries are given the instructions for following a sampling procedure. Depending upon the type of sampling to be followed, the field worker must follow the guidelines otherwise the quality of the study will be hampered. During the pre-testing one could see not only the extent to which the instructions are being followed but also locate the problems in carrying out those instructions. Also what could be the solutions to those problems?
  • How much time is needed to locate the respondents? By following the instructions how easy it is to locate the respondents, and how much time is needed to do that activity. It could help in calculating the overall time for data collection, having relevancy for budgeting thee resources.

4. Staffing and activities of research team can be checked:

  • How successful the training has been? Pre-testing can be seen as a period of extra training. The pre-testing exercise can provide a good opportunity to make an evaluation of the achievement of the objectives of training. For any deficiencies additional training may be provided.
  • What is the work output of each member? The researcher can calculate the average output of each fieldworker and accordingly calculate the number of workers needed to finish the work on time. It can also help in making the budget estimates.
  • How well the research team works together? It is a good opportunity to observe the kind of coordination the research team has. The integrated work is likely affect the efficiency of the team. Any shortcomings could be looked after.
  • Is the logistical support adequate? Of course we are leaving the field functionaries in isolation. They shall be in need of other logistical support like the transportation, boarding, lodging, guidance and supervision. Some of these aspects could also be appraised during the pre-testing

5. Procedure for data processing and analysis can be evaluated:

Make dummy tables. See how can we tabulate the data and use the appropriate statistics for purposes of interpretations

VN:F [1.9.14_1148]
Rating: 9.7/10 (3 votes cast)
VN:F [1.9.14_1148]
Rating: +2 (from 2 votes)
PILOT TESTING OF THE QUESTIONNAIRE , 9.7 out of 10 based on 3 ratings