Chapter 3 Conducting the training

This chapter targets trainers, providing details on how to deliver effective and efficient interviewer training that facilitates learning without overstretching survey resources. The chapter assumes in-person training as is the case for most surveys conducted with in-person interviews. The first sub-chapter covers general considerations for the training. The remainder of the chapters is organized by training module. Each sub-chapter provides details on the rational, optimal timing and recommendations for the implementation of the respective module. See the training schedule in Chapter 2 on how individual modules relate and are best combined.

Prior to the start of fieldwork training, make sure everything is fully set up.

3.1 General considerations

Throughout the fieldwork training, consider the following points:

• Essentials only. Only train fieldworkers on things that are relevant for them and that their respective role requires them to do. For example, there is no need for interviewers to learn details about the survey analysis or the supervisor role/functionality in the CAPI software.

• Confirm with designers/analysts. If trainers are uncertain in the interpretation of concepts, questions, or responses, it is important that they keep notes to run these by designers/analysts on a daily basis. Ask them to be on standby so you receive timely responses. Provide updates and clarifications to the trainees in a timely manner.

• Eliminate language barriers. Do not train in a language the trainees are not fluent in. If some trainers (usually the survey designers or analysts) do not speak the training language, use an interpreter or ask other trainers (e.g., fieldwork manager) to translate or reiterate their points in the training language.

• Support trainees. Especially during the first few days, trainees often get lost in the CAPI application or questionnaire flow and are unable to follow the sessions. Trainers who are not actively leading a session should walk around the class, check that trainees are in the right place and support those falling behind.

• Probe for understanding. It is wrong to assume things have been understood just because no one speaks up when you ask, “Any questions?” Trainees may be reluctant to ask any questions, especially if they are used to an education system in which instructors speak without participation and students are simply supposed to listen. Ask questions on specific questionnaire details using common examples or scenarios. Ask variations of your test questions. For example, think of working profiles/jobs that are typical in the surveyed population and probe trainees if these are wage employed or self-employed.

• Encourage questions. From the beginning, make it clear to trainees that they should ask if they have ANY question, that it is better to ask than to remain with a doubt, and that there are no bad questions. Take questions seriously and address them. If trainees prefer not to ask in front of the class, they can raise their hand and be visited by a trainer.

• Involve trainees. Ask frequent questions. Let trainees read aloud questions and manuals. Ask them to correct one another. Avoid always picking the most engaged trainees and try to involve everyone. Select a trainee using a random name picker, so it can be anybody’s turn to answer your questions at any moment (and to make the selection process fun). Another idea is to select the person closest to where a paper ball lands that you or anyone throws over their back. To speed things along, let each trainee you call on cover a couple of survey questions.

• Get to know trainees. Ask trainees to wear name tags during the first few days. Learn their names as quickly as possible. This makes the training more personal and helps trainers to better connect trainees’ in-class performance with written tests or test interviews.

• Collect and react to feedback. At the end of each day, use feedback forms to collect anonymous feedback from trainees on how clear different sessions or trainers were, topics they did not understand or questions they have. Review feedback forms prior to the next day and adjust your training schedule or methods if needed.

• Break up and mix groups. If you start noticing groups of trainees sitting together who are not paying attention or are at the bottom end of performance, distribute them across the classroom. Ask low performers to sit next to high performers, so they can help when falling behind. Avoid grouping the same people for exercises, to ensure that everyone has the same understanding of the material. Friends tend to sit together in the training. Assign a number to each person in a row – 1, 2, 3, 4, 1, 2, 3, 4, 1, 2, 3, 4, etc. - group the 1’s together, the 2’s together, etc. - and move the trainees around every day.

• Use your time effectively. Avoid spending disproportionate amounts of time on rare scenarios. This is counterproductive and takes away time from more frequent and important cases that should be covered. Often, trainees love identifying and pointing out hypothetical issues, e.g., “But what if ….?” If these scenarios are common occurrences, they are worth reviewing during training and may merit updating the questionnaire or manual. If not, instruct trainees to leave comments and alert their supervisors if the situation arises in the field and move on with the training.

• Make it fun! Maintain a general positive and fun attitude. Play fun and exciting group games. The last person entering the venue late and unexcused can sing a song in front of the class (YouTube is full of Karaoke videos). Hold an end of training party.

• Use video & audio. It can be a good way of demonstrating certain things like interviewing techniques or special tasks to trainees, so they can see/hear what they should and should not do and learn from examples. Use existing material if it is good, or collect video/audio material during pre-test, pilots and field practices to demonstrate good and bad practices to the class.

• Stress the importance of their work. Repeatedly make it clear to the trainees that it is important that they do a good job, that the survey result, and ultimately policy decisions, depend on it. Give them a sense of purpose. Like any other human being, interviewers perform better at work if they care about what they do.

• Trainees are responsible for their learning. Make clear to trainees that it is up to them to ensure they have understood the covered material. They can read the manual, practice outside the class, learn from peers, ask questions, etc. to make sure they understand everything. Let them know there is a transparent and objective selection process at the end and they must pass it to become a fieldworker.

• Set and enforce rules. Trainees must attend and sign attendance every day. Ask trainees to be on time and respect training hours yourself by not running late or running over time. No side conversations, no (excessive) use of cell phones and no phone conversations in the training room. Enforce rules in a fun but serious way, e.g., anyone breaking rules can donate to a cookie jar, has to hand in their cell phone, sing a song, etc.

• Zero tolerance for cheating. Make it clear from the beginning that there will be serious consequences for anybody being caught intentionally making up data or purposefully breaking protocols (ideally being immediately dismissed). Also make clear that fieldworkers have nothing to fear as long as they follow what has been taught during training and check with supervisors or fieldwork managers if they are unsure about anything.

• Daily trainer meeting. Trainers should meet daily at the end of the day to compare notes, evaluate progress and trainee performance, review tests and class feedback and adjust as needed the plan for the following day(s). If field supervisors are pre-determined, include them in the meetings.

• Document. Throughout the training, note down any changes to the methodology, known problems and general training details, so the details are available for data documentation and to inform future training or survey rounds.

Tools:

• Ideas for group games
  
• Sample training feedback form
  
• Random name picker to select trainees

3.2 Welcome/introduction

The welcome/introduction session serves to introduce trainees to the survey and training. In many surveys, this is unnecessarily long and involves too many people. Keep it as short as possible (1-2 hours are enough) so that some content can still be covered on the morning of the first day. In the welcome/introduction session cover:

• Opening speeches. Try to keep speeches by officials or leaders as short as politically possible. Also, if possible, avoid making the opening/start of the training dependent on the arrival of guest speakers and government officials, as they might be late (e.g., start with introductions in the meantime).

• Introduction to survey/organization. Give a brief introduction of the survey, its objectives, the organizations involved, the project/study, the different roles and responsibilities, etc. Do not go into too much detail here, limiting it to what is essential for fieldworkers to know. If needed, provide more details in the manual and refer trainees to it.

• Introduction of trainers and trainees. All trainers and core survey team should briefly introduce themselves and their roles. If the class size is large, avoid introduction rounds for trainees as they tend to be dull and ineffective. Play a fun ice breaker instead. Ask all trainees and trainers to write their name on a name tag and wear it during the training.

• Training plan. Give an overview of the rough training schedule and daily working hours. Explain the need for punctuality, that there will be frequent tests and evaluation of skills, that trainees are responsible for their learning, that all questions are encouraged, and that there is zero tolerance for cheating. Point out that the rules and procedures for this survey may be different than in other surveys they might have worked on and that trainees are expected to learn and follow the procedures and rules covered in this training (and not those of other surveys).

• Survey details and terms. Run through the overall survey parameters and contract terms, including the selection process, their remuneration package, insurance coverage, areas to be visited, the expected length of field work, the transport and lodging arrangements during field work, expected effort, etc. Importantly, this should be a refresher and not the first time trainees learn details about their pay, other terms or the fieldwork. Make sure all trainees have understood and signed a copy of the general terms prior to the training.

• Administrative/contractual matters. If you need to address administrative or contractual matters with trainees – contract signing, registering for insurance, signing out tablets, etc. – try to do this as much as possible before regular training sessions (e.g., from 8-10 AM), or in another dedicated slot. Trainers do not need to be present.

3.3 CAPI use

This chapter is based on using Survey Solutions, but the same principles apply to other CAPI software packages.

CAPI, like any other software, is best learned by using it. Covering all functionality in a long theoretical session tends to be very ineffective as any functionality that is not immediately put into practice by trainees is forgotten and needs to be retrained later when it is used. It is more productive – and a better use of time – to start by covering only the basic functionality necessary to begin training, and to gradually introduce additional functionality as required throughout the training. For example, trainees do not need to learn procedures for rejected interviews until they learn about post-interview activities in the latter part of training.

Limit the functionality you train to what is relevant for an interviewer and what will be used in the actual survey. For example, trainees do not need to learn how the Supervisor interface works, what exactly happens at HQ, how assignments are processed or what a multi-select question type is if there are none in the questionnaire.

The fewer slides you have in the CAPI training, and the more demo and practice you do, the better it will be. Train functionality by demonstrating it on the big screen and asking trainees to follow on their own devices. Make sure the class can follow and wait for anyone falling behind to catch up. If someone is lost, they should check with their neighbors or raise their hand, so one of the trainers can come to assist. For example, show on the big screen how to log in to the Interviewer App. Wait until all trainees are logged in on their tablets. Then, show on the big screen where to find an assignment card, explain what it is and show how to create a file. Wait for everyone to open an interview file on their tablet, and so on. Clarify any questions or doubts trainees might have along the way.

You can train the questionnaire directly in CAPI, without training it on paper first, if you are using Survey Solutions or another CAPI software that provides a good overview of the questionnaire, makes routing behavior clearly understandable and allows for free navigation between different parts of the questionnaire. This is usually not the case for software that shows only one question per screen.

Training the questionnaire directly in CAPI has the advantages of requiring trainees to only learn the questionnaire in the format they will ultimately use in the field (e.g., no need to learn how skips work on paper). This also means CAPI will be introduced earlier in training and be used more frequently throughout, allowing trainees to more quickly become confident in using the CAPI questionnaire and the CAPI software in general. New trainees do not notice any difference, and experienced trainees who are used to paper questionnaires usually stop missing them after a day or two.

Train the basic CAPI functionality just before trainees use it for the first time. Usually, around 1 hour is enough. If you are training the questionnaire on paper, train the CAPI basics just before trainees practice for the first time on the tablets, usually on the first or second day.

If you are training the questionnaire directly on CAPI, introduce CAPI basics just before starting with the questionnaire content, usually on the morning of the first day. In these sessions, it is very useful to have trainers roaming through the classroom to look for trainees who may be having problems following the session or who may have a problem with the device. For Survey Solutions, this would mean covering the following topics in the initial CAPI session. Similar steps should be covered for other CAPI software packages.

• A general overview, including how the Interviewer App and HQ server interact, what assignments and interview files are, and what synchronizing does. Keep this part short.

• How to log in to the Interviewer App. and how to create an interview file from their assignment. (Trainers should confirm with design and IT teams that tablets and user accounts have been set up prior to the beginning of training).

• A general overview of the Interviewer App, questionnaire layout, and how to navigate through the questionnaire.

• The layout of a question – the question number, the question text (what to read to the respondent and what not), the instructions, and the answer options (different by type).

• Question and section enablement for the first questionnaire section. Other examples can be presented and explained when training covers those sections.

• The complete screen - marking an interview as complete.

• The dashboard screen/view for Started and Completed interviews, including reopening an interview file and discarding an interview file.

Having completed the above steps, trainees should be able to follow in the questionnaire during the questionnaire content modules. The remaining CAPI functionalities are best trained throughout the training when they are used for the first time:

• During the questionnaire content modules, train the different question types, roster forms, static texts, sections and sub-sections as they come up. For example, explain fill-in lists during the household member module, yes/no filter questions during assets, etc. Trainees need to recognize and understand the difference between single select, multi-select, numeric, fill-in list and date questions, and understand how rosters function.

• If using Survey Solutions, prior to practicing or the first written tests, train how to synchronize devices to receive assignments; how to create new interview files and review interviews on the complete screen; marking interviews as complete and synchronizing to send a file. Conducting these written tests on CAPI tends to be useful, as trainees need to correctly synchronize to get the questionnaire/assignment and submit their test (send the completed interview file).

• Prior to group or field practice, train how to insert comments on questions during the interview and on the Complete Screen, so that trainees can flag their problems or concerns directly in the interview files during practice.

• After field practice or the final field test, train how to receive rejected interviews, respond to comments, and resubmit rejected interviews. This will allow trainees to practice post-interview tasks using interview files they have created themselves.

3.4 Questionnaire content

Trainees must understand the entire questionnaire(s) in detail, including underlying definitions and concepts. If training is dry and monotonic, trainees will lose focus and not retain this substantial amount of content. This is not only an ineffective use of training time but also puts survey quality at risk.

Warning: A single, joint reading of a questionnaire is not sufficient for trainees to learn and understand the various concepts, definitions, and scenarios that are necessary to correctly administer a socio-economic questionnaire. Some implementing agencies (e.g., survey firms accustomed to implementing opinion surveys) might not recognize the effort required for complex socio-economic questionnaires and will vastly under-allocate training time for questionnaire content.

Consider the following points to make the questionnaire content modules interesting and effective:

• Avoid content sessions longer than 2 hours. They tend to be information overload for trainees and make it impossible for them to internalize the content. Spread the content training over multiple days and practice newly-trained material on the same day it is presented.

• Less instructional, more interactive. Avoid one-directional training methods in which trainers give long presentations or read through questionnaires without really interacting with trainees. This tends to be a very ineffective way of teaching questionnaire content. Instead, involve the trainees as much as possible. For each question (or a few at a time), select one trainee to read aloud the question(s) and any instructions, answer options or corresponding manual entry. Ask the trainee to explain in their own words what they have understood and how the question is to be administered. Make sure the rest of the class can hear them. Correct them if needed. Ask the rest of the class for input. Check if the trainee has understood by probing with an example. Give feedback on the tone of voice used to ask the question and the way the trainee asked the question. Ask the trainee to read out the question again if needed.

• Explain the big picture. At the beginning of each section (or part of the questionnaire if you have small sections), provide a brief general overview to the class, including overall structure and routing behavior, things to be aware of, key concepts and definitions, target respondent(s), etc. It is better to explain these details section by section (rather than once at the beginning of the training for the entire questionnaire) so trainees can immediately see and, soon after, practice what has been trained and can thus remember it more easily.

• Go into detail where useful. While it is important that all questions are covered during training, some are more complex than others. For these, correct administration and common problems must be covered in greater depth for interviewers to be able to administer them correctly. This can include explanations on the underlying concepts and definitions, how to categorize responses into the pre-defined answer options, common scenarios that require further probing, what error messages mean, under what conditions the question is asked (what enables it), how the question relates to other questions, how to check that responses make sense, etc. These details are not needed for questions that are straightforward and unambiguous.

• Make it practical and concrete. Definitions for survey concepts such as household, parcel, etc. can be very theoretical and hard for trainees to translate into the real world. When training anything theoretical or abstract, always try to illustrate it with examples and scenarios that are common in the surveyed population so that it is easier for trainees to understand. For example, to explain what wage-employed and self-employed are, list jobs or work profiles that are common and show which category they fall into.

• Mini quizzes. Conduct short quizzes to test understanding, engage trainees and stimulate discussion. Show a few scenarios, images or videos and ask trainees to record their answer(s) on paper. Afterwards, call on individual trainees to share their answer(s) and explain their rationale. Provide feedback and discuss with the class. This is very effective training for questions that require interviewer interpretation (e.g., confirming household members or parcels to be listed) or understanding a variety of classifications (e.g., differentiating toilet type, wall materials, etc.), and helps to streamline interviewer behavior.

• Use the manual. The interviewer manual should provide all necessary instruction for correctly administering the questionnaire. The training is the most important opportunity for field staff to learn and understand the critical details provided in the manual and thus its study should be an integral part of the training. Trainees must develop the habit of consulting the manual for additional information or to clarify doubts, or it will never be used. During the training, ask trainees to keep the manual open on paper or as a soft copy on their device. After reading each question, ask trainees to check if there is a corresponding manual entry and to read it as well. To be correct, it must always be up to date. One of the trainers should update the manual directly during the training if the entry is unclear or insufficient. Ask for input from the class if needed and confirm with them if the updated entry is clear. Distribute updated soft copies of the manual to the team every morning/evening.

• Multi-language questionnaires. If the questionnaire has been translated into other languages, display the questionnaire in the common training language on the big screen and ask each trainee to follow the questionnaire on their tablet in the local language they are most likely to use. For each question, ask it to be read in the training and local languages. This serves to verify the translation and to familiarize trainees with the questionnaire in the local language. If the questionnaire has not been translated and interviewers are required to translate questions on-the-fly, ask interviewers to say out loud the translation for each question and ask the class for feedback. This serves to harmonize translations and for trainees to practice translating the instrument.

• CAPI. Training the questionnaires directly in CAPI, ensures not only understanding of the content and format of questions in CAPI but also demonstrates the flow of the questionnaire. This requires inputting different question routings so that every question is enabled and demonstrated at least once. If using Survey Solutions, set up the training version of the questionnaire so that disabled questions are still displayed. As you work through the questionnaire in CAPI, show and explain CAPI functionalities the first time they come up. On the big screen, demonstrate how the questionnaire should be filled in, how different question types function (e.g., text questions with patterns, search in combo box, etc.), point out instructions or other parts that are not meant to be read out, the overall organization of a section and how to navigate it, etc. Ask trainees to follow on their tablets.

• Demonstrate. Show how (groups of) questions or sections should be administered by role-playing that part of the interview in front of class. Trainers can participate and/or ask experienced trainees to act as respondent and interviewer. This is particularly useful for complex modules or parts that may require additional probing or may not be clear, e.g., probing for the age of household members, completing a roster for a child nutrition module, sketching a parcel map, etc. Trainees can follow and record the responses on their tablet. Give immediate feedback, highlight any mistakes and parts done well, and provide additional information if necessary.

• Update the questionnaire, CAPI & translations. Depending on the level of pre-testing, piloting and subsequent change, the questionnaire, CAPI and translations may still contain issues that are uncovered during training. Ideally, trainers not actively leading the training can make immediate corrections (or at the very least maintain a record of such changes), as it tends to be less time consuming and allows corrected versions to be used sooner during the training. Be consistent and ensure the questionnaire, CAPI program, and all translations are the same. Follow updating protocols when making changes, e.g., update the live version of the paper questionnaire, the CAPI program and the translation sheet. Do NOT simply update the questionnaire on CAPI or in one language only, as this will lead to inconsistencies, mistakes and survey error. Ask the class for input if useful, e.g., how to phrase a question, or what answer categories exist.

• Practice with samples. For descriptive classifications (e.g., water source or stove type) or extracting information from documents it is important to expose trainees to a range of different scenarios and standardize their assessments. Bring a diverse set of samples so that trainees can practice assessing the documents or scenarios in a structured way. One option is to run through scenarios together – show photos of different water access points and ask trainees to record the appropriate response for each. Another option is to set up workstations with different scenarios and ask trainees to rotate through the stations, complete the required task and record the information on their tablets. For example, place different medication packages on different tables; trainees go from table to table, identifying each medication’s name, active ingredients and quantity and recording the information on their tablets. Trainers can staff individual stations and provide personal feedback to each trainee.

3.5 Interview practice

Frequent practice is crucial for trainees to fully internalize the content and be able to apply it in the field. Do not leave interview practice to the end. Throughout the training, practice in groups, in front-of-class role-plays and with respondents on-site and in the field. Each of these practice modalities serves a different purpose.

Towards the beginning of the training, frequent group practice and front-of-class role plays are effective ways for trainees to familiarize themselves with new parts of the questionnaire, learn how to administer them and practice using CAPI. Each day that new questionnaire content is introduced, hold practice sessions at natural content breaks (e.g., between questionnaire sections) to practice the material just covered. Immediately repeating newly learned material helps trainees to internalize it and uncovers doubts before adding new content. Alternating between training modalities (content review and practice sessions) also improves attention levels. When the entire day is dedicated to questionnaire content, aim to have 2-4 practice sessions, keeping in mind the time required to set up each practice.

Once the entire questionnaire has been covered and practiced in pieces, practice full interviews with real respondents on-site and/or in the field. Trainees learn how to handle real respondents and how to conduct interviews under real field conditions – both of which are crucial in preparing trainees for fieldwork. If the interview is long, e.g., an LSMS-ISA style survey with separate household and agricultural questionnaires, it can be beneficial to conduct separate practices for each of the questionnaires. Early exposure during the training to real-world scenarios and respondents also makes the rest of the training more tangible for trainees. Trainees should practice the full questionnaire at least once, but ideally more often, with real respondents prior to the final field test.

More details on each practice modality, including their objectives and how to best organize them, are provided in the sections below.

3.5.1 Group practice

Group exercise is most effective in small groups of 2-8 trainees, followed by productive feedback sessions. When assigning groups, break up clusters that formed in the class and mix trainees so that low/high performers and those paying more/less attention are distributed. Change group composition between exercises.

Tip: One option for quickly assigning well-mixed groups is to work out n, the number of groups needed, and point to each trainee while repeatedly counting aloud from 1 to n until each one is assigned a number. Each number corresponds to a group, and each group can meet in a different point of the training venue, e.g., “All number 4s, meet at the blackboard. All number 3s, at the door…”. For practice in groups of 2, pair trainees with their peers in the row in front/behind them.

Each group practice should focus on the questionnaire sections that have been covered since the last practice session (regardless of practice modality). If there is extra time, they can also practice other sections previously covered in training, but they should not work on sections that have not yet been covered.

Provide practice scenarios to trainees that are likely to occur in their local context. The scenarios should include sufficient details to ensure that key questions are filled out properly, addressing commonly encountered complex scenarios as well as scenarios that will take interviewers through the different question routings in the relevant section. For example, when practicing the labor section, ask groups to record different household members as employed, self-employed, supporting family worker, etc.

Within each group, one trainee should ask the questions while another one answers them. Every few questions/sub-sections the roles should be rotated so that each trainee has the opportunity to act as interviewer or respondent. All trainees in the group should follow along on their own tablets and record the answers which, at the end, should be identical for everyone within the same group. Ask trainees to submit these interviews after the debrief session, further motivating them to correctly complete the section(s).

Groups should also contribute to the debrief sessions by submitting written feedback. Simply asking the class if there were any doubts or comments is often met with general silence. Instead, set a minimum number of written feedback points that each group should submit at the end of the group practice, e.g., 3 during the beginning of training when less material has been covered, and at least 1 towards the end of the training. Feedback points can include any question or doubt about the questionnaire, definitions, scenarios, etc. or any mistakes they have found.

During group practices, trainers observe groups allocated to them, correct mistakes and help struggling trainees on the spot. While observing, they should also rate and record trainees’ skill levels and give individual feedback where necessary. They should take note of any issues to share during the debrief session, including the way questions were read or answered, the intonation, misunderstandings, questionnaire navigation, CAPI use, etc.

Debrief with the entire class immediately after the group practice. Discuss their written feedback and answer questions by involving input from other teams/trainees. Ask trainers to provide feedback based on their observations, explaining what was done wrong and how to do it correctly. Trainees who were observed doing specific details well can be asked to demonstrate to the class (e.g., how to explain a concept/question to a respondent), so that everyone learns from good examples.

3.5.2 Front-of-class role-play

Asking trainees to conduct (parts of) an interview in front of the class is a good way to demonstrate how interviews should (and should not) be conducted and to evaluate trainee skills. It also shows whether the class is able to spot any mistakes made and allows trainers to provide feedback on interviewing to the entire class. It is a good means of practicing (portions of) the questionnaire before practicing the entire questionnaire with respondents on-site. It is particularly useful for critical and complex sections that require more back-and-forth and judgement by the interviewer, such as food consumption modules. This exercise usually takes much longer than a normal interview, so plan accordingly.

Project onto the big screen the tablet that will be used to record responses throughout the role play. It is best to not change the tablet during the exercise.

Start by selecting one trainee to come to the front and act as interviewer. Select one at random that has not yet been to the front. It usually makes sense to rotate trainees after a few questions (around 5-10) – this gives more trainees the opportunity to practice in the front and makes the exercise less monotonous. Select a few trainee-interviewer at a time and have them wait at the front of the room, so the rotation of trainees does not take up too much time.

Select someone to act as respondent throughout the whole exercise. This can be another trainee, or preferably one of the trainers or a real respondent. The benefit of using a trainer is that they can guide the interview details with their answers and probe if some scenarios are understood. For real respondents, hire a person similar to the surveyed population to visit the training venue for an afternoon/morning. These real respondents expose trainees to scenarios and respondents that are similar to what they will experience in the field.

Ask the interviewing trainee to administer the questions as if they were in a real interview and to record the responses on the tablet that is projected onto the big screen. The respondent should give normal responses with some difficult scenarios.

Ask the rest of the class to follow the interview and raise their hand if they have a question or comment about it. Pause the interview and give immediate feedback if the interviewer could have done better. Also highlight what they have done well, so it is clear to the class how the questionnaire should be administered. Consider all aspects of the interview process, including tone of voice, reading speed, way of inquiring, etc. Ask the class for input, e.g., if they spotted a mistake or have any ideas how to do it better. Ask the trainee to repeat the question, taking into account the input.

All trainees should follow along with the front-of-class interview and record responses on their own tablets. At the end, their interview files should look the same. Asking the class to submit their interview files after the exercise is an effective way to encourage them to follow closely and record everything.

3.5.3 On-site respondents

Practicing with real respondents on-site is a great way to give trainees more and early exposure to real-world scenarios, while keeping the logistical effort and required training time low. Trainees learn how to apply the theory they learned and how to handle real respondents. Another advantage is that trainers can observe all or most trainees conducting interviews, which is nearly impossible during field practicing when they are scattered across many homes.

The best time for on-site practice is after the questionnaire has been covered completely, including both group practices and role playing, and prior to field practice. For long questionnaires (or surveys with multiple questionnaires), it can be beneficial to practice with respondents on site after the first part of the questionnaire has been covered, before the second part is trained. This allows trainees to practice and internalize the first part and provides earlier exposure to real respondents and scenarios.

Hire a group of respondents to visit the training venue, or another convenient location, where they will be interviewed. Ideally, arrange one respondent for every 4-5 trainees. They should be similar to respondents in the surveyed population, e.g., cultivate land and keep livestock in agricultural surveys, or have jobs or small businesses in labor force surveys, etc. Make sure to compensate respondents adequately, pay for their transport as well as refreshments and lunch or invite them to the training lunch.

For the on-site practice, group trainees into as many groups as you have respondents. Each group should conduct one full interview with the respondent, taking turns as the interviewer, e.g., asking a few questions or a section at a time. All trainees should follow along on their own devices and record responses. Group members should support each other if individuals fall behind, have questions or make mistakes.

If there is sufficient time, groups can switch respondents and conduct a second (or even third) interview.

Observe, collect feedback and debrief in the same way as described above for group practice. Ask everyone to submit/sync their interviews after the feedback session to further motivate them to implement it correctly.

3.5.4 Field practice

The purpose of field practice is for trainees to get first-hand experience implementing the questionnaire with real respondents in real settings. There is no need to completely mimic listing assignments or replacement protocols as is done in dress rehearsals. Rather, aim to maximize the number of interviews each trainee can conduct and the number of respondents and scenarios they are exposed to. Be well prepared for field practice, observe trainees and collect good feedback to make it a success.

Warning: Field practice is often implemented inefficiently with a lot of time wasted on logistics that could have been prepared in advance: Teams waiting for transport, community introductions, looking for respondents, etc. It is not uncommon that, during a whole day of field practice, trainees only manage to practice for 1 hour or do not practice at all because they do not find any respondents. Prepare well so trainees can practice how to interview - not how to wait.

Field practice sessions should be held after the questionnaire has been trained and practiced in class, and before the final dress rehearsal at the end of the training. Often, it also makes sense to conduct this prior to training sessions on household tracking, receiving rejecting interviews, logistics etc., so that the field test can focus only on the understanding and implementation of the questionnaire. For very long questionnaires (or surveys with multiple questionnaires), it is usually helpful to conduct separate field practices for each of the questionnaires.

Try to be efficient when conducting field practices: if relatively short or only parts of a questionnaire are tested, half-day field practices are often sufficient to give some exposure to trainees and identify issues to be addressed prior to the next field practice. Since it is usually difficult to conduct classroom training after returning from the field, try to conduct these shorter field practices in the afternoon, so you can use the morning to train.

Visit communities that are not too far from the training venue and large enough to have sufficient respondents for all trainees. Practice communities should have similar characteristics to the surveyed communities but should not be part of the sample. Ask support staff or fieldwork management (i.e., those not engaged in the training) to contact or visit the communities the day prior to the field practice, do the community introduction and arrange for respondents at the time of your expected arrival, e.g., with the help of a community leader. Compensating respondents for their participation often helps with their availability and willingness to participate and is usually acceptable during field practice. Again, the aim is for trainees to practice the questionnaire as much as possible with respondents.

Try to minimize other logistical issues that could cause delays. Make sure all tablets are fully charged, synchronized and have battery packs available if needed. Arrange for food and drinks for trainees in the field or ask them to bring them. Make sure your transport arrives on time, knows the way and does not need to stop for petrol. Try to arrive early in communities, as respondents tend to become unavailable during lunch hours.

Group trainees into groups of two, pairing high performers with low performers when possible. Allocate a respondent to each pair. Agree on a meeting place for trainees to return to and a time by which they need to return. If they complete their interviews early, they should return – ideally, to be assigned another interview.

Similar to on-site group practices, trainees should take turns interviewing the respondent while both record responses, and then submit the interview files at the end of the day. Working in pairs, one of them can take notes on any issues while the other one is interviewing. They should write down any issues they experience while conducting the interview. Ask each pair to also list at least one question or doubt they had, and something about their interview or respondent that was noteworthy and is useful to share with the class.

Try to observe as many of the pairs as possible, help and correct where necessary, mark their skills, and note down any issues you have observed as general feedback for the class. Focus on trainees that are borderline of being selected or that have not been marked yet.

Collect as much feedback as possible from trainees before you leave the community. Interviewer recall becomes much harder later and the quality of feedback decreases rapidly. As pairs of trainees return to the meeting place, ask for their written notes, debrief quickly with each pair, clarify any issues they had and record general feedback for the class.

Warning: Often trainers assume that the questionnaire is working well and that interviewers are able to administer it correctly because they did not report any issues during feedback sessions. This assumption is wrong. Many issues cannot be identified by interviewers, are forgotten or not reported. Those reported are biased towards trainees’ convenience, e.g., they are quick to report something that caused them extra effort. Trainers must observe interviews being conducted and compile their own feedback. To receive meaningful feedback from trainees, trainees must note down issues or questions as they experience them and must be debriefed shortly after conducting interviews.

After each field test, hold a general debrief session with the entire class in which you respond to the written feedback from the team, answer any questions, and provide feedback from your observations. This might involve short retraining of concepts or questions that have been ill understood, practice or role-playing. Clarify all issues prior to the next field test. It is normal for quite a few things to go wrong in the first field test and to improve quickly.

Sometimes, you can debrief to some extent directly in the field – in an empty classroom, a community center, the shadow of a big tree, etc. This way, interviews are still fresh in everyone’s head and there is no need to go back to the classroom after the transport back from the field. If you are required to debrief the same day in the classroom, make sure to leave the field in time and to allow for a long break, as everyone tends to be really tired after field practice, especially after the transport back.

Design your training schedule such that field practice days are followed by classroom days, so you can have a detailed debrief session in the classroom with the big screen and all other required facilities.

3.6 Interview techniques

In addition to understanding the questionnaire content, interviewers must master a range of interviewing techniques that are crucial to collect good quality data. Some examples are:

• Introduction. Being able to introduce themselves, clearly explain the purpose of the survey and what participation entails are key determinants of respondent’s participation and the perception of the survey and institution on the ground.

• Securing consent. The ability to persuade reluctant respondents to participate in the survey is crucial to keeping unit non-response error low, particularly in urban areas or other settings with high refusal rates.

• Interview continuation. Avoiding partial refusals (interviews ending before being completed) by keeping respondents engaged in lengthy interviews minimizes item non-response and measurement error.

• Probing. Respondents often do not understand or misinterpret a question, give incomplete answers, respond “I don’t know” or do not want to disclose some information. Interviewers need to be able to inquire in a neutral and non-leading manner to reduce item non-response without introducing bias.

• Controlling interviews. Respondents sometimes digress, give lengthy explanations, or talk about other things. Interviewers need to be able to lead the conversation in a pleasant and courteous manner to maintain the respondent’s focus on the interview.

Warning: Unfortunately, interview techniques are rarely trained. Instead, it is assumed that interviewers automatically know them or will learn them over the course of the survey. In reality, while some interviewers are experienced or naturally good at it, many others struggle or resort to undesirable practices such as making false promises to obtain consent or probing with leading questions. This can exacerbate interviewer effects and lead to high levels of measurement error and unit and item non-response. Training on interview techniques mitigates these effects, e.g., see Groves and McGonagle (2001).

Teaching interviewing techniques goes beyond theory. Interaction between interviewers and respondents encompasses all features of communication, including the choice of words, tone of voice and volume, as well as attitude, facial expressions and body language. To learn a technique, trainees need to see and hear examples of what should and should not be done. To apply these techniques correctly, trainees need to repeatedly practice and receive feedback until they can (re)act correctly and quickly.

The interviewing techniques to be trained depend on the survey and what challenges are expected in the field. For example, refusals may be rare in a household survey but very frequent in an enterprise survey, which would require dedicated training on how to address them. In addition, best practices should be tailored to the culture and context, e.g., how to inquire if respondents are hesitant to disclose their income.

3.6.1 Identify challenges and responses

To determine which interviewing techniques should be trained, identify which ones are most relevant for the survey. Start by compiling a list of key challenges that interviewers are expected to face in the field, focusing on those that are most common and have the biggest effect on survey error. Consider any response, concern or action by the respondent or any other situation that can negatively affect data quality if interviewers do not apply the correct technique. Examples are:

• Respondents refusing an interview, stating reasons related to time, etc.
  
• Other household members being present during sections that should be administered in private.
  
• Respondents being unable or hesitant to quantify consumption amounts.
  
• Long and damaging interview breaks due to a tablet needing to be restarted.
  
• Error messages flagging gross mismatches in harvest quantities, land size or sale prices.

List different ways in which a challenge may manifest itself so that interviewers can recognize them in the field. Include phrases and colloquial terms a respondents would use. As an example, when refusing and stating reasons related to time, respondents may say “I am really busy”, “I need to leave” or “I need to cook/wash/…” when refusing and stating reasons related to time.

For each challenge, identify good interviewer responses. These can include verbal responses, actions or procedures. Often, there is no single response that works equally well with all respondents and in all circumstances. Good interviewers often have a repertoire of responses and tailor them to observable features of the respondent or the context. For example, if a respondent refused based on time and appears to be rushed or pre-occupied with something (e.g., fussing with children) the best response tends to be to ask for a different moment, e.g., by saying “I see that now it is difficult. When would be a good moment for me to come back?”. However, if the respondent refuses because they have to do a chore but seems generally collaborative, it is often good to offer to do the interview while they do their activity.

For each challenge, also identify bad practices or responses that interviewers should avoid, if any. As an example, if respondents do not know the quantity of rice consumed, a bad probing practice would be to ask “Was it maybe 1 kg?”, as this is leading the respondent to your suggested answer.

Allocate challenges and their responses into separate topics, taking into account importance and anticipated time required to train each. It is usually good to have at least one module on Introduction and securing consent, one on Inquiring and controlling interviews and one on Challenges observed during field testing. You might need additional modules on key questions or sections, such as Estimating land size or Completing a consumption module.

Ideally, challenges and their responses should be compiled prior to the training to facilitate planning and make training sessions more productive. Use a combination of any of the following sources to identify challenges and responses:

• Experience from previous surveys with similar context, populations and subject.
• Focus groups with experienced interviewers.
• Observations and feedback from pre-test or pilots.

If you are unable to prepare prior to the training, use challenges and responses observed during field practice sessions or hold short “focus groups” with the class at the beginning of each module.

3.6.2 Train & practice

Schedule training modules on interviewing techniques just prior to field practice and final field test so trainees can apply the techniques soon after. You might need to schedule additional training after field practice and field test to address new situations that came up or retrain those that were not yet implemented well. Modules between 45 min and 1.5 hours tend to be sufficient to cover each core topic.

For each module, first train interviewers on what challenges are likely to occur and what are the best responses. Project onto the big screen the list of challenges (by categories) and good responses. Work through each category by encouraging the class to identify ways in which challenges may manifest themselves and discussing any tips and tricks for a good response. Also cover bad practices that interviewers should avoid. Try to keep this presentation brief so that more time can be spent on discussion and practice.

To establish best practices, identify trainees who are good at a particular task (either because of their interview experience or good interpersonal skills) and ask them to role play in front of the class. Give feedback if necessary and ask the class for input to reach consensus on responses that are appropriate for culture and context.

Trainees should then do practice drills until they are able to respond to a challenge quickly, correctly and naturally. This is best done by asking trainees to role play the interviewer-respondent interactions in front of class. Select one trainee to act as interviewer. Another trainee can act as respondent, though usually it is more useful for a trainer to act as respondent, guiding the conversation as needed. Explain the scenario and challenge and ask the participants to role play as if it was a real field situation, speaking loudly so the entire class can hear. Give immediate feedback on what they have done well (and what they have not). Ask the class for input and if the response can be improved, ask the trainee, or another trainee, to repeat the scene. Repeat these steps until there is agreement that the scene is done well.

Allow time for as many trainees as possible to practice role playing. Depending on class size and skill level, the class may need to be divided into large groups, each supervised by a trainer. This can be a fun way to train and also a good way of breaking up longer theoretical sessions.

Two examples to illustrate:

To practice introduction to the survey, select one trainee and ask them to stand up. Tell them to imagine they have just knocked on a door and they need to introduce themselves. Evaluate their role play based on whether they provide all necessary information, say nothing wrong, are easy to understand, use the right tone of voice, have good facial expressions and gestures, etc. Collect feedback from the class. If needed, ask the trainee to repeat their introduction until it is correct in content and well presented.

To practice securing consent, select a trainee to introduce themselves as above. Another trainee or a trainer acts as a reluctant respondent, stating one of the challenges discussed in the training, such as “I am really busy now” or “What do I get out of this?”. The “interviewer” must try to persuade the “respondent” to participate. Again, together with the class, check if the trainee’s response was quick, correct and natural. Give feedback on possible improvements and let the trainee repeat the scene until satisfactory.

3.7 Expert measurements

In some surveys, fieldworkers take measurements that require relatively skilled expertise and thus need special attention during training. Examples include:

• Anthropometry – height and weight measurements, middle upper arm circumference.
  
• Agriculture – plot size measurements, soil samples, crop cuttings.
  
• Learning assessments – testing literacy or numeracy, early childhood motor skills.
  
• Medical tests – measuring blood pressure or hemoglobin levels, testing for specific illnesses.
  
• Environmental – water sampling and quality testing, lead paint testing.

For these specialized measurements to be accurate and precise, fieldworkers usually need to correctly handle specialized equipment and strictly follow testing procedures. For example, in learning outcome assessment tests, interviewers need to simultaneously handle testing booklets, a timer, and the CAPI questionnaire, while proctoring a young child and maintaining a neutral testing environment. For water quality tests, interviewers need to handle the testing equipment and respect the procedures for taking and testing water samples to ensure reliable results.

Learning to implement any of those tasks at sufficiently high standards usually takes several days. Trainees need to learn theory and practice extensively to gain confidence and be able to consistently take quick and accurate measurements. In many cases, repeated standardization exercises are needed to streamline the way trainees conduct the measurements to ensure their reliability and validity.

If the available training time does not allow for trainees to adequately learn and practice both interviewing and specialized measurements, fieldworker roles can be split - some will conduct interviews while others will focus only on taking measurements. Measurement specialist training can take place in parallel to interviewer training, either at any time from the start of fieldwork training (e.g., if a special profile is required, such as trained nurses), or at a point during training when it becomes possible to assess which trainees are better suited for which role.

To effectively and efficiently train measurement experts, cover the four training modules below.

3.7.1 Theory

First, cover the theory including background and purpose; functionality; calibration and use of equipment; detailed steps of the measurement procedure; things to be aware of or to avoid; how to answer common respondent questions; etc. Make this part tangible for trainees by including demonstrations by experts or videos showing the correct use of tools and measurement procedures. Test the trainees’ understanding using written tests.

3.7.2 Practice each component

Second, practice as much as possible all components of the measurement process, such as setting up testing equipment, taking the sample or reading the measurement. Drill trainees on each component until they can implement them routinely, quickly and correctly. Breaking down the tools and procedures of the overall measurement process helps trainees become familiar with the tasks while not getting overwhelmed. Since test subjects (e.g., children, adults, variety of water sources) might be hard to come by and testing kits can have expensive per-unit costs, another benefit of practicing individual components first is that resources are put to better use.

As an example, when training on anthropometric measurements for children, the following components can be practiced separately:

1. Set-up and handling of equipment. Scales, height and length boards need to be correctly packed, unpacked, and carried around the training center to simulate how they will be carried throughout communities. Trainees also need to practice setting up on uneven terrain. Scales should be calibrated or calibration checked.

2. Measurement procedures. (The type of scales available will inform the measurement procedures). First the caretaker steps onto the scale alone. The specialist either records this weight or sets the tare. Then the child is handed to the caretaker. Finally, the specialist records this reading (to be practiced as a separate step). This process can be easily practiced by trainees using backpacks of various weights in place of the children.

3. Taking and reading measurements. Especially for length, measurers only have a moment to read the measurement, as children are prone to fidgeting out of the measurement position. Depending on the length board, taking the measurement quickly is far from straightforward and requires a significant amount of practice. A significant amount of error can be eliminated at this stage simply by repeated practicing. Practice by asking trainees to measure things (of lengths known to trainers only) and gradually decrease the time available per item. Compare individual measurements to the known length.

3.7.3 Practice full measurements

Third, once trainees have a good command of each component, bring it all together by practicing the entire process with actual subjects or real samples. Provide enough and diverse subjects/samples, rotating them so that trainees can practice repeatedly and are exposed to different scenarios/cases. For example, for plot measurements, trainees can travel in teams to nearby farm households to measure their fields. Observe trainees and provide immediate feedback to individuals or the entire class. Trouble shoot any issues and re-practice individual components if needed. Practice until trainees can implement the entire measurement procedure correctly and have developed a good routine for it.

3.7.4 Standardize

Fourth, conduct standardization exercises to improve the accuracy and precision of trainees’ measurements. In standardization exercises, trainees’ measurements are compared against one another and against those of an expert. The exercises serve to identify and eliminate as much as possible any idiosyncratic measurement practices trainees might have which would cause measurements to deviate from the true value. It also serves as a tests for the final selection of fieldworkers.

Hold standardization exercises towards the end of training, once trainees can implement the measurement procedures reliably and quickly. Often, it is useful to hold at least two separate standardization exercises, 1-2 days apart to allow time to retrain and practice as needed.

For a standardization exercise, set up stations with subjects/samples. For example, for water testing this can be printed photos of water sources (for identification/classification practice), bottles of water from those sources, and the equipment required for testing the water. Ideally, stations have variation in characteristics to expose trainees to a range of scenarios, e.g., pictures of different water sources and water of differing qualities for all the parameters being tested, etc.

The number of stations required depends on the number of trainees. It should be large enough for each trainee to get enough practice and exposure to different scenarios. For larger groups of trainees, it might be best to split trainees into groups and let each group rotate through a subset of stations. Consider the burden on the persons who form part of the station, especially on children for anthropometric measurements.

Start by having the expert measurer take two independent measurements at each station and record the values. Their measurements serve as the benchmark against which trainee measurements are compared and should not be shared with trainees.

Subsequently, trainees should rotate between all stations and take their own measurements. Use different starting points or directions to speed things up and avoid queues at individual stations. Trainees should record their measurements at each station, together with the station name/ID and their own name. This can be done on paper and later entered into a spreadsheet or, ideally, using a short, customized CAPI questionnaire, Google Form or the like. Observe trainees and provide immediate feedback (during practice, not testing). Try to identify why their measurements are deviating and what they need to do to correct it, e.g., extend the child’s legs more to not under-measure their length.

Ideally, two independent rounds of measurement should be taken, so that each trainee has measured each station twice.

The difference between the first and second round of measurements for each trainee at each station is an indication of the precision/reliability of the measurements. The difference between the measurements taken by trainees and by the expert is indicative of their accuracy/validity. Calculate and interpret the technical error of measurement (TEM) to assess the variation of measurements taken by each trainee. For more details, see e.g., WHO/UNICEF guide on anthropometry in children < 5.

Tools:

• WHO/UNICEF guide on anthropometry for children < 5
  
• Video on anthropometric measurements (French)
  
• Video on LandInfo module for soil characterization and identification
  

3.8 Pre-interview tasks

Interviewers have some tasks that must be done before they begin an interview; These actions and the tools for them need to be covered during training. Examples are:

• Identifying & locating/tracking of respondents.
  
• Selection protocols, if the fieldwork includes sampling.
  
• Using the identifier system and assignment cards.
  
• Completing the questionnaire cover and creating an interview file.
  
• Revisit and replacement protocols.

Usually, it is best to train pre-interview tasks and tools towards the end of the training, just before interviewers are able to apply this knowledge in the field. By immediately putting the lessons into practice, trainees learn the tools and tasks more quickly. At this point in the training, they are also already familiar with the CAPI tool and overall questionnaire content, making details like questionnaire identifiers and revisit protocols less abstract. Although these pre-interview tasks will be the starting point for interviewers during field work, it is usually not effective to teach these at the beginning of training. Trainees will learn a lot of other material before they practice these tasks and may forget what was taught before they can apply it. Good moments to train pre-interview tasks are the day prior to field practice and/or the day prior to the final field test, when these components are more tangible.

Most pre-interview tasks are best trained hands-on with concrete examples. Prepare samples of the tools that can be used during training. For example, to train tracking sheets, household identifiers, assignments and how they link, show the tracking sheet for one PSU together with corresponding assignments and demonstrate how they relate. Use the PSUs of the final field test to reduce the preparation effort (The tools need to be prepared anyway for final field test) and to make it very tangible for trainees (they will use them the following day).

Ensure trainees have sufficient practice time, especially if any of the tasks are complicated, e.g., the use of Kish grids for in-field unit selection.

3.9 Post-interview tasks

After completing an interview, there are additional tasks for interviewers to conduct, such as checking for completion, responding to feedback or inquiries, correcting mistakes, revisiting respondents, etc. These tasks are often underdefined and insufficiently trained, which can result in significant measurement and processing error. For example, interviewers may be inclined to “fix” interviews by simply changing recorded responses to make issues go away, without actually addressing the underlying problem. A dedicated module towards the end of the training is needed to teach interviewers what they must – and must not – do.

For surveys conducted in Survey Solutions, one of the main post-interview tasks is to review and respond to rejected interview files. This is used as an example to illustrate how these tasks can be trained. The same principles also apply to other tasks.

Reviewing and responding to rejected interview files is best trained practically, by looking at actual examples. Ideally, trainees can practice with interview files they have produced themselves. This can also show trainees the mistakes they made and how to prevent them in the future. Use interview files from the field practice or even better, the final field test because these interviews will generally be of better quality and be produced under conditions most similar to fieldwork.

Prior to training this module, trainers should carefully review submitted interviews and flag any issues. Normally, the first few interviews tend to have at least one issue per interview. In addition, trainers can also create some scenarios that include specific challenges they want to ensure are practiced. Trainers should write comments and feedback in the interviews as would be done during fieldwork and then reject the interviews so that they bounce back to the interviewer. This should be done with at least one file per trainee (or pair of trainees working together during field test).

To train these tasks, first give a quick general introduction of the rejected files tab on the interviewer dashboard, explaining how to open a rejected file, how to read the rejection comment, and how to see all and respond to supervisor comments. Also review how to process, resubmit, and sync rejected files.

Then, project one of the rejected interviews to the main screen and ask the responsible trainee/pair to come to the front. They should explain what led to their mistake and what they could do if this occurred during fieldwork. Ask them to respond to the issues and resubmit the file on screen. Ask the class to help and comment. Repeat this with as many files/trainees as possible and useful. Ask all trainees who did not solve their file on the big screen to solve it on their tablet and resubmit them.

This exercise is also a great way to identify and streamline the rules around responding to rejected files, revisiting respondents, etc. If any changes are made, update the manual to reflect this.

3.10 Final field test (often called pilot)

At the end of the training but prior to starting field work, a final field test should be conducted in which all survey questionnaires and protocols are tested under real field conditions. It is an important exercise that will:

• Expose interviewers to field-like conditions and give them the opportunity to practice without affecting the sample
• Determine if the field team is ready for field work
• Identify and address any outstanding issues with questionnaire or field procedures

In surveys that do not conduct a separate pilot exercise prior to the training, the final field test is effectively also the pilot (and is referred to as such), in which questionnaires and protocols undergo their first reality check under field conditions. This should be avoided when possible and a separate pilot should be held (see details here).

Usually, 2-3 days of final field testing are necessary but also sufficient. If short debriefing/feedback sessions can be held at the end of each day (or at the start of the next day), they can be on sequential days. If not, schedule debriefing/feedback days between field days. Allow for at least one classroom day after the final field test for an extensive feedback session and to teach how to deal with rejected files, inquiries from HQ or any other post-interview tasks. Additionally, you might need to allow additional time for administrative details, rest days or travel time.

To create conditions as close to fieldwork as possible, the final field test must be conducted with units that are similar to those in the survey sample but, crucially, that are not part of the sample. For household surveys, this usually means conducting the final field test in un-selected enumeration areas within the survey areas and will sometimes require the team to travel from the training location closer to the selected areas. For surveys that are part of RCTs, this means practicing within the treatment areas.

Ideally, the final field test should be located where the entire field team can be supervised and debriefed together in a central location. This ensures that all field workers receive the same input and reduces team effect. If there is a large variability in the sample, split the final field test into multiple locations so that all questionnaire modules and protocols are tested. If doing so, try to streamline as much as possible the supervision and feedback given to all groups, e.g., by trainers exchanging notes before providing feedback to their respective groups.

Test all components of fieldwork, not just how interviews are conducted. This can include activities such as listing and selection of survey units (e.g., households), interviewer assignments, identification and tracking of units (especially for panel surveys), convincing respondents to participate, making appointments and managing time, the flow of the interview through the review/approval process, among others. Trainees should experience the same sort of challenges and issues that will be encountered during actual fieldwork. Do not pre-arrange respondents as one would do during field practice, as this does not reflect real field conditions. If non–response is high, you might need to relax replacement protocols and add additional replacement units to ensure all trainees practice at least some interviews.

Equip interviewers with the same assignments and/or tracking sheets they would receive during field work. Often it is easiest to select the test EAs during the main sample design and include them in all preparation steps, such as assigning EA IDs and adding them to the CAPI questionnaire, procuring reference data/maps, conducting community sensitization, conducting listing exercise (e.g., as part of final field test in lister training), selection of units, producing tracking sheets and assignments, etc. Preparing testing PSUs separately can be very laborious and cumbersome.

If you are unable to prepare listings (e.g., households, respondents) as part of the final field test exercise, send someone to the communities prior to the test to compile a listing. Use the listing to produce the assignments/tracking sheets so that interviewers receive them exactly as they would during fieldwork.

Tip: When doing the final field test for a panel survey’s second (or later) wave, use households that are being ‘retired’ from the panel as they have similar panel features – tracking down of previously interviewed households, interaction with respondents that were previously interviewed, how to navigate interviews with pre-loaded questionnaire data, etc.

Allocate interviewers to field teams and let supervisors assume their management role, including the assignment of cases to interviewers. For the final field test, it is often beneficial for interviewers to conduct interviews in pairs. This reduces the number of cases required in any of the visited communities, increases the probability of finding available respondents and allows interviewers to help one another. Each pair can conduct multiple interviews per day and take turns in administering the questionnaire. As in the field practice, both interviewers should record all answers on their own tablet and submit the interview in the end.

For the final field test, you might need to relax replacement protocols and add additional replacement units, so interviewers can practice conducting some interviews even if non–response is high.

Make sure field teams arrive early in the communities, so that interviewers have enough time to locate cases and conduct interviews before respondents start cooking/having lunch and become unavailable. As for the field practice, make all necessary logistical arrangements to prevent delays or hungry/thirsty field workers in the field.

Collect feedback and debrief as described for field practice. Ask all interviewers to submit their interviews at the end of the day. Use the data to finalize your data system and quality control system, e.g., by updating the data checks. Review submitted interviews using the standard processes of the quality control system. In addition, visually review as many cases as possible and compile feedback for interviewers and the quality assurance team.

The data collected during dress rehearsal is a great way to learn and practice receiving feedback with the team, see chapter feedback.

3.11 Supervisor training

A short training for supervisors is required to prepare them for their role. It should be held outside the hours of the main fieldworker training so supervisors can attend both. Being fully trained as interviewers will allow them to better supervise interviewers.

If supervisors are pre-determined, a short supervisor training can be held prior to the start of the main fieldworker training that covers administrative and logistical matters that do not build upon the main fieldworker training.

If supervisors are selected from the pool of fieldworker trainees, try to select supervisors and conduct part of their training prior to the final field test. This way supervisors can practice their role during the final field test while being observed/evaluated by trainers. Depending on the tasks their role entails, 1-2 days of supervisor training tend to be sufficient. Use the afternoons, evenings or any rest or administrative days after the main fieldworker training. If supervisors are required to conduct interviews for support questionnaires (e.g., community or market), significantly more training time is required.

Prior to supervisor training, supervisor tasks should be clearly defined and written up in a supervisor manual.

The content of the supervisor training tends to be very survey dependent. It can include topics such as:

• Supervisor responsibilities
• PSU tracking and community introductions
  
• Workload management, including assignment of units to interviewers
  
• Quality assurance checks of complete interviews (any unanswered questions, errors, comments?)
  
• Checking PSU completion
  
• Observing interviews, conducting back-checks
  
• Administrative and logistical matters, such as handling finances, accommodation, transport, etc.

Use the final field test to train supervisors on tasks that are best learned by doing. This will provide concrete examples of the types of issues that will need to be addressed during fieldwork and can significantly reduce problems during the beginning of field work, such as incomplete interviews or PSUs. For example, if the supervisor tasks include checking completed interviews against a checklist, they can take turns reviewing a few interviews from the final field test on the big screen while the rest of the supervisors follow along and provide feedback. If their responsibilities include checking PSUs for completion, review with all supervisors the completion status of each of the final field test PSUs, showing how to cross check with tracking sheets.

Although supervisors will be the first point of quality assurance for submitted interviews–checking completion and basic quality assurance of interviews – in-depth data monitoring should be conducted by centralized data monitors to reduce data monitoring effect. Should supervisors be assigned (greater parts of) this role, they need to also be trained as data monitors (see details in the following sub-chapter).

3.12 Data monitor training

Data monitor training is an important component of quality assurance in surveys. It serves to:

• Prepare data monitors for their role, so they can correctly identify issues in the received data, conduct back-checking exercises, take the appropriate actions and provide meaningful feedback.

• Harmonize data monitoring as much as possible, to ensure all field teams receive the same level of scrutiny and to reduce the data monitor effect.

• Improve the data monitoring process. The issues occurring in a survey and the actions required by data monitors tend to differ from survey to survey. Extensive practice using cases from the actual survey is a great way to refine the tools, processes and tasks of data monitors, improving the overall interview review process.

The data monitor training is best held after the final field test, once a good number of interviews created under near-field conditions are available for practice. Importantly, this also allows data monitors to attend the main fieldworker training, providing a solid understanding of the content and the field activities they will be monitoring. Data monitors must be trained and ready to start their work as soon as the first interviews are synced and available for review, typically on Day 2 of fieldwork. It is critical that fieldworkers receive timely feedback, especially at the very start of fieldwork, and that a backlog of un-reviewed cases does not build up.

In most cases, 2-3 days of data monitoring training is sufficient, provided trainers are also available to supervise and work closely with them during the first few days of fieldwork. The training can be held in the afternoons or evenings of the last few days of fieldwork training. If data monitors will work from a central location, training can be conducted on the day the field teams travel to the field and the first day(s) of fieldwork before cases become available for review.

Prior to the data monitor training, prepare a short manual that clearly defines their tasks, procedures, checklist, etc. (see an example here). Add to the manual a list of scenarios that data monitors will face frequently and the actions they should take. For example, If GPS question is missing -> troubleshoot tablet's location setup with interviewer. If not working -> report to survey management. If working -> Check if they can revisit household. If yes -> reject interview back to the interviewer to make another attempt. If no -> comment in interview file, explaining the nature of the GPS issue. Alternatively, the list of common issues and responses can be prepared as a Google sheet or similar file that is made available to data monitors for their regular reference. Update and refine the list throughout training and the beginning of fieldwork.

At the start of data monitor training, provide each trainee with material that data monitors will need to do their work, such as copies of interview cases, back-check questionnaires, tablets or laptops, reports, etc.

Begin training by covering all theory and an overview of expected tasks and procedures. While the details depend on the survey, this may include:

• A general overview of the quality assurance process and the data monitor’s role in it
  
• How to use tools or software required for their role, such as Survey Solutions Supervisor interface, dashboards, etc.
  
• How to check interview completion, use field check tables, etc.
  
• How to review submitted interviews, what to check for, interpreting error messages, etc.
  
• How to back-check cases, conduct re-interviews and audio audits, etc.
  
• How to provide feedback to field teams and report to survey management

Keep presentations concise to allow more time for hands-on training. For example, while explaining how to review an interview file in Survey Solutions, data monitors can follow along on their own devices, immediately practicing how to find comments, see unanswered questions, approve/reject cases, etc.

Next, allot as much time as possible for data monitoring practice so trainees learn and internalize their tasks. Do this using real interviews that are not part of the main sample, such as interviews from field practices. Ideally, use interviews from the final field test, as they more closely resemble the data from actual fieldwork.

Using these interviews makes the training more tangible for data monitors, as they can see the types of issues that are likely to occur during field work and learn how to address them. Another key outcome of the training is to harmonize behavior across data monitors. Use the practice to update and refine the data monitoring process, i.e., the list of key issues to check, types of feedback they should provide and actions they need to take.

An easy-to-prepare and very effective way to practice is to first run through a couple of cases together. Project the necessary material on the big screen in front of class and ask one data monitor to come to the front and work through the case on the big screen. Ask the others to observe and provide input. Provide immediate input if they have done something wrong, missed something, but also if they have done things correctly. Discuss any points of doubt and update procedures if necessary. Repeat this with other cases and other data monitors until all data monitors have worked through several cases.

Another effective training approach (more demanding in the preparation) is to provide the same case for all data monitors to work through and then compare results. As an example, provide each data monitor with the audio recording of the same interview, and ask them to fill in an auditing form while listening to the interview. Once complete, compare the forms as a class, discuss any discrepancies and reiterate important points.

Practice until data monitors are able to correctly and consistently implement all tasks, until you have run out of practicing cases, or until field work has started.

Once the first data is received from the field, it is important that data monitors shift to working on these actual survey cases. Often, however, at this stage data monitors are not fully prepared to implement all monitoring tasks in a consistent, harmonized way. Therefore, it is paramount that data monitors are closely supervised by one of the trainers during the first few days. Good and transparent communication channels between data monitors and survey management are necessary from the start of data monitoring. This will allow management to reply to troubleshooting and protocol clarifications in a way that enables both timely workflow and consistency in how issues are addressed throughout fieldwork.

If working from a central location, data monitors should work from the same room so they can quickly exchange troubleshooting ideas and issues with each other and one of the trainers can observe and provide feedback as necessary. If working from different locations, a common group chat or channel on WhatsApp, Teams, etc. should be set up to facilitate quick and easy communication and ensure all data monitors receive the same information. Data monitors should immediately communicate on the common channel if they have any doubts about a case or scenario or if they have made any noteworthy observations. If necessary, update the manual, tools or processes based on their findings.

Conduct spot checks of some of the cases that have been reviewed by data monitors. Hold regular debriefs with all data monitors to provide feedback.

Although survey management will not be directly involved in the day-to-day data monitoring tasks, they should still be trained on all monitoring outputs so that they understand the basics of the data monitoring tasks and systems and are able to monitor and use the dashboards, reports, etc. and are capable of taking action on them, if needed.

Tools:

Data monitor manual prepared as part of IFAD data collection package

References

Groves, Robert M., and Katherine A McGonagle. 2001. “A Theory-Guided Interviewer Training Protocol Regarding Survey Participation.” Journal of Official Statistics 17: 249–65.