2022.12.08
Effective Use of Surveys for Training Effectiveness Measurement
However, many may wonder, "At what timing and with what content should surveys be conducted?"
The key to effective surveys lies in designing them according to the timing and objectives of measurement.
This article provides an in-depth guide on how to effectively utilize surveys for evaluating training effectiveness.
1) The Role of Surveys in Training Effectiveness Measurement
Surveys are one of the most accessible tools for measuring training effectiveness.
Their simplicity—posing questions to learners and collecting responses—makes them easy to implement. Furthermore, surveys are flexible:
- They can be conducted anytime (before or after training).
- They can be implemented anywhere (offline or online).
Due to these characteristics, surveys serve as a valuable tool for training evaluation and should be actively integrated into measurement strategies.
However, conventional survey methods focused solely on evaluating training content (e.g., “Was the instructor’s explanation clear?”) may not provide sufficient insights into effectiveness.
This is because training has shifted from an informational format to a performance-oriented approach, changing its primary objectives.
1)-1 Training Ensures Two Key Outcomes: “Mindset Change” and “Behavioral Change”
Modern training aims to enhance learners’ performance. Specifically, it seeks to provide employees with the knowledge required to execute business strategies and achieve goals, thereby fostering new perspectives and actions.
Training effectiveness measurement is critical because it evaluates how employees, as investment targets, have changed through training.
Thus, the key points of training effectiveness measurement are:
- Mindset Change (Changes in values and perspectives that influence decision-making in the workplace)
- Behavioral Change (The ability to perform actions that lead to business outcomes)
While surveys are often associated with training satisfaction assessments, they can be structured to capture a broader range of information, including mindset and behavioral changes.
Understanding these two key aspects will help in designing effective surveys for training effectiveness measurement.
2) How to Design Effective Surveys
A common misconception is that a single survey is sufficient for evaluation. Many companies conduct surveys immediately after training, but a one-time survey is often insufficient to accurately assess training effectiveness.
To properly evaluate mindset changes and behavioral transformations, at least two surveys should be conducted—one immediately after training and another 3–6 months later.
The Five-Level Model of Training Evaluation by Jack J. Phillips
The Five-Level Model of Training Evaluation, developed by Jack J. Phillips, categorizes evaluation into different levels based on the timing and objectives of assessment.
Surveys are conducted at two points in this evaluation model: Level 1 (immediately after training) and Level 3 (three to six months later).
Why are these timings considered optimal?
The Level 1 survey aims to give learners an opportunity to reflect on how they can apply the training content to their workplace.
To help visualize the timing of Level 1 and Level 3 surveys, we have created a simple diagram.
The objectives of these surveys differ, requiring different question structures and considerations. The next sections will detail how to design and implement these surveys effectively.
3) How to Design a Level 1 Survey (Immediately After Training)
The first survey should be conducted immediately after the training.
The purpose of this survey is twofold:
to assess how much the learners have absorbed from the training, and
to help them form a concrete image of how they can apply it in the workplace.
herefore, the first survey should be designed to consider points such as:
- What changes in perception occurred before and after the training
- Whether they gained a clear image of how to apply what they learned on the job
- Whether they feel that using the acquired skills can improve their performance
In addition to standard questionnaire items,
it is recommended to include the following three key areas:
- Progress toward goals
- Degree of application and improvement
- Potential barriers to implementation
By conducting a survey immediately after the training that includes these three elements, learners are prompted to reflect once more on questions such as,
“Did I achieve the objectives?” or “What challenges did I face?”
This leads to valuable insights into how learners perceive the achievement of training objectives and the challenges they encountered.
Specific example questions related to these three elements are summarized in the diagram below.
Let’s take a closer look at each item:
①Progress Toward Objectives
Learners assess how well they achieved the training goals and explain their improvements.
This allows for quantifying achievement that is otherwise hard to measure.
②Application & Action Planning
This checks how well learners can imagine applying the training at work.
The more specific and numerous the ideas, the higher the chance of real action.
③Barriers & Support Needed
Learners identify obstacles to applying what they learned, including where support from others may be needed.
Understanding these helps smooth the path to implementation.
Show learners the survey questions before the training.
This makes them more aware of what’s important and improves the quality of feedback.
4) Tips and Considerations for Designing Level 3 Surveys
The second survey should be conducted 3 to 6 months after the training, once learners have had the opportunity to apply what they learned in the workplace.
This time gap is necessary to assess whether behavioral change has actually occurred.
Since the primary goal of this follow-up survey is to evaluate behavioral transformation, the key is to ask learners to describe specific episodes or examples from their actual work experience.
One important consideration is to have participants refer back to what they wrote in the Level 1 survey when answering the Level 3 survey. This helps connect initial intentions with later actions.
The Level 3 survey should include the following four components:
- Progress toward the action plan
- Episodes of success or failure
- Outcomes created based on the training
- Other insights or observations
Let’s take a closer look at each item and explore example questions for each.
①Achievement of the Action Plan
Learners are asked to quantify how much of their action plan, created immediately after the training, has been achieved.
They should also be encouraged to describe what specific actions they took.
By collecting data on action plan achievement over time, organizations can build a set of quantitative and qualitative insights to support training review and improvement.
②Episodes of Success and Failure
By gathering both successful and unsuccessful episodes from learners, you can identify valuable insights for improving future training.
Key points to include in the questions:
- Results achieved thanks to the training
- Specific success stories of actions taken
- Items included in the action plan but not implemented
- Things that were attempted but did not go well
Design questions to prompt detailed, reflective responses in these areas.
③Outcomes Based on Training
This category focuses on understanding what outcomes were realized as a result of the training.
By asking these questions after some time has passed, the following aspects can be evaluated:
- How well learners remember what they learned
- Whether they are consciously applying the training in daily work
- The degree of retention and adoption of the training content
Additionally, you can include questions about failures or risks that were avoided thanks to the training.
This allows for evaluation from a risk management perspective as well.
Designing Surveys for Measurable Impact
Surveys can be designed to measure both quantitative and qualitative training effectiveness.
Since the indicators used will vary depending on the company and training goals, it’s important to clarify the following before designing the survey:
- What are the expectations for this training?
- What indicators or outcomes should be measured?
With these clarified, you can move forward with a learning design that aligns with your overall goals.
5) Conclusion: Designing Purpose-Driven Surveys for Improved Training Evaluation
To optimize training evaluation, organizations should conduct surveys at two key points:
The most effective timing for conducting surveys is at Level 1 and Level 3 of the Five-Level Model of Training Evaluation. Administering two surveys for a single training session—one immediately after and one several months later—is key to ensuring successful survey outcomes.
The timing and key design points for each survey have been summarized in a comparison table.
| Level | Timing | Focus |
| Level 1 | Immediately after training | To organize from the perspective of practical application in the workplace, include the following items: ・Progress toward the training objective ・Visualization and planning for workplace application ・Identification of potential obstacles and areas for improvement in practice |
| Level 3 | 3–6 months later | To measure the level of on-the-job application using indicators such as “degree of improvement,” “frequency of application,” and “specific examples,” include the following items: ・Achievement level of the action plan ・Success and failure episodes ・Outcomes gained based on the training content |
Surveys are a powerful tool for gathering actionable feedback. By aligning survey objectives with training goals, companies can enhance both learning effectiveness and overall business performance.