{"id":5088,"date":"2022-12-01T09:00:00","date_gmt":"2022-12-01T00:00:00","guid":{"rendered":"https:\/\/learningshift.co.jp\/?p=5088"},"modified":"2025-06-09T10:02:39","modified_gmt":"2025-06-09T01:02:39","slug":"measurement","status":"publish","type":"post","link":"https:\/\/learningshift.co.jp\/en\/library\/measurement","title":{"rendered":"Effective Training Impact Measurement: Top-Down and Bottom-Up Approaches"},"content":{"rendered":"\n

1) The Importance of Measuring Training Effectiveness<\/h2>\n\n\n\n

Corporate learning must contribute to business performance. However, given time and budget constraints, achieving a return on investment in training is becoming increasingly difficult.<\/p>\n\n\n\n

The term “reskilling” has become more common, emphasizing the need to quickly acquire knowledge and skills required for evolving job roles.<\/p>\n\n\n\n

As corporate learning evolves, the importance of measuring training effectiveness has increased. This necessitates a review of traditional evaluation methods. Relying solely on post-training surveys is no longer sufficient; human resource development professionals must now rethink how they assess training outcomes.<\/p>\n\n\n\n

1)-1\u3000Defining Outcomes in Terms of Productivity<\/h3>\n\n\n\n

An effective training impact measurement strategy begins with a clear definition of “outcomes.” One widely accepted perspective is to view outcomes in terms of productivity.<\/p>\n\n\n\n

If employees move from merely “understanding” to “being able to do,” overall organizational productivity will increase. This transformation validates the training’s contribution to a company\u2019s competitive advantage.<\/p>\n\n\n\n

Productivity can be quantified by dividing “business performance improvements achieved through learning” by “resources invested, such as time, cost, and personnel.”<\/p>\n\n\n\n

\"\"<\/figure><\/div>\n\n\n\n

Training impact measurement should focus on maximizing the “business performance” component rather than merely reducing costs associated with training logistics.<\/p>\n\n\n\n

There are two primary approaches to evaluating training impact: top-down and bottom-up. The next sections will explain these approaches in detail.<\/p>\n\n\n\n

2) The Top-Down Approach to Training Design and Impact Measurement<\/h2>\n\n\n\n

To assess “business performance improvements achieved through learning,” a structured approach based on outcome definitions is required.<\/p>\n\n\n\n

Learning does not directly translate into business results; there are multiple stages in the process. One widely used evaluation model that illustrates this progression is Kirkpatrick\u2019s Four-Level Training Evaluation Model.<\/p>\n\n\n\n

1-1 Kirkpatrick\u2019s Four-Level Training Evaluation Model<\/h3>\n\n\n\n

Developed in 1959 by Donald Kirkpatrick, this model categorizes evaluation into four levels:<\/p>\n\n\n\n

\"\"<\/figure><\/div>\n\n\n\n

The four-level evaluation model begins at Level 1, and as the levels progress upward, they move closer to the ultimate goal.<\/p>\n\n\n\n

According to this model, achieving results (Level 4) requires a foundation built on behavior (\u201ccan do\u201d), knowledge (\u201cknow\u201d), and mindset (\u201csense of ownership\u201d or \u201cclarity of purpose\u201d).<\/p>\n\n\n\n

In other words, to design an effective training program, it is essential to clearly define the intended business outcomes at the goal level, and then work backward to identify the necessary behavior changes that will lead to those outcomes.<\/p>\n\n\n\n

When actually designing a learning program, you start by defining Level 4, and then work through the following steps:
\u2460What behaviors are required to achieve Level 4 outcomes? \u2192 Level 3
\u2461What knowledge and skills are needed to enable those behaviors? \u2192 Level 2
\u2462What mindset is necessary to acquire those knowledge and skills? \u2192 Level 1<\/strong>
In this way, we examine the model from top to bottom in the design process.<\/p>\n\n\n\n

1-2 Example of Effectiveness Evaluation Based on the Four-Level Model<\/h3>\n\n\n\n

The characteristics of evaluation at each level can be summarized as follows:<\/p>\n\n\n\n

  • Level 1: Reaction Evaluation (Mindset)\uff1aThe quality of the training is assessed based on participant surveys and the content written in action plan sheets submitted after the training.<\/li><\/ul>\n\n\n\n
    • Level 2: Understanding\uff1aKnowledge acquisition is measured through tests. In addition, learners are asked to write or present how they envision applying what they\u2019ve learned.<\/li><\/ul>\n\n\n\n
      • Level 3: Behavior Change\uff1aEvaluation focuses on changes in actual workplace behavior, using tools such as post-implementation reflection and interviews with supervisors or senior colleagues.<\/li><\/ul>\n\n\n\n

        In the case of soft skills (e.g., communication or sales negotiation skills), role-playing is conducted to assess whether the skills are being effectively demonstrated.<\/p>\n\n\n\n

        • Level 4: Results\uff1aThe outcomes of learning are measured using both quantitative and qualitative data, assessing impact on the individual and the workplace.<\/li><\/ul>\n\n\n\n

          \u305f\u3060\u3057\uff14\u6bb5\u968e\u8a55\u4fa1\u3082\u9032\u5316\u3057\u3001\u73fe\u5728\u3067\u306f\u3053\u308c\u304b\u3089\u7d39\u4ecb\u3059\u308b\uff16\u6bb5\u968e\u8a55\u4fa1\u3067\u6574\u7406\u3055\u308c\u308b\u50be\u5411\u306b\u3042\u308a\u307e\u3059\u3002<\/p>\n\n\n\n

          2-1\u3000The Six-Level Evaluation Model and a New Perspective on Measuring Effectiveness<\/h3>\n\n\n\n

          While the four-level evaluation model helps assess whether training objectives were achieved, many researchers argue that it does not sufficiently measure whether the training actually improved employee performance.<\/p>\n\n\n\n

          This is where the six-level evaluation model comes in.<\/p>\n\n\n\n

          This model is considered an enhanced version of the four-level model, and breaks down the evaluation process into the following six levels:<\/p>\n\n\n\n

          1. Level 1: Reaction Evaluation<\/li>
          2. Level 2: Understanding (Test-Based Assessment)<\/li>
          3. Level 3: Behavior Change<\/li>
          4. Level 4: Business Results<\/li>
          5. Level 5: ROI \u2013 To what extent the training contributed to the return on investment at both organizational and individual levels<\/li>
          6. Level 6: Contribution \u2013 To what extent the training expanded the participant\u2019s long-term potential and capability over time<\/li><\/ol>\n\n\n\n

            The addition of two extra levels reflects the growing belief that training must contribute to competitive advantage.<\/p>\n\n\n\n

            By incorporating ROI and Contribution into the evaluation model, we can more accurately assess whether the training fulfilled its ultimate purpose\u2014contributing to business profitability through learning.<\/p>\n\n\n\n

            2-2\u3000Measuring Effectiveness from the Individual\u2019s Perspective<\/h3>\n\n\n\n

            In addition to Levels 5 and 6, a new perspective has emerged:
            Effectiveness should now be evaluated from both the individual perspective and the team\/organizational perspective.<\/p>\n\n\n\n

            \"\"<\/figure><\/div>\n\n\n\n

            Let\u2019s consider the six-level evaluation model from two perspectives: the \u201cindividual perspective\u201d and the \u201cteam\/organizational perspective.\u201d
            From the individual perspective, the focus of measurement is on the learner\u2019s attitude and reaction.
            In contrast, from the team or organizational perspective, the focus shifts to different evaluation points, such as:<\/p>\n\n\n\n

            • Was the training content something participants would want to recommend to others? (Reaction)<\/li>
            • Were the learners able to acquire knowledge autonomously and efficiently? (Learning)<\/li>
            • To what extent did the training help evolve the business, generate profits, and drive innovation? (ROI)<\/li><\/ul>\n\n\n\n

              This shift in thinking has led some companies to begin incorporating future- and team-based perspectives into their training evaluation practices.
              For example, during Level 1 evaluations, some organizations ask questions like:
              \u201cWould you recommend this training to others?\u201d
              to gauge whether participants were motivated to share or promote the experience.<\/p>\n\n\n\n

              Going forward, effective training evaluation must go beyond just the \u201cindividual\u201d and the \u201cpresent.\u201d
              We will need to focus more on how much potential can be developed in the \u201cteam\/organization\u201d and in the \u201cfuture\u201d\u2014looking at medium- to long-term impact.<\/p>\n\n\n\n

              3 Steps for Learning Design with Evaluation in Mind<\/h3>\n\n\n\n
              \"\"<\/figure><\/div>\n\n\n\n

              Steps for Learning Design with Evaluation in Mind<\/p>\n\n\n\n

              The learning design process follows these steps:<\/p>\n\n\n\n

              STEP1: Define the Business Impact
              Here, \u201cbusiness impact\u201d refers to the ultimate goal.
              By clearly defining the goal, the foundation for the learning design is established.<\/p>\n\n\n\n

              STEP2: Develop a Strategy to Achieve the Business Goal
              Create a strategy or plan to realize the business impact, and set measurable indicators (KPIs) to track progress.<\/p>\n\n\n\n

              STEP3: Identify the \u201cMindset\u201d and \u201cBehavior\u201d (Capability Development Areas) Linked to the KPIs
              Translate the required actions for achieving the KPIs into specific behaviors. This step helps build a clear pathway toward the goal.<\/p>\n\n\n\n

              STEP 4: Design a Learning Program Aimed at Cultivating the Desired Mindset and Driving Behavioral Change
              Work backward from the goal to design a course that intentionally fosters mindset shifts and behavioral transformation.<\/p>\n\n\n\n

              To help illustrate this process, we will use the design of a sales training program as an example and explain how to define behavioral change.<\/p>\n\n\n\n

              By aligning the evaluation model with changes in mindset and behavior, and by visualizing the path toward achieving the business impact (goal), the overall flow can be summarized in the diagram below.<\/p>\n\n\n\n

              \"\"<\/figure><\/div>\n\n\n\n

              And when designing a training program, you must work backward from the goal.<\/p>\n\n\n\n

              \u3010D\u3011 Sales \u2014 Define the business impact
              \u3010C\u3011 Changes in the number of proposals or closing rate that lead to increased sales \u2014 Develop strategies to achieve the goal and set KPIs
              \u3010B\u3011 Aim to increase large new deals by enabling proposals that match customer needs \u2014 Clarify the mindset and behaviors that drive KPI improvement
              \u3010A\u3011Training programs designed to foster the mindset and behavioral change needed to strengthen proposal skills \u2014 Design learning that cultivates the intended mindset and behavior<\/p>\n\n\n\n

              Only by designing training in this way can effective evaluation be achieved.<\/p>\n\n\n\n

              3) Bottom-Up Thinking and Training Evaluation Methods<\/h2>\n\n\n\n

              The top-down approach involves planning training content in advance and then validating its effectiveness against predefined goals.<\/p>\n\n\n\n

              However, in practice, the actual benefits and outcomes that emerge from training almost always include unintended or unexpected results.<\/p>\n\n\n\n

              This is where the bottom-up approach becomes important\u2014it involves gathering feedback and insights from the field.
              By incorporating bottom-up thinking into evaluation, we can measure the unintended but valuable effects that top-down models often overlook.<\/p>\n\n\n\n

              For example, when learning about communication strategies for sales, the same principles may unexpectedly prove helpful in conversations with supervisors or in coaching subordinates.<\/p>\n\n\n\n

              Likewise, as participants develop stronger proposal skills and begin actively making suggestions to clients, these interactions may evolve into new opportunities\u2014such as forming new business partnerships\u2014beyond the original intent of the training.<\/p>\n\n\n\n

              Such creative outcomes often emerge from the bottom up rather than from structured planning.<\/p>\n\n\n\n

              Experiences like \u201cThis part of the training helped me in a real situation\u201d are difficult to capture through top-down evaluation methods.
              In contrast, bottom-up approaches\u2014using surveys and interviews\u2014make it possible to collect qualitative, experience-based success stories that can\u2019t be quantified.<\/p>\n\n\n\n

              \u201cThis specific part of the training was especially helpful.\u201d
              \u201cI shared what I learned in the training with a colleague, and that person achieved great results.\u201d<\/p>\n\n\n\n

              Collecting and accumulating these real examples transforms learning outcomes into valuable organizational assets.<\/p>\n\n\n\n

              Not only is it important to collect useful information through a bottom-up approach,
              but it is equally essential to create a system for sharing that knowledge\u2014this increases the overall impact of learning across the organization.<\/p>\n\n\n\n

              4) Conclusion<\/h2>\n","protected":false},"excerpt":{"rendered":"

              1) The Importance of Measuring Training Effectiveness Corporate learning must contribute to business performance. However, given time and budget constraints, achieving a return on investment in training is becoming increasingly difficult. The term “reskilling” has become more common, emphasizing the need to quickly acquire knowledge and skills required for evolving job roles. As corporate learning […]<\/p>\n","protected":false},"author":3,"featured_media":2061,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_locale":"en_US","_original_post":"https:\/\/learningshift.co.jp\/?p=2060","footnotes":""},"categories":[31,1],"tags":[42],"class_list":["post-5088","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-column","category-library","tag-evaluation","en-US"],"acf":[],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/learningshift.co.jp\/wp-json\/wp\/v2\/posts\/5088","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/learningshift.co.jp\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/learningshift.co.jp\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/learningshift.co.jp\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/learningshift.co.jp\/wp-json\/wp\/v2\/comments?post=5088"}],"version-history":[{"count":5,"href":"https:\/\/learningshift.co.jp\/wp-json\/wp\/v2\/posts\/5088\/revisions"}],"predecessor-version":[{"id":5138,"href":"https:\/\/learningshift.co.jp\/wp-json\/wp\/v2\/posts\/5088\/revisions\/5138"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/learningshift.co.jp\/wp-json\/wp\/v2\/media\/2061"}],"wp:attachment":[{"href":"https:\/\/learningshift.co.jp\/wp-json\/wp\/v2\/media?parent=5088"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/learningshift.co.jp\/wp-json\/wp\/v2\/categories?post=5088"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/learningshift.co.jp\/wp-json\/wp\/v2\/tags?post=5088"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}