Last week, as a part of our Learn with Teaching Lab series, we hosted a discussion focused on Evaluating the Success of Professional Learning. Specifically, we discussed how Teaching Lab evaluates the impact of professional learning in school systems, what the data says about how teaching and learning are changing in school systems we partner with, and how COVID-19 and school closures may shift our thinking around evaluating professional learning.
Why Evaluate Professional Learning? (Do We Need To Do It… Is It Hard?)
Every school system creates and provides professional learning of some kind to their educators. Sometimes it is done inhouse, sometimes a provider, like Teaching Lab, is hired. However, it is rare that the school system evaluates whether that professional learning is actually effective. Often, if evaluation is done, teachers are asked whether they thought it was useful or relevant. Those are good starting questions, and we use those questions, too, but that’s just the start.
We recognize that the word “evaluation” is scary to many. Evaluating professional learning may sometimes be an afterthought, even if you recognize that it’s important. And, many school and district leaders may feel they or their team lack the skill and expertise to become involved in rigorous evaluations. As a result, we’ve seen that school and district leaders either leave them to “evaluation experts” or neglect evaluation completely.
The good news is: good evaluations don’t have to be complicated. They simply require thoughtful planning, the ability to ask good questions, and a basic understanding of how to find valid answers. A strong evaluation plan can provide meaningful information that you can use to make thoughtful, responsible decisions about professional learning.
So, yes, if there is professional learning in a school system, it should be evaluated… and evaluation does not have to be hard.
Teaching Lab’s Theory of Change for Professional Learning
Evaluation does not start with the survey questions you want to ask; it starts with a solid theory of change. At Teaching Lab, our theory of change for our professional learning services is fairly simple:
We start by getting clear about the very last step (the “then”), or the changes we would expect to see if we were successful in our efforts. Like backward planning as a teacher, ask first, what do you want students to learn at the end of a lesson or unit? Then, we plan out the content of the professional learning we’d have to create and/or deliver in order to be successful and achieve those changes.
Our Approach to Evaluation
Supporting the point that evaluation does not have to be hard, there is a great and widely used framework for evaluating professional learning or professional development, which was developed by Thomas Guskey. The Guskey framework has been so widely used because it is intuitive to educators and leaders. School systems themselves can take this framework and do their own evaluation. At Teaching Lab, we use the Guskey framework as a starting point and adapted it slightly to align with our theory of change. The framework describes 5 levels of evaluation: (1) participant reactions, (2) participant knowledge, (3) organizational change, (4) participant practices, and (5) finally student learning.
Essentially, the framework offers that if participants are satisfied with their professional learning experience, and if they gained knowledge from that experience, and if the school system supports them to put into their actions their learning, and if the participants actually change their practice because of what they learned, then student learning experiences will also change.
In our theory of change, we state our belief that good outcomes from our work mean educator mindsets, skills, and practices will improve. So in our evaluation, we use:
Teaching Lab’s Data Collection Efforts
In summary, when it comes to evaluating professional learning:
Teaching Lab’s Impact: What We’ve Learned
By enacting our evaluation plan, Teaching Lab has learned a lot about how our 4,000 participants perceive our professional learning, as well as what impact our work has had on them and their 360,000 teachers across more than 30 school systems in 12 states.
More data and impact stories can be found in our Impact Report.
Did you miss our discussion? Watch the recording here.