Categories
4.5 Professional Learning Facilitator EDTC6106 ISTE Standard for Coaches ISTE standards

Evaluating Professional Development

Understanding Practitioners' Perceptions in PD Evaluation | USC Center on Public DiplomacyEvaluating the effectiveness of professional development can be time consuming, costly, complex, and inconclusive. An analysis of teacher development came from a report from TNTP “The Mirage” (2015) by surveying 10,507 teachers and 566 school leaders. Their conclusion was that “in short, we bombard teachers with help, but most of it is not helpful – to teachers as professionals or to schools seeking better instruction.” The cost of professional development is significant. The school districts participating in this research had an average yearly cost per teacher of 18,000, and yet there was little evidence of the effectiveness of such professional development. The mirage reports that “only 3 in 10 teachers demonstrated substantial improvement” However, “Even when teachers do improve, we were unable to link their growth to any particular development strategy” (p. 2).

ISTE Standard, Performance Indicator 4.5c states: Evaluate the impact of professional learning and continually make improvements in order to meet the schoolwide vision for using technology for high-impact teaching and learning.

What are research-based strategies and elements that need to be included when evaluating professional learning programs?

Addressing the issues with professional development and the difficulty of their evaluation, The Mirage gives recommendations for designing effective professional development. While their research suggests that professional development needs to be highly individualized, they explain how schools and districts can create conditions for successful teacher development. Further recommendations are to redefine, reevaluate and reinvent professional development. For example:

  • Define “development” clearly, with observable and measurable goals
  • Evaluate the effectiveness of existing development activities
  • Reconstruct the teacher’s job

Another recommendation comes from the website from the Office of Educational Technology presents a professional learning toolkit for planning and evaluating professional learning. Future Ready Schools: Empowering Educators through Professional Learning toolkit presents a 5 step approach:

  • Determine district readiness
  • Assess current professional learning
  • Refine professional learning
  • Implement professional learning
  • Measure refined professional learning

Step #5 – Measure refined professional learning addresses ISTE standard 4.5c.

This site presents an evaluation tool that facilitates the collection, analysis and dissemination of evidence on the impact of professional learning activities have on educators and students. The tool is a 5-level evaluation framework including:

  • Immediate: activities and interactions: educators’ experience immediately after their participation
  • Potential: knowledge and social capital: knowledge and skill acquired by participants
  • Applied: Changes in practice: educators apply knowledge and skill in practice
  • Realized: Performance improvement: evidence is gathered from assessments, portfolios, videos, etc.
  • Reframing: Changes in what is valued: Educators share successes and the knowledge and skill move beyond their classroom

This toolkit addresses “new and innovative forms for online and blended professional learning.” A possible drawback to this toolkit is explained in the “about” section: “There is still debate among educational researchers about the features of professional learning that generate value, and, to date, there have been a limited number of large-scale experimental studies of professional development and, in particular, the new and innovative forms for online and blended professional learning on which this toolkit focuses. The results of some such studies that have been completed are inconclusive or contradictory.”

One needs to consider that innovation, although necessary for improvement and growth, is not always supported by research.

Yet another report that gives recommendations for professional development is Evaluating Professional Learning. A tool for schools and districts. This resource focuses on the following topics:

  • Importance of evaluating professional learning
  • Designing or selecting high-quality professional learning
  • Developing a logic model
  • Developing strong evaluation questions
  • Developing a data collection plan
  • Making meaning of your data

This toolkit draws from Guskey’s research and findings.

In the article Does It Make a Difference? Evaluating Professional Development (2002) Guskey explains that good evaluations “require thoughtful planning, the ability to ask good questions, and a basic understanding of how to find valid answers” (p. 46).  Thoughtful planning is essential. The goals need to be clearly defined at the beginning, following a backward design model. He recommends to define the student learning outcomes (level 5) first. Next, determine the evidence that can be gathered, such as lesson plans, student work and school records. Then, determine research-based instructional strategies that would produce the desired results. This is a good place to include digital tools that would support and transform the instructional strategies.

My conclusion is that the initial planning and design is essential for a successful evaluation of professional development. Following the backward design model when designing professional development provides concrete elements for evaluating its effectiveness.

Backward design Model

References

Breslow, N. & Bock, G. (2020). Evaluating Professional Learning. A tool for schools and districts. REL Northeast & Islands.  https://ies.ed.gov/ncee/edlabs/regions/northeast/pdf/NE_5.3.5_Evaluation_PD_Brief_12-22-20_accessible.pdf

Guskey, T. R. (2002). Does It Make a Difference? Evaluating Professional Development. Educational, School, and Counseling Psychology Faculty Publications 7. http://uknowledge.uky.edu/edp_facpub/7

ISTE Standards: Coaches. https://www.iste.org/standards/iste-standards-for-coaches 

Office of Educational Technology. Future Ready Schools: Empowering Educators through Professional Learning toolkit.  https://tech.ed.gov/futureready/professional-learning/

TNTP. (2015). The mirage: Confronting the hard truth about our quest for teacher development. Brooklyn, NY: Author. https://tntp.org/assets/documents/TNTP-Mirage_2015.pdf

Treacy, B, Lleiman, G., & Peterson K. (2002). Successful Online Professional Development. Learning and Leading with technology 30(1). http://courses.edtechleaders.org/about/research/articles/SuccessfulOPD.pdf

3 replies on “Evaluating Professional Development”

I find your post is very useful. Indeed, this issue is an issue that must be addressed both for teachers and educational institutions. Neither schools nor teachers should devote a lot of resources and time to professional development that cannot ultimately help teachers improve their teaching quality. There is no benefit to both students and teachers. At the same time, we strongly agree that any form of professional training must be able to meet the daily teaching needs of teachers. enjoy reading you post.

Another great post! I find the toolkit draws from Guskey’s research and findings is extremely helpful to evaluate professional development effectiveness. I can’t agree more than “thoughtful planning is essential” and it requires time and clear goal. Another great point is to connect back to backward design model from our pervious discussion. Thank you and always enjoy learning from your post!

Leave a Reply

Your email address will not be published.

css.php