EdTech 505: Chapter Discussion

Kim Hefty

EdTech 505

Week 7 Assignment:

  1. Discuss chapter 5 and chapters 2 & 6

Chapter 5, Choosing an Evaluation Model, provided a lot of clarity, “…evaluation differ(s) from research strategies in that evaluation results are provided to the appropriate stakeholders for the purpose of program or project improvement … the purpose of research, in contrast, is causal links …”. This is why we are not supposed to have a control group! Until now, this has been a huge struggle conceptually for me. Chapter 5 presents multiple evaluation models: adversary, art criticism, decision making, discrepancy, goal based, goal free, systems analysis and transactional. The chapter also clearly defines the evaluation design format and their purposes.

Chapter 2 of the User-Friendly Handbook for Mixed Method Evaluation explores preservice training for elementary math teachers and future teachers. A ten day workshop is developed to focus on the content and pedagogy in line with state standards. The project has four main goals 1) further knowledge 2) enable and encourage this new knowledge is used in the classroom 3) stimulate student interest and 4) test a model for achieving goals.   Chapter 2 also addresses the formative and summative evaluations and provides sample questions.  The ABCs of Evaluation also presents an application of the evaluation process, but focuses more on the different types of evaluations and the components of evaluation. Both chapters are illustrating the effort to make when developing a project to determine what parts of a project are effective and what parts can be improved on.

Chapter 6 of the User-Friendly Handbook for Mixed Method Evaluation provided a significant amount of detail for the preservice project. It clearly outlined steps. Step 1: Develop Evaluation Questions. Step 2: Determine Appropriate Data Sources. Step 3: Reality Testing and Design Modifications. The tables were separated into project goals, evaluation questions and stakeholders for both the summative and formative evaluations. The tables were very clear and presented great samples of questions. I found chapter 6 to be the most detailed of all the readings and the most helpful.

All of the readings emphasized the importance of gathering information about prior to asking questions. An evaluator must have a good understanding about the existing project before trying to recommend changes. All of the readings recommend including measurable objectives of what will be accomplished once the plan is implemented. Other important components that must be included in an evaluation that were included in the book and the internet readings were data sources such as interviews, observations and focus groups, data collection design, responsibility and audience.

I gained a lot of insight into my evaluation project after the readings. The biggest impact was the subtle, yet very important, distinctions between research and evaluation. The specific questions outlined in the Internet reading also helped me to generate better questions for my project.

Boulmetis, J., & Dutwin, P. (2011). The ABCs of evaluation (3rd ed.). San Francisco, CA: Jossey-Bass.

  1. Application of Reading Material to Project:

As I began reading chapter 5 immediately thought that I would use the decision-making model for my evaluation report. The purpose for my project is to decide if I should use study guides with my other online math students, this is clearly a “long-range” goal affecting the future. But as I continued to read the model was not so clear cut. Goal-based method also made a lot of sense since I will be measuring data using both quantative and qualitative methods. This method is the most research like; an aspect of my project I have been struggling with. I have really wanted to do a compare and contrast, using a control group. But after the reading I now realize that this is research … not evaluation.

I thought my mind was made up until I read table 5.1. Goal-free caught my attention. The intended outcome is to explore the usefulness and impact. The tasks are right in line with a goal-based approach which utilizes data collection, analysis and interpretation (just like goal-based as well). This method asks “what happened in the program?” So I had to go back to the reading. This just won’t fit since I am entering this with a goal but I was intrigued by the concept. Finally, the systems analysis model also seemed like a good fit after looking at the table. Maybe it’s the math teacher in me but I really like a systematic approach.

All of these methods warranted exploration and I felt that I could soundly support using any of them. But as I reread the chapter I was struck by the quote from Worthen that many evaluators perform evaluations using multiple models either consciously or unconsciously. These models provide sources for new ideas and techniques. (p. 109)

I have decided to implement the goal-based method but I expect to be heavily influenced by several of the other methods.

Boulmetis, J., & Dutwin, P. (2011). The ABCs of evaluation (3rd ed.). San Francisco, CA: Jossey-Bass.

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s