Project Development – Evaluation of Project Goals
Evaluating the quality and effectiveness of your project and/or intervention is an important step in implementation. It is important to to know whether or not your educational goals have been met, and how your project might be improved to better meet the needs of your audience.
For this task, you will focus on developing a comprehensive evaluation plan and instrument (s) to assess the quality of your project. I have included some resources above to help with this task, but in the end every project is unique, and as the developer of your project, you are supremely qualified to determine not only the criteria for which the final product should be assessed, but the purpose of the evaluation and how and when it should take place. For your evaluation plan, please include the following in a Google Document:
- A list of questions that you would like the evaluation to answer. You may be interested in answering multiple questions; how well your curriculum aligns to current standards, the extent of differentiation in your activities, or the relationship between student outcomes and satisfaction for example. Try to focus on only one or two primary goals for your evaluation which would ideally be aligned with the goals of your project. Setting too broad an evaluation plan will make it more difficult to develop an appropriate methodology.
- A description of all evaluation components and methodology. For example, you may wish to conduct a pre/post test evaluation method to assess student learning before and after participation in your project. Or you may want to look for a relationship between student outcomes and course satisfaction using an end of course evaluation and student grades. The results don’t need to be shared but the instruments do.
- Link to or include the assessment instrument or instruments that you have developed for your evaluation.
- A clear description and articulation of the research and/or standards that support your evaluation method and criteria. For an example, the Quality Matters program provides an exhaustive literature base supporting the standards used in the development of their course evaluation rubric. Your support does not need to be as detailed as this example, but should provide at least a short description of the literature for the major criteria you have chosen for evaluation.
- In your Evaluation Plan google document, include a link to your project plan and a link to your project site. Here are some Sample Project Sites and Evaluation Plans.
Here is my Final Project Plan
Here is my Evaluation Component:
EDTECH 523 Project Evaluation Plan
March 15, 2015
The goal of my evaluation will be to determine whether or not the tutorials, resources, notes, and discussions provided on the Sammamish Mathematics website are sufficient for meeting both the stated learning objectives and the overall needs of the students. Do the content, organization, text, multimedia, and collaboration components of this site contribute to achieving the goal of the site?
The Sammamish Mathematics website is available to all my students at all times. Students may participate in the discussions or collaborative activities at their leisure. This course evaluation seeks to answer one question based upon two sources of data:
- How well does the Sammamish Mathematics site meet the course goals and objectives based on summative achievement data? Students will be asked to take a short quiz during and after each tutorial.
- How well did instruction meet the course goals and objectives based upon the relationship between student outcomes and participant satisfaction? Students and or parents will be asked to complete a course evaluation survey at designated intervals.
The results from the quizzes and surveys will be used to improve the sites appearance and future content.
Evaluation Components and Methodology
The goal is to receive honest feedback that reflects upon the tutorials, resources, notes, and discussions provided on the Sammamish Mathematics website are sufficient for meeting both the stated learning objectives and the overall needs of the students.
- The first tool used to evaluate this site will be to add a survey. The survey will be open to both students and parents. The first survey questions will include what features both students and parents would find beneficial.
- Secondly, students will be asked to answer questions imbedded in the tutorials, using EdPuzzle.
- Finally, an additional survey will provided that will ask participants to evaluate the components of the site and to indicate how they are using the site and how it can be improved.
Palloff & Pratt (2007) state, “When students are given the opportunity to provide course and instructor feedback anonymously in a face to face classroom, they sometime feel comfortable being painfully honest about their experiences in the class” (p. 218) Honest input from both students and parents are critical for me to be able to evaluate the effectiveness of my site and it’s content. The anonymity of surveys allows respondents to answer with more candid and valid answers. To get the most accurate data, I need respondents to be as open and honest as possible with their answers.Based on this I chose to conduct surveys.
Weebly Survey – This was super easy to use since it exists as a free option with Weebly; however it had limitations of only being able to have multiple choice or open-ended essay responses.
Google Forms Survey – After exploring multiple surveying options I decided to go with a Google Forms survey. This was the best option for creating a balance of sliding scale, multiple choice, and open-ended questions.
Palloff & Pratt (2007) state, “The goal is to use the course content to teach students to think – in other words, to help them develop good critical thinking ability and to apply what they are learning in other contexts” (p. 207). To help address student performance I decided to imbed questions within my tutorials using EdPuzzle.
Edpuzzle – EDpuzzle lets users create their own videos or modify existing videos and is ideal for project based learning (PBL) as well as flipping a classroom. Also, it allows educators to create videos for guided learning and allows them to assess students by adding questions.
Research Supporting Selection of Evaluation Method and Criteria
I decided to base my survey and assessment criteria using The Online Learning Consortium (OLC) Scorecard. The OLC Quality Scorecard process is for measuring and quantifying elements of quality within online education programs in higher education. Here are the features that I will be using/modifying for my self evaluation.
Moss, J., & Hendry, G. (2002). Use of electronic surveys in course evaluation. British Journal of Educational Technology, 33(5), 583-592. Retrieved from http://onlinelibrary.wiley.com.libproxy.boisestate.edu/doi/10.1111/1467-8535.00293/abstract
To be used successfully in course evaluation, online surveys should be infrequent, short, simply designed and free from password access, and de–identified results should be displayed to students on their completion of a survey.
Huang, W., Huang, W., Diefes‐Dux, H., & Imbrie, P. K. (2006). A preliminary validation of Attention, Relevance, Confidence and Satisfaction model‐based Instructional Material Motivational Survey in a computer‐based tutorial setting. British Journal of Educational Technology, 37(2), 243-259.
Using a tutorial as an instructional method is based on the behavioural aspect of learning (Driscoll, 2000). The main idea supporting this method is the interactions between stimuli and responses are applied for instructional purposes. By providing corrective feedback to learners, a tutorial is able to shape students’ behaviours (Newby et al). Alessi and Trollip (2001) suggested that tutorials can model the desired behaviours of learners and guide them through the initial use of newly acquired information or skills.
Alessi, S. M. & Trollip, S. R. (2001). Multimedia for learning: methods and development. Boston, MA: Allyn and Bacon.
Driscoll, M. P. (2000). Introduction to theories of learning and instruction (2nd ed.). In M.P.Driscoll (Ed.), Psychology of learning for instruction (pp. 3–28). Boston, MA: Allyn and Bacon.
Newby, T., Stepich, D., Lehman, J. & Russell, J. (2000). Instructional technology for teaching and learning (2nd ed.). Upper Saddle River, NJ: Prentice Hall.
Palloff, Rena & Pratt, Keith. (2007). Building online learning communities. San Francisco, CA:
John Wiley & Sons.