Week 10 Assignment:
- Read chapter 8 – “Is It Evaluation or Is It Research”” Do the two-part exercise on p. 184. Minimum 75 words in response to each question.
Does my project represent research or evaluation? How do I make that decision?
I have significantly struggled with this since the beginning. My instincts are to do research. I keep inadvertently creating control groups. I just prefer to do comparison. But the comparative aspect is what makes it research and not evaluation. To determine what I am actually doing, it is important to refer back to the original intentions for the evaluation design. In the text, Boulmetis and Dutwin (2005) state, “Your balance point in the evaluation-research continuum depends on your target audience for the results, and their purpose.” (p. 171).
My project is to evaluate the effectiveness of study guides for students taking an online math course. This project could easily be done as a research project by having a control group; however, to assure that it is evaluation and not research, all participants will be given the study guide. I must make sure that I do not attempt to control the conditions or manipulate the variables. The guides themselves will be evaluated and the data collected will involve amount of time spent on the guides and whether or not grades are improved. The data collected will be used for decision-making purposes. The results will be reported to all stakeholders: the students, the teacher, and the district.
Does my project involve sampling? If so, what type of sampling will I employ?
This project will involve sampling. I will be collecting data from all algebra 2 students taking an online math course. This sample group represents a part of the larger population, the online community of math students taking pre-calculus, calculus, algebra 1 and geometry.
I will be using a stratified random sample which is looking at a specific group within a population. The specific group is the algebra 2 students.
I could also employ a judgment sample. The online algebra 2 students will be used because they are immediately available and I see them regularly and they have a high likelihood of participating.
In addition, a purposive sample could be used. Since I am significantly restricting the size of my population to only the students in the local school who are taking the online algebra 2 course and not any of the other online courses.
Boulmetis, J. & P. Dutwin. (2005). The ABCs of evaluation: Timeless techniques for
program and project managers. San Francisco, CA: John Wiley & Sons, Inc..
- Summarize the contents of A Practical Guide to Sampling in at least 75 words. (PDF document) – http://www.nao.org.uk/publications/Samplingguide.pdf
A Practical Guide to Sampling provides an overview of sample design and methods along with tips on interpreting and reporting results. The guide is a supplement to hands-on support by the NAO Statistical and Technical Team. The information is presented in an attractive, colorful format and provides practical examples of sampling techniques. The guide starts with “Why Sample,” and explains the rationale for sampling. The next section “Sample Design” explains how to structure and interpret sample design. The information needed requires precision and should avoid unintended bias. The goal of design is to “achieve a balance between the required precision and the available resources”. The guide presents ideas for defining the population but provides cautions relating to privacy of data and contracting of outside evaluators. Sample size is another important aspect of sampling. The size depends on five key factors: margin of error, amount of variability, confidence level, and population size and population proportion. Within the guide a data table illustrates the quantity needed in a sample to achieve various levels of precision, with a 95% confidence level. The guide also provides multiple examples of weighting and post-weighting samples to demonstrate ways that data may be made more meaningful. The guide does a good job of explaining the definitions, uses and limitations of a variety of sampling methods. A flow chart illustrates how to choose the best sampling method for a designer. In order to extract a sample, the guide provides instructions for use of Excel and SPSS and how to analyze results. In addition, relevant formulas and tips on are provided for reporting results in the Appendix.
Summarize the contents of in at least 75 words. (YouTube video) – http://www.youtube.com/watch?v=jUBtNBVy42M&feature=related
The Simple Random Sample video focuses on defining and identifying the properties of a simple random sample (SRS). Each individual and sets of individuals has an equally likely chance of being selected. You could randomly select from a small group by pulling names out of a hat, but the names must be shuffled and not put back into the hat. SRS has many advantages such as being unbiased and easy to implement. But an SRS has disadvantages such as a lot of variability and does not always represent the entire population. Also the list of the population may not be adequate. To produce a good SRS requires good planning with details and including methodology. Another SRS method is the random digit table which reduces the amount of human error. The video illustrates how to apply the random digit table to a small group, such as a classroom of students. The final method illustrated is the random number generator found on any graphing calculator or computer program. Not a lot of detail is given for the final method. The video is short and easy to follow.