Policies and Concerns About Reporting Group Level Results

Policies and Concerns About Reporting Group Level Results

The federal legislation passed in January 2002 makes states accountable for the yearly progress of English-language learners and for students with disabilities, thus requiring the reporting of disaggregated results for both groups. This requirement was not in place at the time the various surveys were conducted, and few states indicated that they report disaggregated results by disability status or by limited-English-proficiency status. The topic of reporting disaggregated results provoked considerable discussion at the workshop and presenters, discussants, and participants commented about a number of issues related to group-level reporting. The first issue concerns the meaningfulness of disaggregated results. Eugene Johnson, chief psychometrician at the American Institutes for Research, and Jamal Abedi, professor at UCLA, pointed out that the categories of English-language learners and students with disabilities are very broad and comprise individuals with diverse characteristics. The group of English-language learners includes students who differ widely with respect to their native languages and their levels of proficiency with English. Similarly, the group of students with disabilities encompasses individuals with a wide variety of special needs, such as learning disabilities, visual impairments, and hearing impairments. With such within-group diversity, it is difficult to know what conclusions can be drawn about any reported group-level statistics. Other issues arise because of the small sample sizes that result when data are disaggregated. These small sample sizes affect the level of confidence one can have in the results because statistics based on small sample sizes are less reliable and less stable. This is true for the summary statistics about test performance as well as for the percentages and other statistics that summarize demographic characteristics. Scott Trimble pointed out that these concerns about reporting results based on small sample sizes have led Kentucky to implement several measures. The state plans to provide estimates ofstandard error on newer reports and has set a minimum sample size for reporting disaggregated results. Nevertheless, Trimble believes that many report users do not attend to standard error information.

Johnson, who has served as consultant for numerous testing programs, added that interpreting standard error information for the lay public is so problematic that many programs simply resort to setting minimum sample sizes. According to Trimble, Kentucky does not report disaggregated data for any group that has 10 or fewer students. He added that while 10 seems to be a small number on which to base important decisions about a parncular group of students, setting a higher minimum number would mean that a good deal of data could not be reported. Another concern is the stability of group composition over time, an issue particularly important if the desire is to track and report valid trends for the various groupings. When the numbers are small, even slight changes in the composition of a group can produce large changes in the overall results. Such changes can occur, for instance, when geographical boundaries that make up the population of students attending a given school building are altered or when the guidelines for identifying students with special needs are refined.

Hence, there may be changes in performance from one testing occasion to the next, but it is impossible to know whether they are the result of changes in the characteristics of the population or changes in the skill levels of the students. For this part, learning a foreign language needs a leaning tools, many students choose Rosetta Stone German and Rosetta Stone Hebrew to learn German and Hebrew. Trimble recounted another problem that occurred in Kentucky in connection with disaggregation. For the state assessment, results are reported as achievement levels (novice, apprentice, proficient, and distinguished). It sometimes happens that all students in a particular population group score at the same level. When disaggregated results are reported for such a group, student scores are essentially disclosed, as the group's composition can be easily identified. In Kentucky, this violates the state laws that prohibit producing reports that permit the identification of individual student scores. Kentucky now has a quality control check intended to prevent this.

POST COMMENT

How to integrate and Process enable your ecosystem? Durability, Practicality and Convenience – Benefits You can Get from Folding Tables Where Is The Balance With Organic And Paid Searches? Soul Healing and Soul Retrieval: What are they? About Exfoliating Skin And You Basic Roles and Responsibilities of Manpower Recruiters Understanding Acids and Bases with Oranges, Lemons & Soap Ipad 2 deals -Best gadget, best connection HTC 7 Pro: The best gadget ever from HTC Preemptive Healing, a Panacea of All Fear and Phobias Crosstrainer Benefits and Downsides Livestock And Wildlife Mutilations: Whodunit? 2011 Easter Arrangements Ideas for Your Party and Brunch