Review of the Literature: Robert L. Armstrong

Part of the NCA Commission on Accreditation and School Improvement Journal of School Improvement, Volume 3, Issue 1, Spring 2002

Review of the Literature

Robert L. Armstrong

About the Author: Dr. Robert L. Armstrong is professor emeritus of Arizona State University and research consultant for NCA Commission on Accreditation and School Improvement.

Editor's Note: The first two reviews are on the subject of Math/Numeracy, which is the theme for our fall 2002 issue.

Geist, E. (2001). Children are born mathematicians: Promoting the construction of early mathematical concepts in children under five. Young Children, 56(4), 12-19.

As is obvious from the title, Geist (Ohio University) posits that very young children are natural mathematicians with quite possibly an innate ability to comprehend mathematical concepts. However, more often than not, adults confuse such acquired skills as counting whole numbers with authentic conceptual understanding. Constructs such as relative quantity, categorization, and order are concepts that can be brought out in very young children by means of questioning strategies, task activities, and simple games. It is important that the teacher recognizes these traits and also understands that these concepts cannot really be taught in the conventional manner. They can be constructed by the children within themselves if the teacher provides the freedom and the varied experiences and opportunities that will lead the youngsters to recognize these concepts. The teacher needs varied materials and needs to provide opportunities that will allow the young people to come to their own conclusions, usually through trial and error, often by way of dead-end efforts and restarts. If in the long view we are to substantially improve the mathematics competence of our graduating high school students, we need to look at the early childhood level for the foundational learning that will enable these young folk to be quantitatively and logically literate in later years.

Perry, M. (2000). Explanations of mathematical concepts in Japanese, Chinese, and US first and fifth-grade classrooms. Cognition and Instruction, 18(2), 181-207.

In this study involving 40 each Chinese and Japanese, and 80 US first and fifth grade classrooms, investigator Michelle Perry sought to determine the reasons for the usually superior performance of Chinese and Japanese students in the area of mathematics. Classrooms were observed in Taipei, Taiwan; Sendai, Japan; and Cook County, Illinois (including Chicago). In each of the Asian cities 10 schools participated, and in the US 20 schools participated. Observers in each class wrote down what they saw and heard happening during the entirety of each observed class.

The results were complex and not always obvious but there were noteworthy differences. For example, Japanese and Chinese children generally heard more explanations than their US peers. Also, in first-grade the Japanese children were learning more in the way of complex topics than either their Taiwanese or US peers. The quality of explanations appeared to differ across the three sites although comparative quality was less clear. Although the results were not as definitive as might be hoped (no doubt some differences in the academic performance between these Asian and US students is a matter of culture), it does seem likely that US teachers should try to engage in more mathematical dialogue with their students, so that explanations, both direct and alternative, become accepted as a legitimate classroom mode. Students not only need to know how things work in mathematics, they need to know why they work. Understanding the "why" of things gives students a better basis of theoretical understanding of any particular technique.

The following cluster of three abstracts is taken from the special section on school reform in the 83(1) issue of Phi Delta Kappan. In each case, author, title, and page numbers are noted.

Elmore, R. & Fuhrman, S. Holding schools accountable: Is it working? pp. 67-72.

It has been the practice in NCA to allow its member schools a maximum amount of freedom in conducting the business of school improvement: selecting goals, assessing results, and developing interventions. In this sense our schools are not held accountable by NCA, only to varying degrees by their communities and their constituents. Our schools are held accountable for meeting the criteria of the standard, of course, in that accreditation may be withheld. However, authentic, constraining accountability generally involves both positive and negative consequences, publicity, monetary rewards or withheld support, official recognition (either positive or negative), and even the threat of reconstitution.

The Elmore and Fuhrman worked through the Consortium for Policy Research in Education (CPRE) to examine 20 public, charter, and independent schools, then followed up in three high schools in each of four states with differing accountability approaches. Seven operational principles emerged from the study:

Accountability systems do attract the attention of teachers and administrators. This had both positive and negative aspects, including pressure and stress.
Teachers and schools vary in their responses to accountability systems. Accountability may have the effect of both broadening curricular programs, if a demand is made for something the school did not offer, and narrowing the program, if the accountability focus is narrow and the stakes are substantial.
Internal accountability precedes external accountability.
Accountability for performance requires changes in the school's internal capacities for instruction. The school must do different things, not simply try harder at doing the old things.
Capacity matters, but not much is being done about it.
Stakes matter, but we need to know more about how they matter.
The expectations underlying performance-based accountability systems are often unclear to all concerned parties.

Cohen, D., & Loewenberg Ball, D. Making change: Instruction and its improvement, pp. 73-77.

The point of this very pungent article is that classroom instruction is a complex interaction of teacher to student to curriculum with further influence from parents, community, and post-school expectations. It involves the interaction of all these entities, as well as involving the environment, the coordination of content, resource use, and the management of incentives. Change in instructional practice requires substantial inservice for the teachers to take into account the various perspectives that improvement strategies must encompass. Improvement strategies must encompass authentic change of process, persistence, comprehensiveness, and the need for teacher fill-in of visions, principles, and directions. This timely and insightful article should be required reading for any school undertaking an improvement effort.

Kirst, M. & Venezia, A. Bridging the great divide between secondary schools and post-secondary education, pp. 92-97.

The NCA CASI movement toward Transitions almost makes this article required reading. The authors deal openly with the linkage problems between high school and college, among which are the following:

Limited access to needed college prep courses.
Grade inflation and the reliance on grades as predictors of college success.
The need for remedial coursework in college.
A lack of early and high quality college counseling for many students.

Among the recommendations they make are the following:

Provide all students with information about access to courses that will prepare them to meet college-level standards.
Examine the relationship between the content of higher education placement exams and K-12 exit-level standards and assessments.
When relevant, use data from state K-12 assessments as an additional indicator of college readiness.
Allow students to take placement exams in high school.

There is much more in this pithy article. It is worth a read by any of our schools but particularly by our schools seeking a Transitions recognition.

McClelland, M., Morrison, F., & Holmes, D. (2000). Children at risk for early academic problems: The role of learning-related social skills. Early Childhood Research Quarterly, 15(3), 307-329.

The authors, all located at Loyola University College, Chicago, are suggesting that children's learning-related social skills, both interpersonal and work-related, contribute to early school learning and performance. In the present study they examined the relationship between work-related skills and academic outcomes at the beginning of kindergarten and at the end of second grade. Children with low work-related skills were found to score lower on academic outcomes both at the beginning of kindergarten and at the end of second grade. They also had more behavior difficulties, lower IQs, and more medical problems.

It seems important that children's families and child-care providers be educated about the need for socialization in the area of work-related skills. Is it really the job of the schools to move into the domain of parents and pre-school care? It may be. If the education of a child is to be a partnership endeavor between school and home, then the professionals (school) may need to take on the task of helping the home become an authentically contributing member of the team.

I also had another reason for reviewing this article. As an idea source in early childhood education it is probably no stronger or weaker than many other such articles. However, I wanted to use it to point up another thought: that as we look ahead one improvement cycle at a time, we should also be looking ahead 10 or 20 years at a time. All interventions aside, the strongest assurance of improvement in learning in the high schools, for example, is for incoming students to be better prepared, both in background knowledge and learning skills. It might be beneficial for the three levels of schools to look at inter-level articulation in a more comprehensive way. In the long haul, better-prepared kindergartners will lead to better-prepared sixth graders, which will lead to better-prepared eighth graders, which will eventually lead to better-prepared seniors exiting from high school. Early childhood education is everybody's business, and "everybody" also includes all faculties beyond the early childhood level.

Ehrenberg, R., Brewer, D., Gamoran, A., & Willms, J. (2001). Does class size matter? Scientific American, 285(5), 78-85.

The noted authors, respectively of Cornell, UCLA, Wisconsin-Madison, and University of New Brunswick, have once again visited that time-exhausted question, "Does class size matter?" So much seemingly inconclusive research has been conducted on this topic that I probably wouldn't have taken a second look at this article if not for the fact that it appears in Scientific American. When a highly respected publication of the "hard" sciences takes an interest in an educational question, it's probably worth another look.

Was it worth a look? As you might suspect, the investigators did not conduct a study of their own but instead tried to make some sense out of a mass of studies that have already been conducted, most notably Project STAR (Tennessee in the 1980s), a large California initiative (1996+) and SAGE (a five-year pilot study in Wisconsin). All of these studies concentrated their efforts on the early grades. The investigators tried to find answers to the following questions: "Do small classes in fact improve school achievement?" "If they do, in what grades do they accomplish the greatest good?" "What kind of students gain the biggest benefit?" "How great is the benefit?"

Answers to these questions turned out to be very general, but preponderance of evidence did allow some answers, to wit: "some evidence that smaller classes benefit students, particularly in the early grades, and especially kids at risk or being underachievers." Based primarily on the well-analyzed results of Project STAR, it appears that teachers' aides do not make a difference. One analysis indicated that the students in small classes out-performed the control groups by one-fifth of a standard deviation. There was disagreement among analysts about the persistence of gain. In short, the results of these analyses, although encouraging, were largely inconclusive. One informal but seemingly critical conclusion was drawn: "much of the benefit of reduced class size may depend on whether the teachers adapt their methods to take advantage of smaller classes." This latter conclusion may well be the key to the future success of class reduction efforts.

Greenfield, T. & Klemm, E. (2001). When "good" school restructuring efforts fail. American Secondary Education, 30(1), 2-25.

In almost all of our seminars, workshops, and literature dealing with the overall concept of school improvement, we urge school people to seek out or develop interventions that are substantive and fundamentally meaningful. So far, much of our school improvement effort has been squandered on band-aid interventions that may be superficial, labor-intensive, or simply not well thought through in terms of basic learning theory. Nevertheless, attempts at deep restructuring can go wrong, too, as shown in this report by Greenfield and Klemm, both of University of Hawaii, Honolulu.

The report is a case study of a large suburban high school that became the first school in that state to attempt to make itself more "user friendly" by creating a school-within-a-school (SwaS) program. The notion that a smaller school unit can be more personal seems unassailable and appears to have been a slow but steadily growing phenomenon since its inception in the USA in 1961. A restructuring of this type is a major undertaking and demands commitment and a serious period of preparation and study. In this case, the student body of 1,400 was divided randomly (thus, heterogeneously) into cores of 125-150 students, with four "core" teachers (science, mathematics, English, social studies) acting as core leaders. The teachers had two planning periods each and planned the instructional and behavioral standards together, as well as spending more-than-normal time with students on an individual basis. These core teachers also received strategy training appropriate for their largest group of at-risk students.

Did it work? In the initial year only one ninth grade core was involved. The remainder of the ninth grade class served as the control group. The goals over three years were three-fold: (1) improve GPA by 25%, (2) decrease absenteeism by 20%, and (3) decrease disciplinary referrals by 25%. The evaluation of the program began in the fourth year and extended on as the program grew to include tenth grade, etc. It consisted of three parts: (1) the degree to which program implementation had adhered to original plans, (2) data on the three student goals (above), and (3) assessment of teachers and staff perceptions of the program's advantages and disadvantages. The response in these three areas of assessment were, as follows: (1) The program was found to have been implemented as planned except that neither gifted or special education students were involved. (2) Student-wise, the program generally met the goals in the first year, but in subsequent years there was serious erosion, confused by the increase of core students, both in class level and ability. (3) Teacher responses were mixed and by no means primarily positive. Many disadvantages were noted and, although the core teachers were more positive than the non-core teachers, neither group supported the notion that the advantages outweighed the disadvantages.

So, what happened? I think the flaws in the program may be reflected in the conclusions of the investigators about what is critical to the success of a restructuring program. They named five points: (1) There must be careful planning to allow participants to work out the meaning of the change to them and to share control of it. (2) The focus must be on substantial, not cosmetic, changes. (3) Teachers must be provided with the time, resources, and emotional support necessary to acquire the needed new knowledge and skills. (4) At some point teachers must begin to see how the change is making things better than they were. (5) There must be a highly effective and credible manager, authentically committed to the program, to coordinate participants and activities, provide resources and support, share control, and exert pressure when needed. One lesson here is that substantive change cannot be undertaken lightly or without adequate research and experiential data, if such is to be had.

Ovanda, M. (1999). Support services for capacity building in successful schools. Planning and Changing, 30(2), 76-100.

Among other items of consideration in school improvement, one that NCA CASI has more recently addressed, is that of capacity, that is, the capacity of the school to undertake an improvement effort. In a 1999 study, investigator Ovanda (University of Texas, Austin) addressed this question in a study of 14 Texas schools. The criteria for inclusion in this judgment sample were success in improving student achievement, a disadvantaged student population, and a high attendance rate (95% or above). Three elementary, five middle schools, and six high schools were included in the study, with 491 faculty members responding to a four-part questionnaire. Three major areas of capacity were assessed: staff development, facilitation services, and resources.

In the area of staff development, "workshops," "conferences," and "demonstrations" were favored over other forms of inservice, with "workshops" being the runaway top choice. Within all types of inservice activity, and within the three top choice types, the most preferred content was "responsive pedagogy," with "vision" as a close second and "instructional and schooling practices" a close third. "Summer institutes" and "consultation" were the least favored inservice activities, and "leadership and governance" was the least popular topic.

In facilitation services, "teacher planning" was the strong top choice, with twice the frequency of response of the second choice, "networking." Once again, as in the staff development area, the most popular related concepts were "responsive pedagogy," "vision," and "instructional and schooling practices."

For resources the investigator assessed self-perceived time involvement in eight types of resource concepts. The results showed that "time sharing" had the greatest time involvement, while "time for school visits" had the least. Close behind time sharing were "instructional material" and "time for conference."

The following addendum is included so that all the assessed variables can be known by the reader, thus giving a complete picture of the infrastructure of variables considered in this study. The results may offer some hints as to how a school can prioritize its assets and efforts in order to maximize its capacity to improve student performance.

ADDENDUM
Learner Content: Vision; Leadership and Governance; Responsive Pedagogy; Family and Community; Capacity Building; Instructional and Schooling Practices.

Staff Development: Workshops; Demonstrations; Visitations; Conferences; Study Group; Summer Institute; Consultation; Individual Study.

Facilitation: Team Planning; Mentoring; Coaching; Presenting Training; Reporting; Action Research; Networking.

Resources: Time for Sharing; Time for Conference; Time for School Visits; Instructional Material; Research Reports; Professional Journals; Professional Books; Additional Funds.

Goldberg, M. (2001). Leadership in education: Five commonalities. Phi Delta Kappan, 82(10), 757-761.

Are educational leaders born or made? This is a meaningful question to us in NCA CASI, since one of the basic building blocks of our school improvement efforts is the identification and development of leaders in our faculties. In this interesting article the author reports on his impressions after 43 interviews with individuals who qualify as "educational leaders." He reports that he found no "algorithm for success in educational leadership," but he did find a few basic traits that these leaders did tend to share.

They hold some fundamental beliefs that are viewed as realistic and are seen as being urgently required in their work.
They all tended to have the courage "to swim upstream," to buck the popular tide.
They tended to have sharply defined social consciences, with clearly remembered experiences that defined their beliefs.
They all exhibited genuine seriousness of purpose, illustrated both by the longevity of their efforts on particular projects and their attention to "rigor" and "excellence."
They all demonstrated situational mastery, even though their many situations and the leadership characteristics that were demanded varied greatly.

Are these traits indentifiable? I think so. Can they be developed? Probably, but I think the key is that in our schools, if we put the faculty truly in charge of the improvement effort, the leaders will emerge, and I suspect most of them will strongly resemble the above profile.

Visscher, A. (2001). Public school performance indicators: Problems and recommendations. Studies in Educational Evaluation, 27(3), 199-214.

I have commented on previous occasions that it is generally comforting to me to note that schools and school systems in other countries usually encounter the same problems and frustrations that American schools encounter. Because they do, we can often learn from their experiences or thinking on many topics. A case in point is the present article, written by an associate professor at the University of Twente, the Netherlands. His topic concerns some of the many problems of producing SPIs (School Performance Indicators) as a means of judging the quality of a school. He has listed problems in three categories: technical-analytical, usability, and political-ethical and societal. Many of these problems do not apply to our schools in NCA but I'm going to paraphrase a few of them that seem appropo to the American condition, all from the technical-analytical category:

Raw, uncontextualized student achievement data tell more about the school's attendance area than about the quality of the school program.
Comparison of schools by ranking is not possible because it is impossible to adjust for all relevant variables.
School performance indicators based on cohort data do not contribute to assessment of program quality, in part because of internal school changes over time.
Student mobility makes it difficult to determine precisely how much effect any one school has had on a student.

How does the NCA CASI improvement process seem to fare with these potential problem sources? The first issue is a non-problem in our situation because each school is in competition only with itself, striving to exceed its earlier performance. It's the same for the second problem of ranking schools. We do not compare our schools to one another. NCA monitors the growth or decline in student performance compared to previous achievements. Nor do we use cohort data for our accreditation reporting. We take internal changes in the school as a given, usually an overt part of developing strategies to improve student achievement. Finally, there is no sure way to compensate for student attrition. However, our use of "equivalent" comparison groups for assessment, rather than cohort groups, tends to nullify the problem because the attrition factor in the baseline group should not be essentially different than the attrition factor in the improvement group.