About the Author: Dr. John Woodward is the Director of Research and Development at the NCA Commission on School's Center for Developing Student Success. He can be reached at 520-825-8245.

A Paradox in the Implementation of the School Improvement Process

The process of school improvement involves the identification of goal areas, the identification or creation of assessment instruments to collect data about student performance in each goal area, and implementation of interventions that will improve student performance area on the specified assessments. Teachers and administrators, because of their sincere desire to help young people, want to implement interventions as quickly as they can find or create them. Unfortunately, this creates a two-fold problem: first, it can cause student performance to improve before baseline data are even collected, and second, it can set the staff on a path of intervention that may not be consistent with the goals that were specified or desired by the staff. To make matters worse, if the process of investigation takes too long before teachers can begin implementing interventions, teachers can become impatient with the process. This scenario frames our paradox. We must be reasonably thorough in doing our homework on assessments and interventions, but we must be expedient at the same time. We must remain focused, and all of this must be accomplished in the midst of school schedules that allow little time for staff to do the necessary investigation.

Assistance from the NCA Commission on Schools

The good news is that there are resources available to assist schools with these tasks. One such resource, available through the North Central Association Commission on Schools, is a database of information which contains clarifications of about 50 goal areas, a bibliography of best practice interventions, and a bibliography of assessments that have been reviewed by test and measurements specialists for reliability and validity. NCA schools can access this database free from the NCA Commission on Schools website at http://www.nca.asu.edu/.

How to Use the Expert Database

From the home page, click on the link to "member services." A pop-up screen will ask for your user name. You should enter "ncaschool" in lower case and with no spaces between words. The pop-up screen will also ask for your password; enter "membersonly" in lower case with no spaces. Click "OK." Next, click on the button for "Neville’s Library." Upon entering Neville’s Library you will find "first," "second," and "third" designations. The "second" section contains the link to the target areas, target area goals, strategies, and assessments. This is also known as the Expert Database. Click on this link and begin exploring the database. The first screen lists over 50 target area goals in alphabetical order. (See a sample list of target areas that can be used in goal clarification.) To investigate writing, follow the steps below.

Step 1 Click on the link to Communication Skills: Writing.

Step 2 Examine the "essence" of the goal at the top of the page.

Step 3 Examine the examples of different purposes for writing. This may help to clarify the kind of writing your faculty desires from students.

Step 4 Scroll to the bottom of the page. Click on the strategies and interventions link. A set of best practice interventions will be displayed. After the name of each intervention is a description of the intervention and a grade level designation. Following the description is a list of citations of articles that reported on the use of the intervention and the conclusions for those citations. That is, conclusion #3 corresponds to citation #3 and reports the major findings of the article listed.

Step 5 Click on the "back" button on your web browser to return to the goal page.

Step 6 Scroll to the bottom of the page. Click on the assessment instruments link. The screen will display a list of assessment instruments. These assessment instruments have been reviewed by NCA Research and Development and meet NCA criteria for use in school improvement assessment.

The above steps are provided to assist the reader in navigating the database to gather information. Later in this article there are suggestions for using the information obtained. Those who use the database should recognize that even high quality assessments and interventions may not be appropriate for a specific school.

Suggestions for Using the Expert Data Base

Essence – what is it and how can it be used?

The "essence" of a target area goal is the conceptual heart of the goal. It is like a definition of the competency, and it is specific in nature. The essence of a goal needs to be defined or established by the faculty so that everyone is clear about the meaning of the target area goal. The staff must first be able to define what type of written communication they are talking about before they can choose assessments and design interventions. Therefore, it is suggested that the "essence" of the goal provided in the database could be used to start discussion about what the target area goal means in your school. If the staff members do not have consensus on the essence of the target area goal, they will be inconsistent in their understanding and implementation.

Essence – the problem with ignoring it

The concept of "essence" may be new one to your staff, but overlooking its importance can greatly diminish the effectiveness of your school improvement efforts. Every target area goal is a concept, and concepts are interpreted differently by each staff member. For example, "citizenship" is often cited as a target area goal. If "citizenship" is not defined in the beginning, one-third of the staff might think that citizenship means following rules, one-third might think that citizenship means serving others, and one-third might think that citizenship means participating in governance. The school staff needs to clearly define the "essence" of each target area goal because interventions and assessments are selected or created based upon the goals. When essence is not clearly defined, the school risks having different staff members interpret the goal differently. The lack of consensus about "essence" can lead to some staff members implementing interventions and assessments while other staff members withdraw from the process or actually attempt to subvert it.

Assessment Instruments – how to use this section

This section of the database contains a bibliography of assessment instruments, all of which have been reviewed by psychometricians and found to have solid reliability and validity. This section will provide some guidance about age and grade appropriateness and it will sometimes provide clues about the focus of the assessment. The school improvement team might want to acquire a copy of the assessment from the publisher to see if it measures what they want to measure. If there are no assessments listed that measure exactly what your staff wants to measure, assessments may have to be created. This database may help a school improvement team construct a rationale for why they chose or created a specific assessment instrument. It should help staff understand what the assessment instrument reveals and what it does not. The database may lead to general education batteries that will assess an area like writing, reading, or math; it may also lead to content specific assessment instruments that are more diagnostic.

It is suggested that the school improvement team select assessments prior to selecting interventions. Two different assessment instruments may both assess the same objectives, but students may perform better on one assessment than another. This occurs because the way in which one assessment tests the objective is more consistent with the way the objective was taught (requiring less transfer on the part of the student). Also, selecting interventions before selecting assessment instruments encourages teachers to begin intervention prior to collecting baseline data. It is better if the baseline data are not influenced by early implementation of the interventions.

Interventions – how to use this section

The most important issue to address when selecting or creating interventions is to find interventions that will increase student performance by removing obstacles to student learning. The database contains a bibliography of best practice interventions. It is suggested that the school improvement team contact the vendors or suppliers of the interventions or materials and examine them closely to determine what they do and whether they are likely to help your students. When examining best practice interventions, it may be helpful to identify what they offer that is different from what the school staff is currently doing. Pay special attention to the conclusions section and note the assessment instrument(s) in which improvements in performance were noted. A school that gets improvements with the TerraNova may or may not get improvements on the Iowa Test of Basic Skills. Just because an intervention works on one school site does not mean that it will work in another.

In addition to addressing the reasons why students are not meeting the desired level of performance, the interventions must be within the limits of the school's resources and they must be compatible with the school's curriculum, instruction, assessments, and staff development. When the school improvement team investigates best practice interventions, they should note special requirements for implementation of the interventions. Finally, it may be that the staff will want to meld several interventions or modify interventions to fit their specific situation.

We Want Your Feedback

We are updating the expert database and would be very interested in your experience with the database. What is most helpful to you? What would you like to see added or changed? Do you like the web site interface? Please send comments to Dr. John Woodward, Director of Research and Development, NCA Commission on Schools, P.O. Box 8878, Tucson, AZ 85738.