Superintendent Horne on the AIMS Test

NOTE: As I am attempting to answer the 474+ emails in my legislative inbox, I came across this letter from Tom Horne, Arizona's Superintendent of Public Instruction. The letter is dated July, 9, 2008 ~ about a week or so after session ended.

OPEN LETTER TO STATE LEGISLATORS REGARDING THE AIMS TEST

I. The AIMS Test is Our Only Mechanism for Holding Schools Accountable.

In a poll required and financed by the state legislature, the public was asked “Do you favor or oppose using statewide tests to determine how schools are performing?” Sixty-six percent favored, 30 percent opposed, and 4 percent had no opinion. Carolyn Warner[1] had CUES, Diane Bishop* dumped that and started ASAP, Lisa Graham Keegan* dumped that and started AIMS. When you keep dumping what you have and start something new, there are enormous start up costs each time, there is no opportunity for stability and continuous improvement, and teachers take nothing seriously because they know it will soon be replaced. We have now had AIMS for 12 years, and have the opportunity for continuous improvement.

AIMS is the Arizona Instrument to Measure Standards. To understand AIMS, one must understand the standards movement throughout the country.

II. The Standards Movement.

The quality of public education in the United States began sinking in the 1960s, and the downward trend accelerated. One of the great qualities of a representative republic is its capacity for self correction. The public noticed how little high school graduates knew, and demanded a change. That change was the standards movement.

Standards are developed by teachers brought together from around the state, and ultimately approved by the State Board. They set out in detail what every student should know and be able to do in a given subject at a given grade level. You can review the performance objectives for any grade in any subject at our website www.azed.gov, click on “Standards.” Teachers are now teaching the academic standards to their students.

Every question on the AIMS test is a measurement of a performance objective. This is what makes AIMS unique. AIMS is the only test where you know the students have been taught the subject of the test questions, because the teachers have agreed to teach the standards, and every question is a measurement of a standard. That is why AIMS is the only test that can fairly hold schools, teachers, and students accountable.

AIMS is no longer a secret test. You can take a sample AIMS test by going to www.azed.gov “AIMS – Take the Test.”


III. Was the AIMS Test “Dumbed Down?”

Some people charged that the AIMS has lost value because it has been “dumbed down.” But last year there were 3,000 students whose teachers had given them passing grades, and who had the credits to graduate, but who could not pass the AIMS test after five tries. The same people who claim that AIMS has lost value because it has been dumbed down, often support the decision of the governor to sign a bill that permitted these same students to graduate with “augmentation” using the subjective grades that they received from teachers. You can’t have it both ways. If the AIMS test is too easy, or was dumbed down, then the governor should not have signed a bill permitting graduation by the students who could not pass an objective test, based on subjective grades given by teachers.

The difficulty of the test has not been changed since 2005. In 2005, the AIMS tests had to be revised because for the first time we had standards for every grade level, and needed new AIMS tests to assess these new standards. That is the only change since the test started in 1996, except high school math in 2001.

The method used to determine the cut scores (the percentage of right answers needed to pass) had to follow certain standards, as required by the federal government. The determinations were initially made by task forces of teachers drawn from all over the state, who decided what level of difficulty constituted “proficient.” This then had to be approved by the State Board.

The level of difficulty for grades three through seven did not change significantly. Eighth grade math had always had a much higher failure rate than all other grades, (including high school) and its failure rate was brought into line with the other grade levels. In high school math, I thought the recommendation of the teachers reduced the level of difficulty too much and recommended to the State Board that a higher level be applied. The vote was eight to one against me. The other eight board members wanted to follow the process that had been set up with the teachers, and I can understand their position. I still believe I was right. But understand that if the Board voted as I recommended, the number of students with the credits to graduate who could not graduate because they did not pass the AIMS test would have been higher than 6,000. Here again, no one can consistently criticize the Board’s decision, yet support the governor’s decision to allow students to graduate who had not passed the AIMS test.

The only way I could have seen to it that the level of difficulty was higher in the recommendation that went to the State Board would have been to limit the task force to a group of elite teachers, rather than the diverse group that we did use. I believe that would have been wrong.

IV. Is AIMS a “Bad” Test?

Some people have been quoted as saying that AIMS is a “bad” test. There is absolutely no basis for their saying that, and it is a totally false statement. Although Arizona teachers write the questions, the psychometrics are validated by one of the largest two companies in the country in that field. Knowing that we could be sued over students not graduating, as many other states have been, we have had a national advisory board of the very top national leaders in psychometrics approve every step that we take. Under No Child Left Behind, the federal government must approve the standards-based test (in our case AIMS) that every state must have. Arizona was one of the first ten states approved out of 50 states. The Fordham Foundation is the principal institution for ranking standards. Arizona standards ranked 7th out of 49 states. Arizona ranks first out of 50 states in the category “curricular content, standards-based reform, and school choice.”

V. AIMS Rewards Not Only Proficiency, But Also Excellence.

To be a highly performing or excelling school, the school must have not only a given percentage of students proficient, but also a given percentage of students as exceeding proficiency. Students who exceed on all three AIMS tests, and meet certain other academic requirements, receive a full tuition scholarship at our state universities. 3,800 students now receive this benefit, and I believe their families are appreciative.

VI. 2:00 A.M. Legislation Is Not Good Public Policy.

At approximately 2:00 A.M. the night that the Senate passed its budget, a provision was added regarding the AIMS test. It called for a task force to study switching from AIMS to something new called AAAT. It would combine current AIMS questions with the ACT college entrance test, as is now done in Michigan. It prohibited the State Board from entering into any contracts for the AIMS test for more than one year.

Representative Crandall, a member of the governor’s P-20 Council, has bragged to the media that he got the Senate to add this provision to the budget at about 2:00 A.M. He has bragged further that he had previously met with education stakeholder groups about this provision, but carefully kept it secret from the Department of Education, because he feels that I am an obstructionist. I am not a committee chair, and I do not have the power to obstruct anything. I only have the power to give you information, because the Department of Education has detailed information that can be valuable to you in making decisions about education. His purpose was to keep you ignorant of the information that I could have provided you, and that you could have considered before making a decision. This is not good public policy making.

Here is what I could have told you had I been given the opportunity.

First, there was no need to restrict the State Board from entering into a contract for more than one year for AIMS. Even if you were to decide to adopt the AAAT test, the fastest that it could be implemented would be 2012 (see Exhibit A, showing the time line).

Second, we are in the fifth year of our current five-year contract for the AIMS test. We were ready to send out bids for a new five-year contract. When the bid can only be for one year, the costs become prohibitively high. Asking vendors to submit a bid every year means the vendor will have to charge much more to recoup the costs of developing and managing the test.

Third, Arizona’s cost for all of its statewide tests (both AIMS and TerraNova) is about $11 per student. Michigan, the state that the 2:00 A.M. provision seeks to copy, anticipates its cost next year at $115 per student – TEN TIMES AS MUCH. (See Exhibit B, document received from the Michigan Department of Education.)

Fourth, Michigan made a presentation to the Council of Chief State School Officers regarding this combining of the statewide test with the ACT. Exhibit C is a copy of one of the pages from the Michigan PowerPoint. It states:
Sticker Shock

· Lobbying claims
· ACT-based NCLB test will be less expensive
· Better aligned to Michigan standards
· Take less time to administer

· Reality
· More than triple the cost
· Requires significant augmentation
· Takes significantly more time to administer

I do not deny that there can be benefits from combining the statewide test with the ACT. I believe that reasonable people can differ about it, and I am certainly willing to study it with an open mind. But making a policy decision like this, which ties the Board’s hands to a one-year contract and thereby makes Arizona’s cost skyrocket, at 2:00 A.M. so that legislators can be kept ignorant of any input from the Department of Education, is not good public policy making.


Tom Horne Superintendent Tom Horne Arizona Department of Education 1535 W. Jefferson Street, Bin #2 Phoenix, AZ 85007 (602) 542-5460 (602) 542-5440 Email: tom.horne@azed.gov

[1] For the more youthful readers: these are former state school superintendents.