Book Review – C. Woodley and D. Cox

<<< Back to Book Overview

10/31/08

Cynthia Woodley
V.P. of Operations at Professional Testing, Inc.

David Cox
President of Professional Testing, Inc.


Question: What have you specifically gained from the book?

Cynthia:

  • The chapter by Peter Kronvall from SWEDAC has been especially helpful to me. It provides a history of ISO/IEC 17024 from a European standpoint. It describes how the precursor document led to 17024 and how 17024 is being implemented in Sweden. Europe has a different perspective from the United States on how we got to 17024 and often implements certification differently than the United States. For example, Mr. Kronvall describes how schemes are developed outside the certification organizations in Sweden and how certification bodies all develop certifications around the same scheme.
  • The leadership chapter by Richard Smith was also really helpful. David and I run the company with 27 employees. I identify with the issues raised in this chapter, especially employees being resistant to change. We have wanted to go through 9001, but we need to develop more documented systems and procedures before applying for certification. This development is difficult due to the resistance.

David:

  • Improving Testing is one of the first attempts to develop quality testing practices. Improving Testing provides ammunition to clients about what they can do to create quality in certification and how they can build that quality incrementally. In the past, standards have provided some guidance to quality, but there is a commercial battle between the goals of providing valid tests, providing tests quickly to the market, and minimizing the costs of development and maintenance. Especially smaller certification organizations, which have less time and financial resources, often want products developed that directly conflict with some standards. Testing companies often address legal defensibility of a certification but don’t always go the next step to provide evidence that a test is valid. The fact that nobody complains about adverse impact is viewed as sufficient. Certification organizations do not want to support ongoing audit and improvement efforts. The large CBT providers use sales people who aren’t knowledgeable about psychometrics and often direct clients down paths that are beneficial for the CBT provider but not psychometrically appropriate.
  • Looking into the future, I don’t see government oversight of testing. However, accreditation is helping. There is an inherent conflict of interest in a vendor doing a validity study of their own test or attesting to the quality of the services provided. However, a third party system of verifying quality management techniques will help the overall quality of testing tremendously.
  • Purchasing is an area of poor quality in the testing arena. (There is a chapter on contracting by Judson Turner in Georgia.) There is lots of poor purchasing and this impugns the results of the whole process.
  • Not allowing adequate time for a vendor to respond.
  • Poorly written RFP often shoots the client in the foot because they cannot get a dialogue going with vendors to identify the true needs and specifications.
  • Nonprofits are sometimes compelled to run competing bids even when they have no intention of accepting a new vendor.
  • Performance bonding is misunderstood, and not an insurance policy. It is an inappropriate product for the testing industry. We did some research at CLEAR – no one has ever had an experience of a performance bond being paid for them. If an organization calls on a performance bond, the surety company steps in and takes over the project – the surety company has the freedom to pick and choose whom to work with and may be someone that the organization doesn’t want to work with.

Question: Did you expect to find the book practical? Was it practical?

Yes, to both. The combination of quality assurance and testing has not been done before. It is a uniquely positioned book that gives a lot of practical advice.

Question: What are you doing differently or planning to do differently since digesting the book?

We have provided all the psychometricians with copies of Improving Testing.

We are definitely interested in going down the quality management path. We have developed procedures and a method to obtain feedback on the effectiveness of the procedures. The next step is to address people issues and the time requirements for moving forward.

Cynthia will head Working Group 30 on the revision of 17024 and the book has been helpful in understanding ISO/IEC 17024 in Europe.

Question: Finally, can you help me complete this thought, based on your own experience?

“Readers of Improving Testing are likely to benefit from the book if…”

  • They are interested in developing and maintaining a quality examination program – particularly if they don’t know where to start.
  • They want some practical techniques for developing a testing process.

Purchase the book now directly from the author:


















Or click here to buy Improving Testing at Amazon.com

Purchase this book from Amazon

<<< Back to Book Overview


Connect with Cheryl on LinkedIn