Four Ways to Go Wild

You and I are a minority. We like testing. We believe testing can provide benefit to individuals and to society. Certification programs are especially helpful to individuals, employers, and the public. As clients, participants in my workshops, and readers of my publications, you know that my passion is to help certification organizations provide services that delight their customers.

Many of the readers of my e-zine and blog ask me specific questions to help them delight their certification customers and clients. There are four ways to experience my thinking and the first way is FREE.

The other three ways are Workshop Enrollment and/or Buy my Books, Limited Engagement and Wild’s Signature Engagement.  FREE is a first, easy, and no-excuses way to Go Wild.  I invite you to start planning to participate in these free services as part of your New Year’s planning. Build them into your game plan for 2012.

Each year I make many efforts to present at professional meetings.  I get no compensation for the preparation, the presenting, the travel costs so, that makes it one of my FREE services for you.  In February, I will be at the Association of Test Publishers meeting in Palm Springs California. I’ll be presenting in a session called Building Quality Communications to improve Stakeholder Relations, on February 27th with Dania Eter(National Council of State Boards of Nursing, Inc.), Georgia Patrick (The Communicators, Inc.) and Cheryl Wild (Wild & Associates, Inc.).

Another free feature this year is the first annual Benchmarking Day for Improving Certification on March 28, 2012.This event was specifically requested by participants in my Improving the Quality of Your Certification Program: A Learning and Planning Experience and by folks in my ICE workshop. Only clients, past workshop participants, and e-zine subscribers are invited to this free workshop.

There’s a very good reason why the Free Benchmarking Experience is designed for people who know me, read my books, and invite me into their organization to work with them. Benchmarking is not just sharing what we do. Effective benchmarking requires identifying and sharing best practices – after all the purpose of benchmarking is to make major improvements or to design services to be best practice from the beginning. Benchmarking requires:

  1. A willingness to admit that you can learn from others
  2. A desire to achieve best practice rather than an attitude of doing enough to get by
  3. A willingness to share nonproprietary information even if it means discovering that your processes aren’t the best
  4. An understanding that to identify best practice we need to develop metrics and measure ourselves

Those of you who have had experience with me are among the best certification organizations in the industry and have exhibited anopenness to improvement. Benchmarking is one of the highest forms of equal sharing of very precious information and we don’t do that with strangers. This only works in a trusted environment and the one point of trust everyone in the room has is me, as their consultant, as their experienced guide.

What are some of my other free services? If you are reading this you know about my blog. You may also sigh up for my e-zine at http://www.wildandassociates.com/

If you have specific questions that you would like answered in the blog or ezine, write me at cheryl@wildandassociates.com or call me at 732-774-5188. If you are interested in more than the free venues offer, consider one of the other ways to get involved:

  • Workshop and Books: Call me (732-774-5188) to discuss your participation in the two-day workshop just prior to the free benchmarking day. Buy my book.
  • Limited Engagement: Call me to discuss issues and concerns you are struggling with that could be addressed in a limited consulting engagement. I frequently conduct internal audits against ISO/IEC 17024 standards and help organizations benchmark specific processes or procedures
  • Wild’s Signature Engagement: Call me to discuss operation issues that need expertise and process improvements or new certification development. These often include staff training so that you can move this work in house in the future.

Building Quality Communications to Improve Stakeholder Relationships!

As a consultant in quality, I have the privilege of talking to lots of testing professionals about their problems and I’m always surprised how often the problem really boils down to communication. Reliability, validity, cut score studies, and stand setting are all important. The success of these psychometric niceties rests on effective communications with stakeholder.

Communication seems so obvious, why would we even need to discuss it? Last February, during my workshop on Improving the Quality of Your Certification Program, we were discussing the importance of communication to quality. We all agreed that lots of communication happens, but often communication doesn’t occur in an organized and deliberate fashion. As test publishers that use data all the time to evaluate our tests, we forget that communications with stakeholders are important data for planning and evaluating our tests and services. Every participant in the workshop develops their own follow-up project and Dania Eter from NCSBN chose to focus on communication for the NCSBN.  Click here to see more about the upcoming workshop in March.

The popularity and confusion about social media sites also became part of the discussion. So many sessions talk about using social media, but often social media is used in a haphazard way. We “have” to be on Facebook because it is the in thing to do – but what do we do with it? Georgia Patrick started talking to Dania and me about being purposeful in social networking, and the idea for developing some positive examples for effective communications in the testing industry was born.

Do you want to learn more about building quality communication programs? Dania Eter (NCSBN), Georgia Patrick (The Communicators) and I encourage you to attend our session on Monday afternoon, Feb. 27 during the Association of Test Publishers [MSOffice1] national conference from Feb. 26-29 at Westin Mission Hills, Rancho Mirage, CA.  Our presentation will give you just a little theory about why communication is essential and to move on to examples from the Certified Commission of Healthcare Interpreters, the NCSBN, the Board of Certification for the Athletic Trainer, the National Institute for Certification in Engineering Technologies, the Woodwork Career Alliance of North America, and the Security Industry Association.

Happiness

The Happiness Project: Or, Why I Spent a Year Trying to Sing in the Morning, Clean My Closets, Fight Right, Read Aristotle, and Generally Have More Fun by Gretchen Rubin is the best book I’ve read this year. (True Confession – I even liked it more than the latest book by Charlaine Harris, who is my current favorite escape novelist.) A combination of self-help book and memoir, you can’t help but be happy when you read about someone spending a whole year working on becoming happier! What excited me about the book, even beyond its ability to evoke happiness, was Ms. Rubin’s application of a process approach (similar to process approach used for quality planning and quality improvement) in her quest for happiness.

 

So what was the process for Ms. Rubin’s happiness project? First, she did her basic research (secondary research in the quality jargon). She read widely – positive psychology, philosophy, biography, and history. Using what she learned, she defined happiness – what it is and what it isn’t. She identified some overarching principles for her happiness project (activities I’d call defining the scope of the project). Continuing to define her project, she identified the subjects she wanted to cover and the time frame for covering the topics. (No good quality guru forgets to have a project plan!) Ms. Rubin developed resolutions for each month. Resolutions needed to be concrete and measurable. Each evening she used her chart to record whether she was meeting her resolutions. (Reader, please let me know if you would like some references on process control charts.) Each month she set out to try various ways of improving happiness. Some worked and some didn’t. She continued with what worked. (Pilot projects are an effective way to test a theory – what works in one situation doesn’t always work in another.) At the end of the year she concluded that she was happier.

 

What worked best for Ms. Rubin to achieve this goal?

1.      Learning about someone else’s successes and failures was a better catalyst for change than studying a theory. Some ways she learned from others included

a.      Reading biographies and memoirs. For Ms. Rubin those about Saint Thérése of Lisieux  and her memoir Story of a Soul were very meaningful

b.      Asking for feedback and experiences through her blog. She actually self-published a book of her blog posts (learn more on the happiness project web site http://www.happiness-project.com/ )

c.       Encouraging feedback from her husband, family and friends through-out the project

2.      The resolutions chart was the single most effective step she took to make sure she was following through on her resolutions. By the way, follow-through was the hardest part for Ms. Rubin. (Implementation is often the hardest part of a quality project too.)

3.      The feeling of greater control that resulted from identifying resolutions and then following them was a critical element to Ms. Rubin’s increased happiness.

 

Summer calls for a vacation from our usual business reading and I would recommend The Happiness Project as a the perfect summer break. Reading about someone becoming happy simply made me happy. If you are one of those personality types questing for examples of ways to improve, the processes followed by the author are useful tools both in and out of the office. Enjoy. Be Happy.

Managing Certification Programs Using Measurements

I have been teaching a course for the National Graduate School called Performance Based Management. Management courses are often based on theory. In this course we emphasize that theory must be tested with the actual performance of a process in practice. So in this course we talk about how to evaluate a process and develop measures for the process. Developing measures in theory is easy. Most process measures relate to the time it takes to finish the process, how much it costs, or to the quality of the product or service. If your goal is to decrease cost, then the measure is how much something costs. When time is critical, a measure related to the calendar time or processing time may be the most important. If not cost or time, the important measure is often related to quality. Cost, time, and quality are often interrelated, and more than one measure is important.

One of the reasons this graduate program is so beneficial is that the students are required to apply what they learn to an improvement process in their organization. So not only do we discuss the theory of developing measures of a process, students have to develop measures for a process they want to improve and then show improvement using those measures. Applying what you learn is hard, but the experience of developing some measures of your process and applying them provides great insight and understanding of the implications of the theory of measuring process.

For example, several of the teams in this class hypothesized that mistakes were happening because employees are not familiar with parts of a process. How do you measure knowledge of the process? You can:

  1. Develop a test about the knowledge employees need to have:
    • Multiple choice
    • Performance test, or
  2. You can develop rating scale to evaluate employee knowledge in different areas (perhaps on a scale of no knowledge to competent):
    • Self rating or
    • Supervisor rating

Students quickly learn that in either case you need to understand the important content dimensions surrounding the job. You have to evaluate the type of “knowledge” that is to be assessed – is it something that you can write a multiple-choice knowledge question about? Is it a performance? Do employees know enough about the dimensions to tell you their level of knowledge? Or would a supervisor know enough to rate the employee?

Certification programs can benefit from performance based management just as much as any other kind of organization. The critical question is what are the important processes to measure in your organization and how should you measure them? If you are interested in learning more about the importance of performance based management, you might be interested in observing a video about Dr. W. Edward Deming’s “Red Bead Experiment” that is available from http://managementwisdom.stores.yahoo.net/redbeexlivo7.html . This video vividly demonstrates how the system of management influences individual performance. Remember this is an example of how NOT using performance based measures can cause failure (please don’t emulate this).

You might also be interested in reading Harley-Davidson: Preparing for the Next Century by R. L. Nolan and S. Kotha (2006), available from Harvard Business School Press (Product #9-906-410) to see a case study of an organization that does use performance based measures.

Deadlines are Important

I have been procrastinating about my blogs. Blogs seem simple. Three-hundred-fifty to five-hundred words are all it takes. The computer is counting the words as I write, the spell checker is underlining mistakes, and my editor is back from her cruise. I have finished teaching my class. Can I use having to go to the bank and the post office and excuse to finish this tomorrow? NO—the deadline is on my calendar today to finish my blog and I have a conference call at 1:00 to work with my IT consultant on how to post blogs more effectively.

According to the World Book Dictionary, procrastination is “the act or habit of putting things off to later; delay, especially repeatedly.” Although the definition says procrastination is something you do repeatedly, it doesn’t mean that you procrastinate on everything. I find that I repeatedly procrastinate on certain things (my blog has been one sore spot for me).

Does this sound familiar? My clients often procrastinate too and one of my unstated jobs as a consultant is to help them move forward. Here is what I’ve noticed about clients (and myself) and why we procrastinate:

  1. The job seems too hard
  2. There isn’t enough time to do the work
  3. The information or material needed to do the work isn’t available
  4. The person doing the work doesn’t know exactly how to do it
  5. Other priorities seem more important

How do I help my clients (and myself) stop procrastinating and move forward?

  1. Break the work down into smaller pieces
  2. Determine when each step needs to be completed
  3. Determine who will be doing the work
  4. Identify and obtain the resources (supplies, expertise) needed to do the work
  5. Inform managers and appropriate staff about the work deadlines to assure that the staff assigned to do the work will be given the time to do it
  6. Provide training on each step as we go to be sure that everyone knows their part
  7. Have regularly scheduled meetings to review steps (weekly is desirable) with everyone to see how we are progressing
  8. If we fall off the schedule (either for procrastination or a more concrete reason), discuss remedies and how to return to the schedule

My clients hire me for my subject matter expertise – expertise in tests and measurement, expertise in quality management, expertise in standards for certification. Experience in the subject allows me to make quick decisions about next steps and to break the project into the smaller parts. Recognizing when my clients need additional training as they go through each part of the process  helps them work more efficiently (and quit procrastinating).

If you are interested in an insightful book about the importance of expertise in decision making, I’d recommend Malcolm Gladwell’s Blink: The Power of Thinking without Thinking. He provides examples of how experts quickly reach decisions and explains that we often make better decisions by training our minds and senses to focus on the most relevant facts. He illustrates how training enhances high stakes decision making (isn’t that why we all have fire drills).

If you would like to get a little more serious about project management you may want to look at Just Enough Project Management by Curtis Cook (2005). It is a quick read that provides a four step process and includes a very helpful chapter on managing multiple priorities and project overload (problems that always seem to exist). The Project Management Institute has a library of resources on project management at  http://www.pmi.org/ .

References for Setting Cut Scores

I’ve been asked by several organizations to recommend a methodology for setting a cut score that would meet accreditation standards. It sounds like a simple request, but it isn’t. You need to consider the type of test you have (e.g., multiple choice versus performance), the target population, and who should be making the cut score decision. The standards are not prescriptive and assessors are not supposed to be prescriptive either! Many certification organizations would prefer to conduct the study themselves. If you are in that boat, there are several books that have come out recently that can give you direction.

1. Certification: The ICE Handbook (2009, published by the Institute for Ceredentialing Excellence), Edited by Joan Knapp, PhD, Lynn Anderson, PhD, and Cheryl Wild, PhD. See chapter 11 by I. Leon Smith, PhD, and Cornelia Springer , CAE on standard setting.   (Includes results of a 2008 survey on what standard setting method is most frequently used to set cut scores by NCCA accredited organizations!)

[Read more…]

Continuous Improvement of Testing Processes

2011 Florida WorkshopCheryl L. Wild, Rohit Ramaswamy, and David Anderson presented a session at ATP on how a certification or testing organization can continuously improve their processes. To see the power point for this session, click here.  If this presentation was of interest, there are three ways you can learn more about the concepts.

First, this session was based on Improving Testing. You may want to read what others have said about the book at LINK. Although books are useful learning tools, they do not easily allow for exchange of ideas and discussion.

The second option is to participating in Improving the Quality of Your Certification Program: A Learning and Planning Retreat and Follow-Up, a learning and planning retreat with three months of follow-up resulting in a Quality Plan – your plan will apply quality concepts to your program. [Read more…]

Fast Tracking Best Practice Assessment Through Task Force Review at ATP

Fast Tracking Best Practice Assessment through Task Force Review at ATPColleen Anderson, Steve Barkley, Casey Marks, and Cheryl Wild presented a session entitled Fast Tracking Best Practice Assessment Through Task Force Review at ATP. To see the PowerPoint slides for the session, click here.

Wild & Associates is focused on improving the performance of its client organizations. If you are intrigued by the Certified Financial Planning Board of Standards positive results with the Task Force, you will be interested in learning more about the Certification Task Force Package. [Read more…]

Improving the Quality of Your Certification Program: A Learning and Planning Experience

Many of you have asked me where to begin improving your certification program. My usual “it depends” answer is not very satisfying – for either of us.

Of course, I follow up my “it depends” response with lots of questions and we can come up with an improvement plan. However, this individual work with you comes at a price, and many professionals have told me they would like to learn more about quality tools and techniques before embarking on a round-the-world quality journey. [Read more…]


Connect with Cheryl on LinkedIn