Finding and Using the Right Assessment
Assessments are usually a series of questions or situations that require a client to make selections or prepare responses. Assessments are created to provide information about the person or persons taking the assessment and are generally more objective than your observations which may, despite all good intentions on your part, be inaccurate, biased, and subjective.
Assessments give you a snapshot of a person’s characteristics at the present time. The results may be compared to other individuals to determine how well a person is performing in comparison to a similar group of individuals. A person’s results can be compared to a set standard to see if she or he has reached the necessary minimum to enter a training program or perform at a set standard. Some assessments are created to predict a person’s future performance in a training program or on the job. An assessment can be used to determine if a person has in interest in a particular field of study or specific occupational family, or even if she or he is ready to take on a school program successfully and balance it with life’s challenges. Assessment can be used to help identify a compatible career path using interests and abilities or to help them with finding a satisfying job. All these are possible reasons that you may want to incorporate assessment into your services.
There are literally thousands of assessments measuring many types of human behavior and characteristics. Some are designed to measure one’s interests, others measure abilities, still others skills or personality characteristics, just to name a few.
If you have decided to use an assessment because you believe it will help you do a better job with your clients and customers, you would be correct, but only if you select the right assessment and use it appropriately. Selecting the right assessment is an important task which should be done carefully and thoughtfully.
Here are some things to consider when selecting an assessment.
Match Your Purpose with the Instrument’s Purpose: First, determine the purpose or reason why you are administering the assessment. What are you trying to measure or determine? State your purpose in very clear terms, write it down, and compare that information to what the publisher states that their instrument is designed to accomplish. Your purpose must match the capability of the instrument and its stated purpose.
Compare an Assessment’s Characteristics with Your Students/Clients: The instrument you select should fit the characteristics of the person or persons who will actually take the instrument. More specifically, an assessment should have directions and items that can be understood by the same age, education level, or experience levels of your clients. As an example, an interest inventory developed for adults in transition may not be as appropriate for a middle school or high school student. A licensing exam for nurses is not likely to be appropriate for people who are entering basic training programs in the healthcare field. A selection test that requires a high level of language capability may not be a good choice for a non-native speaker.
Study the Development Procedures and Norms: Good assessments are based on developmental procedures that involve gathering data on students or clients with characteristics similar to your clients. For example, if you want to use an assessment with a group of adults with a high school diploma or GED, you should select an assessment that was developed through item tryouts and research that included a broad range of this type of individual. If you are using the instrument with a group of females or primarily with individuals of a particular geographic or ethnic background, check to see if the instrument was developed on a group that included individuals with those characteristics.
If the assessment results are used to compare your client’s results with that of a specified group, often called a norm group, check to see that your clients share characteristics similar to that norm group. Relevant characteristics include age, gender, ethnic group and socio-economic status. You need to read the publisher’s technical information to make that determination.
Review Format and Delivery: If there is an option of taking the assessment on paper or via computer, or over the Internet, you, as the assessment user, must be sure that the format and delivery platform do not influence the outcome of the results.
Some research shows that students prefer a computer format for taking an assessment; some older individuals and some ethnic groups may not be so inclined. Assessment takers should not be intimidated by the format of the instrument; rather they should be concentrating on answering the items to the best of their ability.
Review the Scoring Process: A key factor to be considered in the selection of an assessment is the scoring process. Scoring can be performed by the person taking the assessment, by you as the user, or automatically by a scoring program.
Some instruments provide a self-scoring option. One consideration in self-scoring is the complexity of the scoring process. The more difficult and complicated the process, the more likely it is that errors in scoring will be made by your client, potentially leading to incorrect outcomes and interpretations.
Will you need to score the inventory yourself? If you have just a few instruments, scoring them may not be too onerous; but a classroom full of inventories may require a significant amount of time.
Scoring of online assessments generally offers the best of both worlds – accuracy and quick turnaround. However, with this option, privacy and security may be issues.
Review How Results are Reported: You need to look at the kind of information that is returned to the assessment user. You should review the scoring procedures and resulting output to determine whether it is sufficient for your purposes and for the needs of your clients. Look at the example score reports to determine what information is provided and how that compares to what you and your clients would prefer and what you can understand.
Determine the Clarity and Validity of the Interpretation: It is to the assessment taker’s benefit if results are accompanied by accurate, supportable, and clear interpretations. Clarity and validity of interpretations should be prime considerations in the selection of a good assessment.
The publisher should provide evidence that the interpretations being reported are backed up by evidence. Generally, this evidence can be found in the technical manual or by asking the publisher’s representative a direct and specific question on this topic; but be sure that the oral information you receive from the publisher’s representative is backed up by evidence.
The assessment results and the interpretation should contain enough information to be useful for the client or to the program, but not so overwhelming that it is difficult to sort through what is critical to the client.
Decide About Saving and Storing Results: You need to decide if you want to keep the results/scores from the assessments and, if so, how you will do so. Will they be stored in a file cabinet, a database or in other electronic forms? Will the results be safe and made available only to those who have a need to know? You also need to ensure that you can retrieve the results if you need them again.
Check the Technical Information: Be sure to obtain the user’s guide and technical report from the publisher for any assessment you are seriously considering. Read these documents carefully to be certain that the instrument has the characteristics that you want.
Two very important characteristics that you must check out are reliability and validity.
Reliability. Reliability relates to the consistency of the scores. If you administered an assessment to a person, would you get the same score if you did it again some time later?
An unreliable instrument would give you a different score each time. wouldn’t know which score is the correct one. Reliability is expressed as a number from 0.0 to 1.0. The closer the number is to 1.0, the better the reliability. Characteristics that are easier to measure, like aptitude, knowledge, skill, and interests should have reliabilities in the high .80s and .90s. Ideas like teamwork, initiative, dependability, professionalism are more difficult to measure and the reliabilities would tend to be lower, even on well-constructed instruments.
Validity. The idea of validity deals with the evidence that the assessment is measuring what it claims to measure and you can use the results in the way you intended. That evidence is provided by the test publisher or author through various studies that should have been done. However, you must check to verify this. If you want to use an assessment to predict success in a mathematics training program, for example, you need to look for research evidence that says you can use the assessment results in that way. If you want to see if an instrument can indicate that a person is ready to enter a second year language course, you need to see if the publisher has evidence that the instrument can show a relationship of the scores to success in the course. If you are using an interest inventory, what evidence is presented that the instrument is actually measuring various interest categories? What research shows that the instrument is measuring various important abilities. If you are using a personality inventory, does the publisher show that the personality characteristic relates to success on the job? If you are wanting to help a person select a satisfying and productive career path, what research has been presented that shows the assessment results are linked to specific careers, job satisfaction, or job success.
If supportive research is not provided, then the test does not have the appropriate validity, and you should not use the assessment for that purpose.
Review Your Assessment Background: You should not use an assessment for which you are not properly trained. Some assessments, like the MBTI for example, require that you have a certain educational background or a certificate of training completed. If specialized training is required, you MUST obtain it before using the assessment and interpreting the results for a client or group of clients. Using an assessment without the training can hurt the client because you may not be providing accurate results and interpretation. It is a violation of ethical principles to use an instrument for which you are not properly trained.
Some assessments do not require specialized training. You may need only to thoroughly review the administration and interpretation manuals, take the assessment yourself to get a good feel for what a client would need to do, practice the administration a few times with others, and then you should be good to go. Be prepared to answer any questions about the assessment that your clients may pose before, during and after completing the assessment.
It’s a Matter of Professional Ethics
Yes, it is a matter of ethics. You are ultimately responsible for your selection and use of an assessment with your students and clients. It is important review the options in purposeful, systematic, and objective way to make the best selection. You will need to look at the information made available by the publisher, read instrument reviews, check with your colleagues, and even share the instrument with your students or clients to gather their opinions in order to make a comfortable decision about which instrument might be best to use. The final decision rests on you and your professional judgment.
Janet E. Wall, EdD, MCDP, CDFI, SMMS, NCDA Fellow, is a career development professional in Arlington, Virginia. She is committed to the continuing education of all career counselors, coaches and specialists by providing online courses and webinars through
http://www.CEUonestop.com. She was the developer of the ASVAB Career Exploration Program for the Department of Defense, and was recently named a Fellow of the National Career Development Association. She is co-author of the Ability Explorer published by JIST.
Check out CEUonestop.com’s online course “Selecting the Right Assessment.” CEUonestop.com is an NBCC-approved continuing education provider.