PSYCHOMETRIC PROPERTIES OF TESTS VALIDITY -we each have unique personalities, abilities, interests, attitudes, etc.

-tests assess unique patterns of behaviours, interests, personality, attitudes, intelligence, etc. -crucial that tests possesses good psychometric properties. Reliable Valid What is Reliability? -consistency of a test. -take same test over again & obtain consistent score. -correlating two sets of scores; correlation coefficient. -but is test measuring construct of interest. ............issue of "Validity" VALIDITY TYPES OF VALIDITY: Face Validity Content Validity Criterion-Related Validity Predictive Validity

Concurrent Validity Construct Validity Convergent Validity Discriminant Validity What is Validity? defines the meaning of a test. ability of test to measure what it purports to measure. "It is an agreement/relationship between a test score and the quality it is intended to measure". e.g., will SAT scores accurately predict success in CEGEP? "Valid for what?" -test may be a valid measure for something, but probably not the construct you are assessing. -necessary for test to be reliable for it to be valid; but reliability is not sufficient to demonstrate validity

Process of Validating a test: How do we know whether a test is valid? evidence for the validity of a test comes from demonstrating relationship/correlation between the test and other attributes it purports to measure. -(1985) introduced standards for Educational and Psychological tests, -three types of validity that need to be demonstrated; content, criterion & construct **************************************** -we must have convincing proof that there is a relationship between our test and what it claims to measure before we are justified in saying there is this connection and that this test is a valid measure of what we are interested in.

Types of Validity: Face Validity the "appearance" of validity "Does this measure look valid" e.g., if you are taking a test on aggression -items include "How often you are angry" if all you have is face validity, no inferences or generalizations can be made of actual behavior.
o

need to compared their test responses to actual behavior (i.e., aggressiveness in school)

Content-Related Validity test possesses content validity to the extent that it provides an adequate representation of construct you are trying to measure. "Are the items selected in your test a fair sample of the concept you are trying to measure? e.g., asked to study chps. 3 & 8 for midterm but all questions were on chps. 14, 15, 16. Does test have good content validity? (i.e., Is test accurately measuring your knowledge in psychology?) this type of validity is based on logic rather than statistical analyses. Criterion-Related Validity: -how well test can predict relevant aspects of future behaviour you are interested in. How well will the person do on the job?

Which student is most likely to succeed in university graduate program? Who is likely to get a certain disease? -we want to forecast future behavior, but need the information now. e.g; selection committee needs to select which student will do well in grad. prg. -look at GRE scores, select applicants that fall at 95th percentile. How well the test serves as a "stand in" for future performance.

Two types of criterion-related validity. Predictive Validity Concurrent Validity

Predictive Validity -the "forecasting" function of a test. -how well can test predict future performance. e.g.; the SAT has good predictive validity as a CEGEP admissions test if it will accurately forecast how well high school students will do in CEGEP -predictor: is score you obtain on test; -criterion: what test purports to predict/measure. eg: you want to predict student performance in clinical psychology graduate program by looking at their GRE scores. -GRE score (predictor). -grad school performance

i.e., grades, clinical/supervisor evaluations, (criterion, that which you are trying to predict) -the information/number that will tell you about the criterion validity of the test is the validity coefficient. Validity Coefficient correlation coefficient (r); estimate of degree to which two variables are related. (values are between +1 & -1 (0 = no correlation) -the closer the correlation coefficient to (+1), the better the validity of the test in predicting criterion. "r" indicates relationship between test (predictor) & criterion (what test predicts). -"How much of the variation in college performance will we be able to predict on basis of SAT scores alone e.g.; GRE has validity coefficient of r =.60, for predicting GPA at U of O clinical psychology -the higher the correlation between predictor and criterion, the greater the predictive validity of test for making statements about the criterion. Validity Coefficient Squared: how much variance in your criterion (what you are predicting) can be accounted by test (predictor). e.g.; SAT has a coefficient of .60 when it is related to college performance. -36% of variability that you in college GPA is accounted for by SAT scores. -still 64% of the variability unaccounted for. e.g.; ASDSC has r of .7, when it is related to KMSS (accounts for 49% of criterion)

-many factors that contribute to university performance, impossible to expect that entrance exams (SAT, GRE, MCATS) explain all variance -however, because tests are so expensive to administer, you want them to be able to predict some variation in future performance more than you would be able to without the test -using a test may not be worth the effort if it only contributes a few percentage points (chance level) to the understanding of the variation in a criterion i.e., if test accounts for only 20% of variation in university GPA Concurrent Validity -want to know how person is behaving now -tests are taken at same time; -designed to predict persons current performance (assess simultaneous relationship between test and criterion) -typically used in job settings; relationship between employment screening test & job performance. A) test potential employees on a sample of behaviours that are representative of the tasks that will be required of them. e.g.: applying for a job as a mechanic. B) during job interview: ask questions that relate to the position you are applying for. Distinction between predictive and concurrent validity. time lag between administering the test and checking against the criterion. e.g.; Type A (a personality profile highly prone to get HD) and probability of Heart Disease -the results of a valid Type A personality questionnaire can classify person as type A, type B, etc., but will also predict who has HD now and who will eventually develop heart disease. -question asked to obtain Predictive V:

"Who will eventually get HD" -question asked to obtain Concurrent V: "Which person already has signs of HD". Construct Validity -conducted when you first create a test to measure a construct of interest. -first - must define your construct before you can validate it. -next - conduct studies to demonstrate validity of measure. -two processes used to determine construct validity: a) Convergent Validity -new test needs to correlate well with other tests believed to be measuring same construct. Eg: WAIS-R correlates well with other IQ tests (Binet) -other tests you are converge with need to be valid, -if the converging measures are useless, so is your new measure. b) Discriminant Validity (Divergent Validity) -you need to show that your test measures something a bit different from other tests that purport to measure the same construct (Uniqueness of test). -Why devise new test if there is already one around Relationship between Reliability & Validity: -useless to establish validity of a test if it is not reliable. -it is possible to have reliability without validity, but it is impossible to demonstrate that an unreliable test is valid. WHY? because theoretically, a test should not correlate more highly with any other test/variable than it correlates with itself. General criticism of these tests:

-do IQ or achievement tests predict anything?