Introduction to Assessment; Why and How we assess
Chapter 2
Introduction:
-Why do we assess a person, object or process?
-We may want to see if the person (or object) meets certain requirements.
-We may want to compare certain different things or situations, to help us make a decision.
-We may want to see whether a situation has changed over a given period.
What is Assessment?
-Assessment is defined as the process of determining the presence of, and/or the extent to which, an object, person, group or system possesses a particular property, attribute or characteristic.
-Assessment is the systematic collection of descriptive and judgemental information necessary to make effective decisions.
-Psychological Assessment; involves the classification of behaviors into categories measured against a normative standard.
Measurement:
-Difficult to assess whether a person or object has a certain property, or to specify exactly how much of the property they possess (like beauty or intelligence).
#Measurement; consists of rules for assigning symbols to objects in order to;
a) represent qualities of the attributes (scaling).
b) define whether the objects or phenomena fall into the same or different categories with respect to the attributes concerned (classification).
-Measurement involves applying clearly stated rules to determine how much of a certain property, characteristic, or attribute is present in a particular object, person, system or process.
-Attaching a numeric value to that phenomenon as an aid to assessing or interpreting it.
-Measurement is more accurate than just assessing.
Evaluation:
-Involves interpreting or attaching a judgemental value to an assessment.
-For example, seeing water bubbling, assume water is near boiling point and hot, no need to test it.
-Called Observation.
-Testing, on the other hand, is the use of an intervention of some kind to carry out the assessment.
-Not very accurate like measurement.
Properties of a good measure:
-Will have the following properties;
1. It will attach an observable phenomenon (eg: the height of the mercury) to the unobservable phenomena (eg: temperature).
2. There will be correspondence between the two phenomena (eg: the hotter the water, the higher the mercury).
3. The observable phenomena (eg: the height of the mercury) will be scale-able (have a numerical value eg: degrees Celsius).
4. There will be a relative consistency between the two phenomena, when the temperature is at a certain level, the mercury will always rise to the same scale level.
5. Because the rules are transparent, and consistently applied, different observers will be able to agree on the value assigned to the phenomenon.
The Advantages of Quantification:
-Six major advantages to quantification;
1. Objectivity:
-The extent to which any process and its results are agreed to by neutral or unbiased observers, and is thus independent of the personal or subjective judgement of those involved.
-Objectivity is enhanced when numerical values are attached to an object or phenomena, in terms of known rules.
2. Precision:
-Measurement allows finer, more precise distinctions to be made, leaving room for more subtle effects to be noted than is possible when personal judgements is made.
3. Analysis and Comparison:
-Quantification allows for the more sophisticated analysis of patterns and trends, using statistical techniques like t-tests, analysis of variance, etc.
4. Generalisability:
-Key aspect of any scientific enterprise is to find ways of generalizing from the specific to the general.
-Measurement allows us to quantify and classify objects within larger super-ordinate classes.
-Thus, we are able to specify what each has in common with other cases, and how they differ.
5. Communication:
-Easier to communicate and interpret information that is in symbolic or numeric form.
6. Economy:
-Easier to utilize tools/instruments to measure and state data than using words.
Why do we assess?
1. To obtain accurate information to describe an existing situation.
2. To gain an understanding of reasons for the occurrence of an existing situation.
3. To suggest ways of changing an existing situation.
4. To illustrate the impact of any intervention or change process.
5. To enable us to predict the future behavior or actions of the people involved.
-Also, assess performance of groups, organizations, pairs of people, etc.
Formative and Summative Assessment:
1. Formative Assessment:
-Is concerned with what happens during a process, and is designed to help the person managing that process to gain insight into what is taking place, and to modify it, as necessary.
-More concerned with the steering process.
-Eg: ask students how they feel about the course, to help lecturer adjust aspects.
2. Summative Assessment:
-Concerned with the outcome of a process, or the current status of a phenomenon, for example, the results of the end of a course, test, or exam.
How do we obtain data?
-Six major methods to do so.
1. Direct Observation:
-Means person doing the assessment watches or observes the behavior of the person in question, in a particular setting and draws conclusions based on this.
-Can be time-consuming and labour intensive, but useful when respondents are self-conscious, unable or unwilling to talk or when behavior is complex.
-Possible to observe animals (Branch of Psychology called Evolutionary or Comparative Psychology).
2. Historical Records:
-Because people are generally fairly consistent in what they do, the best single predictor of how they are likely to behave in the future, is how they have behaved in the past.
-This includes consulting existing documents like school, court or medical records, case histories or track records.
3. Referral Information:
-Most properly organized schools, clinics, hospitals and other agencies have some kind of standardized intake form, from which it is possible to pick up potential problems.
4. Interviews:
-To ask questions of the person and/or those involved, like parents, teachers, etc.
5. Written Answers:
-Participant may be required to produce written information.
-Range from unstructured questions, to questionnaires.
-May include a narrative statement, survey, scale, or even a test.
6. Intervention:
-Involves some form of direct observation by the observer, an attempt/effort to answer, "What if?" questions.
#Triangulation; simply means we should not rely only on one form of assessment, but use as many different approaches as are warranted.
-Info gathered across; multiple measures, domains, sources, settings and occasions.
-The assessment process is multidimensional in nature; it entails the gathering and synthesizing of information as a means of describing and understanding functioning. This can inform appropriate decision-making and interventions.
Levels of Measurement:
1. Nominal Data:
-Also called categorical (because it refers t the categories).
-Used to indicate membership of a class.
-Like males and females.
-The only property that is associated to the Nominal scale is equivalence, that is, that all objects with the same number, belong to the same category.
-The only permissable statistical manipulations that one can perform with Nominal data is to count the number of cases.
2. Ordinal Data:
-Ordinal scales place objects in rank order, so that they can be graded harder, bigger, stronger than the other objects.
-Does not say how much each property has.
-The permissable statistics that can be used include percentiles, interquartile range, median and various order based statistical tests.
3. Interval Data:
-With Interval data, the size of the difference (or interval) between scores is regarded as equal.
-There is no absolute zero.
-The permissable statistic that can be used with Interval data are, the arithmetic mean (average) and statistics based on variance, like t-tests, Pearson Correlation and Analyses of Variance.
4. Ratio Data:
-Characterized by equal intervals and an absolute zero.
-Thus "twice" and "half" are meaningful.
-The permissable statistical techniques are the same as with Interval data, but include ratios as well.
-Points about the levels of measurement;
a) Very little difference between ratio and interval data.
b) In almost all cases, Psychological data are, at best, Interval Data.
c) We can easily move from a higher form of data like Interval, to a lower form like Ordinal or Nominal.
How do we know if our measure is a good one?
-There are 3 basic properties;
1. Is my measure relatively consistent?
-If it measures in a constant fashion, we get similar results under different conditions.
-This is known as reliability.
2. Does my measuring device measure what it claims to measure?
-Does the instrument measure what it is intended to.
-Known as validity.
3. Is my measure fair? Does it treat what is being measured, fairly?
-Whether the technique is equally valid for different groups of people (bias, racism, discrimination).
Problems associated with Quantification in Social Sciences:
-Schools of thought argue psychological phenomena are not "real" but rather, constructed by the people involved.
-Two schools of thought;
1. Empiricist (Quantitative Approach)
2. Constructivist (Qualitative Approach)
-The empiricist believes that the world is real, and exists outside the experiences of the observer, and therefore, can be measured in reasonably accurate ways.
-The constructivist argues that everything is in the mind and is created by the observer, in terms of categories and relationships that have been learned.
-In the middle, critical realists argue that there is a real world out there, but that this is shaped and constructed, by our life experiences, value systems and the cognitive schemata and categories that we bring to bear on the issues.