<p>They're NOT junk science.</p>
<p>Correlates of the SAT in minority engineering students: an exploratory study. (Scholastic Aptitude Test)
Jacqueline Fleming Carole Morning
7191 words
1 January 1998
Journal of Higher Education
89
Vol. 69, No. 1, ISSN: 0022-1546</p>
<p>I've skipped some of the methodology but this is far from JUNK SCIENCE.</p>
<p>Measures </p>
<p>The following measures, adapted from research on minority student development in college (Fleming, 1984; Winter, McClelland, & Stewart, 1981), were administered as part of the standard evaluation of Project Preserve students: </p>
<p>Cognitive skills. (1) Concept Formation (Heidbreder, 1948), a measure of basic associative process skills, where the task is to link nonsense syllables with new versions of pictorial concepts. (2) Thematic Analysis (Winter & McClelland, 1978), a measure of comparative analytical ability, where the task is to compare two sets of stories in any way desired. (3) Analysis of Argument (Winter, McClelland & Stewart, 1981), a measure of intellectual flexibility, where the task is to argue against a controversial article on Dr. Spock and permissive child rearing, and then to defend it. Therefore the task requires arguing both for and against one's own beliefs. </p>
<p>Adjustment to college and motivation. (1) An adjustment to college questionnaire, adapted from Fleming (1984), including assessments of: (a) social adjustment; (b) academic adjustment; (c) interactions with college teachers; (d) interactions with high-school teachers; (e) interactions with the mother; (f) interactions with the father; (g) self-concept items; (h) personal threat (adapted from Holmes & Rahe, 1967); (i) subjective perception of the best aspects, worst aspects, and influence of the previous college where they failed (Stewart, 1975); and (j) background information on aspirations, ethnicity, and social class (Hamburger, 1971). </p>
<p>(2) Thematic Apperception Test (TAT), consisting of five verbal cues and scaled for aggression in accordance with previous research indicating its diagnostic value in minorities (Fleming & DuBois, 1981). The cues were scored for: (a) need for Achievement (McClelland, Atkinson, Clark, & Lowell, 1992), a measure of concern for doing things well; (b) Fear of success (Horner & Fleming, 1992) a measure of concern for achievement conflicted by self-sabotaging tendencies; and (c) need for Power (Winter, 1992), a measure of concern for having impact on others. </p>
<p>(3) The Mandler & Sarason (1952) Test Anxiety Questionnaire (TAQ) as a measure of Fear of Failure. </p>
<p>(4) The Holland Vocational Interest Inventory (Holland, 1970; 1973), a measure of six vocational orientations: Investigative (scientific), Realistic (mechanical), Artistic, Social, Enterprising (business), and Conventional (clerical). </p>
<p>Grade point average was available for a maximum of 41 semester grades and 37 cumulative grades. Measures were administered in the order indicated, except that the TAT was given first. </p>
<p>Procedure </p>
<p>Project liaisons arranged for entering students to attend group testing sessions with the project evaluator, a Black female (JF). Groups ranged from 2 to 20 students per session, and the sessions were 2 1/2 hours. Testing sessions were usually conducted mid-semester. Test scores and grades were made available to the evaluator by the project liaison. </p>
<p>Treatment of Data </p>
<p>Twenty-nine percent of the standardized test scores were ACT scores, and the remainder were SAT scores. Thus, ACT scores were converted to SAT scores using a conversion formula given in Langston & Watkins (1976) and updated in Marco & Abdel-Fattah (1991). Test scores were then correlated with the remaining measures in the battery. </p>
<p>Cognitive skills were scored according to procedures described in Fleming (1984) and Fleming, Garcia, & Morning (1995). Scales were constructed from items in the college adjustment questionnaire to measure the following six dimensions: academic adjustment (measuring satisfaction with and commitment to academics), social adjustment (social involvement), interactions with college teachers (nature and degree of constructive contact), interactions with high school-teachers, interactions with the mother, and interactions with the father. Items with scale correlations as well as alpha coefficients were consistent with standard levels. For measures of motivation, total scores were used as well as scores to individual TAT cues and items of the TAQ. </p>
<p>Test scores were correlated with the whole sample and then separately by sex, despite the small numbers, for the sake of thoroughness and in light of significant sex differences in the earlier literature. Although test scores were not correlated with social class in the whole sample, the findings were nonetheless controlled because the dependent variables might be related to SES. Partial correlations were then conducted using father's social class, with mean substitution for missing data (n = 4). An additional control for mother's social class was required for the female results, because their test scores were significantly correlated with this measure. </p>
<p>There are a number of concerns in using SAT scores to predict grades. First, Ramist, Lewis, & McCamley-Jenkins (1994) found that pooling course grades into a composite GPA with no control for comparability of courses diminished predictive validity. However, in their research this problem was minimized among African American students, who comprise 81% of the present sample. Furthermore, lack of course comparability should be less of a problem in the engineering curriculum, because it is heavily dominated by math and science courses, which typically yield the highest SAT validity coefficients. Second, SAT scores were pooled across institutions to perform correlations with grades and other measures because of the small number of students. Such a procedure can be problematic because of expected institutional variations in predictive validity and other school characteristics (cf. Willingham & Lewis, 1990). However, predictive validity coefficients are given separately for each school, and significant school differences in dependent variables are noted in the text. Third, note that no correction for restriction of range of test scores was employed, which may underestimate the size of the predictive validity coefficients. </p>
<p>Results </p>
<p>Predictive Validity of Standardized Test Scores </p>
<p>Table 2 presents correlation coefficients for SAT (or ACT) scores for first and second semester and cumulative grades of all Project Preserve students as well as a function of sex, ethnicity, and college. Due to larger numbers of subjects, the first semester grades are assumed to be the [TABULAR DATA FOR TABLE 2 OMITTED] most reliable and most relevant, because the correlation was produced in the project school, whereas cumulative grades include past performance. In most cases, the first semester correlations were similar to the average for all four correlations. In the aggregate, test scores produced a low correlation with semester grades of 0.242 (5.8% of the variance). The validity coefficient for females was higher than for males but still in the low range, i.e., 0.283 (7.9%) versus 0.209 (4.4%). Predictive validity for African American students was substantially higher than the low negative coefficient for Latino students: 0.281 (7.8%) versus -0.049 (0.2%). By school, only predictive validity for Xavier was in the high range (0.533; 28.4%), whereas the coefficients for both CSUN and CCNY were quite low: 0.163 (2.7%) and 0.015 (0.02%), respectively. Only the coefficients for Xavier were statistically significant. </p>
<p>Correlates of the SAT </p>
<p>Table 3 shows that standardized test scores were correlated with 14 variables suggesting a low achievement orientation. Of the three measures of cognitive skills, test scores correlated significantly only with Concept Formation (r = 0.337, p [less than] 0.05), a measure of basic associative process skill, but not with two measures of higher-order critical thinking skills. Test scores were not significantly correlated with semester or cumulative GPA for the first semester in the program. Nor were there significant correlations with the measure of academic or social adjustment scales. However, two significant correlations were produced with items of the Academic Adjustment Scale: negatively with the importance of getting good grades (r = -0.339, p [less than] 0.05); and negatively with the cramming index, i.e., the increase in studying during exams (r = -0.310, p [less than] 0.05). Test scores were also uncorrelated with the Interactions with College Teachers Scale (ICT) and the Interactions with High School [TABULAR DATA FOR TABLE 3 OMITTED] Teachers Scale (IHST), but there were two negative correlations with items of the IHST Scale: seeking assistance from high-school teachers with courses (r = -0.396, p [less than] 0.01); and seeking assistance from highschool teachers with planning the future (r = -0.341, p [less than] 0.05). Test scores were correlated with two items from the College Climate Questionnaire assessing perception of the previous colleges, where students failed the first year: negatively with "the best of (the previous) college is new people" (r = -0.337, p [less than] 0.05), and "the influence of (previous) college is identity formation" (r = 0.354, p [less than] 0.05). Test scores were correlated with two measures of motivation: negatively with the Test Anxiety Questionnaire, conceived as a measure of fear of failure (r = -0.339, p [less than] 0.05); and negatively with need for Achievement to story 3: A mother is about to punish her child again (r = -0.306, p [less than] 0.05). Finally, test scores were correlated with sex, such that males achieved better test scores (r = -0.328, p [less than] 0.05).</p>