Assessing the Achievement of General Education Objectives:

Three Years of Assessment

 

 

 

 

September 2001

 

 

 

Trudy Bers

Office of Research

Oakton Community College

1600 E. Golf Rd.

Des Plaines, IL 60016

(847) 635-1894

tbers@oakton.edu

 

 

 

 


Assessing the Achievement of General Education Objectives:

Three Years of Assessment

 

EXECUTIVE SUMMARY

 

In spring 2001 the Oakton Student Academic Achievement Team (SAAT) conducted the third year of its project to assess general education learning outcomes.  A sample of 1183 students enrolled in spring 2001 was given and answered questions about prompts covering selected general education objectives.  Trained faculty used rubrics developed for the prompts to score students’ work.  A total of 1763 scores was obtained, because one prompt covered two different general education areas.  Rubrics were based on a 3-level scoring system, with the top two levels indicating a student met standards for general education objectives on that prompt.  These general education objectives were covered:

 

·        Explain how information fits within an historical context

·        Compare/ evaluate alternative solutions to a problem

·        Identify problems

·        Effectively communicate ideas in writing

 

The last two objectives were based on the same prompt and student responses.  Two different rubrics were used to score the students’ work, one rubric dealing with the objective of indentifying problems and the other dealing with the objective of effectively communicating ideas in writing.

 

Researchers examined the association between a number of student demographic and academic characteristics and students’ achievement levels on the prompts.  Results were:

 

·        Overall, 64 percent of the students’ responses were scored at a “passing” level; i.e., 64 percent of the responses indicated that students met general education achievement levels.  Thirteen percent of responses were scored high pass and 51 percent were scored low pass.  The number of responses scored for each prompt and percent of students who passed is presented below:

 

How information fits within an historical context

 

S. Carolina flag

N = 467

Compare/ evaluate alternative solutions to a problem

Illiniwek

N=422

Define problems

 

Clone-problems

N= 437

Effectively communicate ideas in writing

Clone-write

N = 437

 

N

Pct.

N

Pct.

N

Pct.

N

Pct.

High pass

48

10

106

25

44

10

  34

  8

Low pass

241

52

  142

34

241

55

280

64

No pass

178

38

174

41

152

35

123

28

 


 

·        Seventy-two percent of students received passing scores on effectively communicating ideas in writing, a substantial increase from past years.

·        Fifty-five percent of students whose work was scored for both identifying problems and effectively communcating ideas in writing—the same prompt was used for both objectives--received passing scores on both rubrics applied to the prompt. 

·        Statistically significant associations[1] between independent variables and achievements on general education objectives were:

 

·        Gender was associated with the achievements on fitting information within an historical context  (69% of males achieved passing scores and  59% of females achieved passing scores).

·        Age was associated with fitting information within an historical context and effective writing.  On both prompts, a larger proportion of younger (below age 25) students achieved standards than older ones (students aged 25 or above).

·        Cumulative credits earned at Oakton was not associated with achievements on any prompt. 

·        Cumulative grade point average was associated with comparing and evaluating alternative solutions to a problem and effective writing.  Students with higher GPAs were more likely to achieve passing scores.

·        Mathematics competency was associated with achievements on fitting information within an historical context, but there was no logical explanation for why this might be the case.  Students with college-level mathematics competency were most likely to achieve a passing score and students with remedial-level competency were least likely to do so.

·        English competency was associated with achievements on all four prompts.  In all cases, students with college-level writing competency and those for whom no information was available were substantially more likely to achieve a passing score than students at the remedial level. 

·        The percentage of students who achieved passing scores on the general education assessment varied in each of the three years of this general education assessment project (1999, 2000 and 2001).  In 1999, 59 percent passed, in 2000 77 percent passed and in 2001 64 percent passed.  In 2000 one prompt was answered in a multiple choice format and 95 percent of students achieved passing scores, skewing results for that year.  Differences in the prompts, scoring rubrics, general education objectives, instructions to students and faculty reader procedures across the years make it inappropriate to compare achievements from one year to the next.


INTRODUCTION

 

This report describes the three years of the Oakton Student Academic Achievement Team (SAAT) project to assess general education learning outcomes.  Results from three years are presented, with special emphasis and description of the year three project.  More details about the assessments in 1999 and 2000 are available in several articles (Bers, 2000; Bers, 2001; Bers, Davis and Mittler, 2001).

 

PROJECT DESCRIPTION

 

In the spring semesters of 1999, 2000 and 2001 a subcommittee of the SAAT implemented a locally developed institutional approach for assessing students’ achievements of general education objectives. [2]  While refinements to the project were made in the second and third years based on experiences of prior years, the overall process was essentially the same over the three-year span.   The approach used prompts selected to cover a subset of general education objectives.  Prompts were narrative or graphic documents such as editorial cartoons, newspaper articles, or graphs that faculty determined addressed the selected general education objective.[3]  Each prompt was accompanied by a series of questions students were asked to answer.   For example, in 2001 one prompt described a controversy in South Carolina about flying the confederate flag over the state’s capital building.  Students were asked to write a paragraph in which they explained to an international student not familiar with it, the historical context of this controversy.  

 

Different prompts were used in each year, and different general education objectives were assessed.  Prompts were administered to students enrolled in some 90 Oakton classes during the week of April 23-28, 2001. Classes were chosen purposefully, with the intent of identifying courses enrolling large numbers of students with 30 or more college credits.[4]  However, all students enrolled in the selected classes completed the assessment.  Prompts were distributed randomly among students, with each participant completing just one prompt in a class.  Students themselves were asked to select a different prompt if they had already completed a prompt in a different class.

 

Trained faculty members using scoring rubrics developed by subcommittee members scored student responses.  Faculty readers received training at the beginning of each scoring session.  Initially two readers scored each prompt.  Students who received two scores of 3 were awarded a “high pass,” those who received two scores of 2 or scores of 2 and 3 were awarded a “low pass,” and those with two scores of 1 were awarded a “no pass.”   Where readers differed on whether or not the student passed, a third reader scored the prompt.  If the third reader scored the response as a pass, the student was given a “pass.”  If the third reader scored the response as a “no pass,” then the student was given a “no pass.”  Scores were not added together to compute a numeric total.  Rather, the number of “high pass,” “pass” and “no pass” evaluations were used to determine the student’s final score. 

 

     Prompts in 2001 were designed to assess four general education objectives.  One prompt was a website report about a conference announcing that human cloning would soon be available for therapeutic purposes.  It talked about benefits and detriments of human cloning.   Students were asked to define problems associated with human cloning based on the article.  Answers were scored in two different ways.  First, we looked to see whether  the answer indicated the student met the general education objective to define problems.  Second, we looked to see whether the student effectively communicated his or her ideas in writing. 

 

A second  prompt was a statement about Chief Illiniwek written by a University of Illinois alumnus.  The statement focused on the controversy in Illinois about using Chief Illiniwek as a mascot for university athletic teams.  The general education objective examined was students’ achievements in comparing and evaluating alternate solutions to a problem.  Students were asked to write a paragraph to in which they compared and evaluated at least three of these alternatives for moving toward a solution to the problem.

 

The third prompt was about the controversy in South Carolina over flying the Confederate flag above the state capital.  It examined students’ achievements in explaining how information fits within an historical context.  Students were asked to write a paragraph in which they explained to an international student not familiar with it, the historical context of the South Carolina flag controversy. 

 

            We obtained data for the project from both the students’ scores on the prompts and the College’s student information management system.  The dependent variable is the student’s achievement on the prompt.  We also identified a number of demographic and academic variables to use as controls: age, gender, cumulative grade point average, cumulative college-level credits at the college, English composition proficiency, and mathematics proficiency.  These same independent variables were used in both 1999 and 2000 as well.  They are summarized in Appendix A.[5]

 

            A total of 1183 students completed one or more of the general education assessment prompts.  A total of 1763 prompts were scored (recall that one prompt was actually scored twice, and a handful of students may have completed more than one prompt).  Appendix B provides a profile of key demographic and academic characteristics for the total sample of 1183 participants and for the subset of students for each rubric.

 

            As in the year 2000, students could indicate whether they wanted to receive their results.   The SAAT subcommittee organizing the assessment believed that students might take the exercise more seriously if they could obtain their results, and good pedagogy consistently emphasizes the value of feedback to students about their work.  Six hundred and thirty-two students requested their results, which were sent to them in August.  Appendix C contains a copy of the letter sent to them.

 

 

RESULTS

 

Table 1 shows that overall, 64 percent of student responses were scored at a level to indicate students met general education achievement levels, with 13 percent in the high pass and 51 percent in the low pass categories.  There were, however, disparities in pass rates among the prompt/rubric combinations. 

 

One in ten South Carolina flag prompts was scored at a high pass, and slightly more than half were scored as low pass answers.  Thirty-eight percent of student responses to this prompt did not meet general education standards.  The largest proportion of high pass scores was on the Illiniwek prompt regarding alternative solutions, where 25 percent of students received a high pass.  However, barely one third of students received a low pass on this prompt.  Thus the percent of students who met general education standards on the Illiniwek prompt (high and low pass combined) was the lowest among the four scored prompts.  Like with the flag prompt, one in ten cloning prompts scored for defining problems received a high pass, and another 55 percent were scored as low passes.  Thus just over one third of students did not meet standards for the general education objective to define problems.  Though the percentage of high pass scores for the cloning prompt scored for writing was the lowest of the four prompts (8%), the percent scored as low pass was the highest (64%).  Overall, 72 percent of cloning prompts received a passing score when evaluated for effective writing.  This is in marked contrast to prior years, when prompts scored for writing yielded the lowest percent of passing scores. 

 

 

Table 1

Results of General Education Assessment by Prompt/Rubric

 

 

How information fits within an historical context

 

S. Carolina flag

N = 467

Compare/ evaluate alternative solutions to a problem

Illiniwek

N=422

Define problems

 

Clone-problems

N= 437

Effectively communicate ideas in writing

Clone-write

N = 437

 

N

Pct.

N

Pct.

N

Pct.

N

Pct.

High pass

48

10

106

25

44

10

  34

  8

Low pass

241

52

  142

34

241

55

280

64

No pass

178

38

174

41

152

35

123

28

 

 

We next sought to explore whether the independent variables were associated with students’ assessment scores.  We used chi-square analyses for each prompt, cross-tabulating students’ scores on the prompt with each independent variable.  We did so first using the three-part scores (high pass, low pass, no pass) and then again using a two-part score of pass, no pass.  Table 2 presents results when we used the two-part score of pass, no pass.[6] 

 

Gender was associated with the achievements on putting information within an historical context.  Sixty-nine percent of males but only 59 percent of females achieved standards.  Age was associated with achievement on two general education objectives: information within an historical context and writing.  Sixty-seven percent of students under age 25, but only 55 percent of those age 25 or above, met the historical context standard.  Seventy-six percent of younger students but only 65 percent of older students met standards for effective writing.  The results for age run counter what is normally found at the college, where older students generally perform at higher levels than younger ones.

 

Cumulative credits earned at Oakton was not associated with achievements on any of the prompts.  Cumulative grade point average was associated with achievements on two prompts.  As would be expected, students who had higher grade point averages were more likely to achieve standards.  Two-thirds (67%) of students with cumulative GPAs of 3.0 or higher met standards on comparing and evaluating alternate solutions, 55 percent with GPAs of 2-2.9 met standards, and 51 percent of students with GPAs below 2.0 also met standards on this objective.  Eighty-one percent of students with GPAs of 3.0 or higher met standards for effective writing, two-thirds (65%) of students with cumulative GPAs of 2-2.9 met standards, and 71 percent of students with GPAs below 2.0 also met standards for effective writing.

 

Students’ mathematics competency was associated with scores on just one prompt, the one testing students’ achievements in being able to fit information within an historical context.  Over two-thirds (68%) of students with college-level mathematics competency met this standards, 57-58% of students whose competency was at the level of intermediate algebra or for whom there was no mathematics information met standards, and 52 percent of students with remedial mathematics competency met standards. The finding of a statistically significant association between mathematics competency and meeting standards on this general education objective is interesting but difficult to explain, because there does not seem to be a logical connection among the content of prompt, what students were asked to write about to respond to the prompt, and achievement in mathematics. 

 

Finally, students’ writing competency was associated with their scores on all four prompts.  The patterns were similar for all prompts as well.  Table 3 presents these data. Betweeen 60 and 75 percent of students with college-level writing competency met standards, between 60 and 70 percent of students for whom there was no writing information met standards, and between 25 and 36 percent of students with remedial-level writing competency met standards. 

 

Table 2

Association between Independent Variables and Results

 General Education Assessment by Prompt/Rubric

Associations noted by 4

 

             Prompt

Variable

How information fits within an historical context

S. Carolina flag

N = 467

Compare/evaluate alternative solutions to a problem

Illiniwek

N=422

Define problems

 

Clone-problems

N= 437

Effectively communicate ideas in writing

Clone-write

N = 437

Gender

4

 

 

 

Age

4

 

 

4

CUMHRS

 

 

 

 

CUMGPA

 

4

 

4

MATH

4

 

 

 

ENGLISH

4

4

4

4

 

 

Table 3

Percentage of Students Meeting Standards and Writing Competency

by Prompt/Rubric

 

                     Prompt

 

Writing Competency

How information fits within an historical context

S. Carolina flag

N = 467

Compare/evaluate alternative solutions to a problem

Illiniwek

N=422

Define problems

 

Clone-problems

N= 437

Effectively communicate ideas in writing

Clone-write

N = 437

College-level

64

60

69

75

Remedial

25

23

25

36

No information

65

60

61

70

 

             

 

Oakton students for whom there are no English or mathematics competency data may already possess a bachelor’s degree or at least 24 credits of college level work at the C level from other institutions, or have registered for fewer than 13 credits at Oakton.  They would, therefore, not have been required to take placement tests unless they wished to register for English composition or mathematics courses.  Thus, absence of data about competencies in these areas should not be interpreted to mean the student lacks competencies. 

 

            Finally, we examined whether there was an association between students’ achievements on the two scoring rubrics applied to the same prompt.  Recall the same prompt was scored using a rubric for defining problems and for communicating ideas in writing.   There were 424 prompts scored using both rubrics.   Sixty-six percent were scored “at standard” for identifying problems and 72 percent were scored at standard for writing.   Only 46 students achieved standards identifying problems but failed on the writing score, whereas 72 students achieved standards in writing but failed on identifying problems.   Fifty-five percent of students achieved standards on both of the general education objectives assessed on this prompt.

 

COMPARING THE THREE YEARS OF ASSESSING GENERAL EDUCATION

 

In all three years, faculty selected prompts, developed scoring rubrics, and scored students’ written responses.  In the second year one prompt was answered with multiple choice questions only and did not require faculty involvement for scoring.  Some 95% of students received passing scores on the multiple choice prompt.  Faculty decided not to use a multiple choice answer format in the third year.   In all years classes with high numbers of students who had earned 30 credits or more were selected for inclusion in the study, and the same six independent variables were used in the analysis of results.   However, the actual prompts and the scoring rubrics changed in almost all cases from one year to the next, as did the general education objectives being assessed.   Communicating ideas effectively in writing was assessed in all three years, and the scoring rubric for assessing writing remained the same in years two and three.  In year one, students were not told their writing woult be evaluated.  In both years two and three they were told this.   Faculty scoring for writing achievement were given more explicit instructions in the second and third years as well.  For example, they were urged to keep in mind that students’ writing samples were done in class in a limited period of time, and should be viewed as initial products, not final assignments.

 

Students were offered the opportunity to receive their individual results in years two and three.  Fifty-seven percent of the 603 students who completed responses in year two wanted their results and 53 percent of the 1183 students who completed responses in year three requested them.   There is no way to determine whether the availability of results might have affected students’ performances on the assessments.

 

            Changes in prompts, objectives, scoring rubrics and faculty reader instructions make direct comparisons from one year to the next impossible.   Despite these reservations, some observations can be made about the general education assessment project findings overall. 

 

            Only some independent variables were associated with students’ achievements.  Table 4 depicts results for the three years.  Cumulative credits earned at the college, gender and age are not generally associated with students’ achievements on the general education assessment.  Mathematics competency is more often associated with results.  Cumulative grade point average and English writing competency are the two independent variables most often linked with students’ achievements in meeting general education objectives.  Meeting objectives is operationally defined as a passing score on the general education assessment.  As expected, a higher proportion of students with high grade point averages achieve standards as compared to students with lower grade point averages.  The association of writing competency and meeting standards is more complex.  Students whose competency is at college-level do better in meeting standards than students whose competency is at the remedial level, but about the same proportion of students for whom there are no writing competency data available meet standards as the proportion of students whose competency is college-level.  Absence of data about writing competency is not an indicator of lack of competency per se.

 


 

Table 4

Association between Independent Variables and Results

 General Education Assessment by Prompt/Rubric

Associations noted by 4

 

 

 

Year 1 – 1999

Year 2 – 2000*

Year 3 - 2001

 

General education objective

 

 

Independent Variable

Humanities

 

Social  Science

 

Math

 

Communications

 

Gather, analyze & interpret data

 

Apply ethical principles to local, national  & global issues

 

Effectively communicate ideas in writing

 

Fit infor-mation in historical  context

Iderntify & evaluate alternative solutions

Define problems

 

Effectively communicate ideas in writing

 

Gender

 

 

 

 

 

4

 

4

 

 

 

Age

 

 

 

 

 

 

 

4

 

 

4

CUMHRS

 

 

 

 

4

 

 

 

 

 

 

CUMGPA

 

 

 

4

4

4

4

 

4

 

4

MATH

 

 

4

 

4

 

4

4

 

 

 

ENGLISH

 

4

4

 

4

4

4

4

4

4

4

 

·        In Year 2 a fourth prompt assessed students’ abilities to differentiate between facts and opinions.  Responses were multiple choice; 95 percent of students achieved standards rendering further analyses meaningless.

 

 

            In the first year of the general education assessment project, 59 percent of the prompts reflected work that achieved general education standards, although only 44 percent of prompts assessed for writing competency met standards.  In the second year,  77 percent of the prompts reflected work that achieved standards, and the percentage of prompts assessed for writing competency that achieved standards rose to 55 percent.  In the third year,   64 percent of prompts reflected work that achieved standards, and the percentage of prompts assessed for writing competency climbed to 72 percent.

 

            The high percent of “at-standard” scores in the second year is due primarily to the fact that 19 out of 20 students who responded in the multiple-choice format for the prompt to assess one general education achieved a passing score.  Excluding this prompt, the percentage of students meeting standards in the second year drops to 69 percent. 

 

            The increase in the percentage of students who achieved standards in writing requires much speculation.  It is unlikely that writing competency per se could improve so dramatically among students in just a two year period.  It is likely, instead, that a combination of factors influenced these scores.  A revised scoring rubric and more thorough instructions to both students and faculty scoring the writing prompt may in part explain the higher percentage.  Also, faculty in the third year may have been more generous in scoring tests, giving higher scores than faculty scorers in the second year.  It is tempting to claim credit for student improvement in writing, but premature to do so.

 

DISCUSSION

 

            The project described above is a unique approach to assessing general education.  In considering the findings, these issues should be kept in mind.

·        Data about students have limitations. Many students who have attended other colleges and universities do not transfer their credits to Oakton, especially those who have already earned a bachelor’s degree.  Consequently data about credits earned are incomplete because they do not include credits from other institutions. 

·        Students about whom there appear to be no data about English placements/courses or mathematics placements/courses may in fact have taken English or mathematics courses elsewhere.

·        It is still too early in the assessment project history to judge whether the percentage of students achieving a passing score on the general education objectives assessed in this project is a “good” or “poor” percent.  Since the project has now been implemented for three years, it may be time for the Student Academic Achievement Team, general education subcommittee and other faculty groups to begin conversations about what a “good” or “poor” percent might be.
 

·        Different students participated in the project in the three years; therefore student gains measured through a pretest-posttest design cannot be assumed. 

·        Oakton’s general education assessment project has not been subjected to rigorous scrutiny with respect to validity and reliability.  We accord face validity to the prompts and the scoring rubrics.
 

·        Like their colleagues in 1999 and 2000, faculty members who selected prompts, developed scoring rubrics, and evaluated students’ work on the prompts continued in 2001 to find the task of identifying appropriate prompts and formulating clear questions for students to answer to be a difficult one.  

·        Students’ work on this assessment project was not part of their regular coursework.  We can only speculate on whether asking students to do work which did not matter for grades or graduation invited them not to take the work seriously.

·        The finding that writing competency is often associated with general education assessment achievements, especially in the third year, suggests that students’ abilities to write clearly and their competencies on other general education objectives may either be closely linked or that weaknesses in the general education assessment process and design may be creating a situation in which writing skill confounds accurate assessments of competencies on other objectives.  The Student Academic Achievement Team and the general education subcommittee that leads the assessment project may wish to pay particular attention to this concern in 2002.

 

 

REFERENCES

 

Bers, Trudy.  “Assessing the Achievement of General Education Objectives: Second Year of a College-wide Approach and Lessons Learned.”  Journal of Applied Research in the Community College.  [Forthcoming].

 

Bers, Trudy.  “Assessing the Achievement of General Education Objectives: A College-wide Approach.”  Journal of General Education.  49:3 (2000).

 

Bers, Trudy, Davis, B. Diane, and Mittler, Mary L.   “Assessing the Achievement of General Education Objectives in the Community College: A Project Across the Disciplines.  Assessment Update.  May/June, 2001.

 

 


Appendix A

Variables

 

Variable

Operational definition

SCORE

Score on prompt, using rubric appropriate to assess achievement of specified general education objectives:

High Pass = two readers gave scores of 3

Low Pass = scores of 2 and 3 by two readers, or scores of 2 by two readers

No Pass = two readers gave scores of 1

CUMHRS  Cumulative college-level credits at the College though fall 1998 semester (does not include transfer credits from elsewhere)

Four categories:

1 = no cumulative credits

2 = 1-29 cumulative credits

3 = 30-44 cumulative credits

4 = 45+ cumulative credits

CUMGPA  Cumulative grade point average at the College through fall 1998 semester (A = 4; does not include grades earned elsewhere)

Four categories:

1 = no GPA

2 = GPA 0 – 1.9

3 = GPA 2.0 – 2.9

4 = GPA 3.0 – 4.0

AGE

1 = below 25

2 = 25 and above

GENDER

Male or female

ENGLISH

Note:  for analyses 3 categories used:

   1 = College level

   2 = Remedial level

   3 = No information

Highest level of English composition, rank ordered as follows:

1 = college level composition course with grade of C or better

2 = placed into or enrolled in college level composition course, but did not earn grade of C or better

3 = remedial composition course with grade of P (pass) – includes English as a second language and remedial

4 = placed into or enrolled in remedial composition course, but did not earn grade of P – includes English as a second language and remedial

5 = no information about English composition

MATH

Note: for analyses 4 categories used:

   1 = College level (trig or above)

   2 = College algebra

   3 = Remedial

   4 = No information

Highest level of mathematics, rank ordered as follows:

1 = college level (trigonometry or above) course with grade of C or better

2 = placed into or enrolled in college level (trigonometry or above) course, but did not earn grade of C or better

3 = college algebra course with grade of C or better

4 = placed into or enrolled in college algebra, but did not earn grade of C or better

5 = remedial mathematics course with grade of P (pass)

6 = placed into or enrolled in remedial mathematics course, but did not earn grade of P

7 = no information about mathematics

 

 

 

 

 


Appendix B

Key Demographic and Academic Characteristics

(Cell numbers do not always add to totals because of missing data)*

 

 

 

Prompt

 

Total

(Undupli-cated HC)

How information fits historical context

Compare / evaluate alternative solutions

Define problems

Effectively communicate in writing

 

S. Carolina flag

Illiniwek

Clone-problems

Clone-write

 

 

 

 

 

Total Number

1183

467

422

437

437

 

 

 

 

 

 

 

 

 

 

 

 

N

Pct.

N

Pct.

N

Pct.

N

Pct.

N

Pct.

 

 

 

 

 

 

 

 

 

 

 

CUMHRS

 

 

 

 

 

 

 

 

 

 

1 = no cumulative credits/no info

92

8%

71

15%

26

6%

34

8%

27

6%

2 = 1-29 cumulative credits

421

36%

133

28%

133

32%

157

36%

159

36%

3 = 30-44 cumulative credits

201

17%

80

17%

80

19%

70

16%

70

16%

4 = 45+ cumulative credits

469

40%

183

39%

183

43%

176

40%

181

41%

 

 

 

 

 

 

 

 

 

 

 

CUMGPA

 

 

 

 

 

 

 

 

 

 

1 = no GPA/no info

92

8%

29

2%

26

2%

35

3%

29

2%

2 = GPA 0 – 1.9

129

11%

50

4%

41

3%

45

4%

45

4%

3 = GPA 2.0 – 2.9

490

41%

199

17%

186

16%

179

15%

180

15%

4 = GPA 3.0 – 4.0

472

40%

189

16%

169

14%

178

15%

183

15%

 

 

 

 

 

 

 

 

 

 

 

AGE

 

 

 

 

 

 

 

 

 

 

1 = less than 25

778

71%

321

73%

285

72%

301

75%

307

75%

2 = 25 and older

313

29%

117

27%

111

28%

101

25%

101

25%

 

 

 

 

 

 

 

 

 

 

 

GENDER

 

 

 

 

 

 

 

 

 

 

Male

476

44%

202

46%

160

40%

175

43%

176

43%

Female

615

56%

237

54%

236

60%

228

57%

234

57%

 

 

 

 

 

 

 

 

 

 

 

ENGLISH

 

 

 

 

 

 

 

 

 

 

1 = college level

879

74%

360

77%

325

77%

325

74%

330

76%

2 = remedial

59

5%

24

5%

13

3%

24

5%

25

6%

3 = no information

245

21%

83

18%

84

20%

88

20%

82

19%

 

 

 

 

 

 

 

 

 

 

 

MATH

 

 

 

 

 

 

 

 

 

 

1 = college level – trig or above

547

46%

231

49%

214

51%

202

46%

208

48%

2 = college algebra

156

13%

71

15%

46

11%

58

13%

57

13%

3 = remedial

231

20%

88

19%

72

17%

87

20%

88

20%

4 = no information

249

21%

77

16%

90

21%

90

21%

84

19%

 

*  Some students’ gave inaccurate or unreadable social security numbers on their responses, preventing matching assessment results with data from the student information system.  Age and gender are for the 1091 students for whom SSN matches were available.


 

Appendix C

Letter Sent to Students Who Requested Their Assessment Results

 

General Education Assessment Results

 

Spring 2001

 

Last spring you participated in a short activity to assess your achievement in meeting selected general education objectives at Oakton.  You requested that your results be sent to you.  This note lets you know whether your assessment indicates that you have met Oakton standards for the general education objective(s) for which you were evaluated.  We are also providing you with information about how all Oakton students did on the general education assessment.

 

There were actually three different “prompts” to which students responded.  They were randomly distributed to students enrolled in about 90 Oakton courses.  You probably responded to just one of the prompts. 

 

The prompt about cloning was a website report about a conference announcing that human cloning would soon be available for therapeutic purposes.  It talked about benefits and detriments of human cloning.  You were asked to define problems associated with human cloning based on the article.  We scored your answers in two different ways.  First, we looked to see whether your answer suggested that you met the general education objective to define problems, and then we looked to see whether you effectively communicated your ideas in writing. 

 

The prompt about Chief Illiniwek was written by a University of Illinois alumnus concerning the controversy in Illinois about using Chief Illiniwek as a mascot for university athletic teams.  The general education objective we were looking at was students’ achievements in comparing and evaluating alternate solutions to a problem.

 

The prompt about the controversy in South Carolina over flying the Confederate flag above the state capital examined students’ achievements in explaining how information fits within an historical context.  Recall you were asked to write a paragraph in which you explained to an international student not familiar with it, the historical context of the South Carolina flag controversy. 

             

Prompt

Number who answered prompt

Percent who passed

Your result

Cloning conference – scored for writing

437

72%

_______

Cloning conference – scored for defining problems

437

65%

_______

Chief Illiniwek – scored for comparing and evaluating alternative solutions to a problem

422

59%

_______

South Carolina flag controversy – scored for explaining how information fits within a historical context

467

62%

_______

 

The General Education Assessment is just one small measure of students’ achievements.  Please remember that the work you did in this activity did not count in your course grade; it is not part of your transcript or record at Oakton.  We are using the results to help us evaluate our educational programs.  If you have any questions please call the Oakton Office of Research, 847-635-1894. 

 

Thank you for taking part in this activity.



[1]   “Statistical significance” simply means that the association between the independent and dependent variable is stronger than we would expect from chance alone.   Statistical significance is often attained when sample sizes are large, even when there is no practical or substantive significance.

[2]  Year 2001 subcommittee members were Gene Atkin, Trudy Bers,  Maureen  Douglas, Vicki Giambrone, Linda Korbel, Suzanne Stock and Gabe Wallace.  Diane Davis and Mary Mittler were advisors. 

 

[3]  When the project first began the college had 34 general education objectives.  As the project evolved in spring 1999, SAAT members recognized objectives were duplicative, unclear, and too tightly linked with specific disciplines.  New general education objectives were adopted for fall of 1999.  They are to develop the student as an educated person who can and will

Ø       Engage in the process of inquiry and problem solving through the following:

Ø       Work and communicate effectively with people from a variety of backgrounds, individually and in teams

Ø       Apply ethical principles to local, national, and global issues

Ø       Recognize skills necessary to build and maintain effective human relationships

 

[4] We wanted to capture students more likely to have taken general education courses; the operational method for doing so, since we could not feasibly administer the assessment to individually identified students, was to give the assessment in courses drawing primarily students with prior college credits.

 

[5]  Most variables need no further explanation; however, two warrant elaboration.  ENGLISH was constructed as a proxy to depict students’ proficiency in English composition, based on courses and placement tests at the institution.  Transcript and placement test records were examined and students were placed into mutually exclusive categories in descending order of demonstrated proficiency. That is, all students who had received a grade of A, B or C in a college-level composition course were considered to be in the highest category of composition proficiency, those who had enrolled or placed into a college-level composition course were in the next highest level, etc.  Students for whom no data were available were arbitrarily assigned to the lowest category.  The same approach was used to classify students according to their competency in mathematics (MATH), with intermediate algebra considered separately from college algebra and other college-level courses because it is not typically accepted for transfer and is a prerequisite for other college-level courses.   When we analyzed the data we collapsed some categories within each of these independent variables to minimize the incidence of cells with small numbers of observations.

 

[6]   We also collapsed the ENGLISH categories from five into three (remedial, college, no information) and MATH categories from seven into four (remedial, intermediate algebra, college-level mathematics, no information).