SACSCOC Fifth Year Interim Report
Spacer
Assessment : 2013 - 2014 : Educational Programs :
English BA
English BA
2 Goals 3 Objectives 3 Indicators 3 Criteria 4 Findings 3 Actions
| GOAL: Literature And Literary Theory (4000-level) |
| Objective |
|
||||
| Students will be able to use various approaches and methodologies presented in analyzing literary texts and demonstrate the ability to interpret texts by communicating their understanding of those texts in analytic essays. |
| Indicator |
|
|||
|
Reading and writing are part and parcel of each other. Essays written to analyze and/or apply literary texts suggest the depth and quality of the students' reading, as well as their understanding of the assignment. Thus, during spring semester, we will collect writing samples of English majors from 4000-level (senior-level) classes and examine them to ascertain the effectiveness of reading that they evince. Our goal is to read 25 percent of the essays, chosen at random, written by English majors in 4000-level literature courses. We anticipate an enrollment of some 105 students in any given long semester and so should expect to read 26 to 30 essays. Two experienced English professors agree that 70 percent of the students write at college level. College-level writing is defined as fluent, coherent, nearly error-free writing. For the purpose of evaluation, clear criteria were developed (see attachment). |
| Criterion |
|
|||
|
The chosen essays will be assessed by a holistic scoring done by Department of English faculty. A score equal to or greater than 5 will be deemed acceptable. One weakness evident last year was that we did not receive enough essays for the results to be meaningful and reliable. (We received only fifteen essays.) We will rectify that weakness this year. Seventy percent of the sample of collected 4000-level essays satisfies the requirements of mature academic BA-level writing as assessed holistically by two scoring professors. Students write fluent, coherent, and nearly error-free analytical essays which show sophistication in literary analysis that goes beyond mere superficial plot summaries, and their essays have a point (see attached evaluation criteria). We were concerned last year with whether our process was reliable. We are taking steps to ensure reliability of the process. |
| Finding |
|
|||
|
During Spring 2014, 125 students were enrolled in 4000-level English classes. An end-of-semester writing sample (essay or research paper) was collected from every third or fourth student (depending on class size) on the roster of each 4000-level class. Thirty-four (34) papers were submitted for the holistic evaluation by eleven professors of English; hence, our goal of collecting more papers than in the spring of 2013 was met. The reading was carried out collectively on May 7, 2014. Each essay was read by two professors, and essays received scores on the scale of 2-8. Interrater reliability was high: only two essays had to be read by a third reader. The results follow: Score N 2 0 3 1 4 7 5 9 6 9 7 5 8 3 ____ 34 Twenty-four (24) percent (N=8) of essays (N=34) were assessed as failing; they received the combined score of 4 or under. Seventy-six (76) percent (N=26) of the essays (N=34) passed with a score of 5 or higher. Our goal of seventy percent was exceeded. |
Actions for Objective:
| Action |
|
|||
| In all 4000-level writing-enhanced classes, the importance of writing academic research papers will be emphasized. All professors will require a major research paper in their classes. |
| GOAL: Gaining Knowledge In World And Multicultural Literature (2000-level) |
| Objective |
|
||||
|
Students will demonstrate understanding of basic literary terms and a basic knowledge of important writers. |
| Indicator |
|
|||
|
During Fall 2011, an objective test for the core English sophomore course was developed by professors who teach World Literature II (ENGL 2342). Seventy percent of sophomores are expected to pass the posttest with 70 percent correct answers. Comparing the pretest, given in the beginning of the semester, to the posttest in May will indicate whether any learning took place or not. |
| Criterion |
|
|||
|
Seventy percent of the sample of 2000-level posttest results will show that students have necessary rudimentary knowledge (score of 70 percent or higher) of literature after having taken a sophomore World Literature course. This basic knowledge is necessary before continuing to junior- and senior-level English classes. |
| Finding |
|
|||
| Even though the idea of measuring students' knowledge of literary terms and major writers was worth exploring, it turned out that implementing the objective test across twenty-three (23) sections of ENGL 2342 (World Literature II)--taught by eight (8) different professors during AY 2013-2014--provided several logistic problems. Until the objective test is adopted more widely across sophomore instructors, the Department of English will abandon the idea of measuring knowledge of literary terms and facts of major authors objectively. |
| Finding |
|
|||
| 234 students of World Literature II were given a pretest of literary terms in the beginning of spring semester 2012. The same test was presented to 190 students in the spring to measure any possible improvement. The major finding was the attrition of 44 students (19 percent of the initial student population). For the students who remained in the class, the finding was obvious: the number of those scoring low went down (from 46% to 35%) and those scoring relatively high went up (from 5% to 16%). The number of those scoring highest (90-100% correct answers) remained at one percent from January to May. |
Actions for Objective:
| Action |
|
|||
| An objective test to measure students' knowledge of literary terms and major authors was developed two years ago. For four semesters, the Department of English tried to implement this testing in World Literature II classes, with little success. Using a test would constrain the vast area of literature that can be covered in this class (any major world and multicultural piece of literature after 1700) and unnecessarily unify, restrict, and centralize the variation that students can now enjoy. This does not mean that central concepts are not taught; controlling that they be taught via an objective test would, however, be too simplistic. This seemed to be the consensus of the sophomore professors. The test is still available for all those who wish to use it. |
| GOAL: Gaining Knowledge In World And Multicultural Literature (2000-level) |
| Objective |
|
||||
| 2000-level students are able to write about literature. |
| Indicator |
|
|||
|
Two English professors assess 70 percent of ENGL 2331 essays as acceptable. This assessment is done through blind review. Acceptable is defined as a score 5 on an scale 2-8. See the attached grading criteria. |
| Criterion |
|
|||
| Only 30 percent or fewer of the essays are classified as not fulfilling requirements for acceptable academic writing. |
| Finding |
|
|||
|
Previous years, a comparison between English sophomore and senior writing has been carried out, with the purpose of evaluating whether senior-level writing is better than sophomore-level writing, i.e., whether students have learned to write better while in the English program. We found that, indeed, 66 percent of English seniors write at passing level (as determined via our department-internal criteria), while only 50 percent of sophomores do. For the academic year 2012-13, we focused on evaluating our sophomore writers, of whom only 50 percent had passed our departmental criteria during the previous evaluation period (AY 2011-12), i.e, two years ago. We found that only within one year, a jump from 50-percent acceptance rate to a 62-percent acceptance rate had taken place. We attributed this to consciousness-raising of what sophomore writing should consist of. Also, the new sophomore curriculum could be credited for higher writing ability. In order to see whether this trend has continued or not, we carried out a similar assessment in May 2014. Table 1 shows the numbers of the student population in both of our sophomore courses, ENGL 2332 (World Literature I) and ENGL 2342 (World Literature II), during Spring 2014. Both of these courses satisfy the requirements of Core Component (Language, Philosophy, and Culture). Table 1 also shows the numbers of the collected sample essays: Sections Students Essays Evaluated 2332 (World Lit I) 10 365 22 (6%) 2342 (World Lit II) 11 202 66 (33%) Total 21 567 88 (15.5%) Table 1. Number of students in ENGL 2332 and 2342 sections and sample population. Spring 2014. As during the past years, in the spring of 2014, an electronic mail was sent to all English faculty teaching sophomore courses, with instructions to collect mechanically the last substantial essay assignment from every third student for evaluation purposes. The four large non-writing-enhanced World Literature I sections were not targeted, which explains the smaller percentage in Table 1 (above) of essays submitted from ENGL 2332 students (6%). The sample of essays from ENGL 2342 sections was 33%. The entire sophomore sample size consisted of 88 essays (15.5% of the entire sophomore population of 567 registered students). The sample consisted of more essays than last year (an increase from 57 to 88 essays), but the percentage was smaller (a decrease from 20% to 15.5%). We still consider this percentage as representative of sophomore writing. The reason for the change was the larger number of students enrolled in the two English sophomore classes during AY 2013-2014. On May 7, 2014, a volunteer group of eleven professors gathered to evaluate the essays. Instructions to evaluators are attached. See Table 2 (Comparison between scoring results in 2012, 2013, and 2014) for scoring results from 2012 to 2014 (attachment). Interreader reliability was very high; only three essays required a third reader (tie-breaker). Discussion: From the year 2013, the difference in 2014 between failing and passing essays (in terms of the established departmental criteria; see attachment) is smaller (48% fail; 52% pass). In fact, it is almost back to the level of the even split of 2012. There continues to be a discrepancy in the scores between the two sophomore courses, ENGL 2332 (World Literature I: Before the Seventeenth Century) and ENGL 2342 (World Literature II: Seventeenth Century and After) (see attachment of Table 2, Comparison of writing scores bewteen ENGL 2332 and ENGL 2342 students). It is interesting to note that while 68 percent of World Literature I (ENGL 2332) students write in a passable manner (an improvement of last year’s 62 percent), only 47 percent of World Literature II (ENGL 2342) students do so (down from the 53 percent last year). Last years trend has not only been repeated; it has become stronger as the gap has grown. We will continue to follow up, and will explore reasons for this discrepancy. The goal of only thirty (30) percent or fewer of our sophomore students not passing the departmental writing criteria was not met. Thirty-two (32) percent of World Literature I students failed to meet these criteria, and over half (53 percent) of World Literature II students failed. |
Actions for Objective:
| Action |
|
|||
| As student writing is not at the recommended level, students must be informed early on about the English Department's high writing standards. Grade inflation is to be avoided, and professors will be encouraged to work with struggling writers. |
| Previous Cycle's "Plan for Continuous Improvement" |
|
Next year or the following year we will compare the results of ENGL 2331 and ENGL 2342 again, to see if we are looking at a trend or whether this year's discrepancy in results was a coincidence. All professors will be made aware of the results, and they will emphasize overall excellence to all students. It may be that students self-select to these two core courses, more diligent ones choosing ENGL 2331 (World Literature I: Before Seventeenth Century). This means that ENGL 2342 must be made aware that the expectation are the same for these two courses, despite the difference in the chronology of reading materials.
|
| Update on Previous Cycle's "Plan for Continuous Improvement" |
|
Nota bene: In the previous section (Previous Cycle's "Plan for Continuous Improvement," third line from the bottom, "expectation" should read "expectations."
Last year's discrepancy between the writing skills of World Literature I and World Literature II students was partially a coincidence (a result of a smallish sample); however, the same trend continues: World Literature I students, on an average, write better than World Literature II students. In May 2014, a discussion was held during the grading session to explore reasons for the discrepancy. The faculty consensus was that both courses must be as demanding. |
| Plan for Continuous Improvement |
- Two years ago, a rule was implemented that English majors and minors must receive a C or better in all courses that count toward their English major or minor. We anticipate the effects of this rule to start showing up soon as more reflective writing. It has been too early to measure this until now. - The English Department and MFA program will publicize the monetary award that we have established for students who write an essay about the Bearkat common reader. - Good writers will be nominated by faculty for the Academic Success Center writing awards in the spring of 2015. - All 4000-level professors will share with their students the departmental criteria (attached) that are used to evaluate senior-level writing.
- All 2000-level instructors will share with their students the departmental criteria (attached) for sophomore writing. - Students will be encouraged to submit their essays for the Bearkat common reader contest for the monetary award established by the department for the coming year. - We regard sophomore courses as our recruitment courses. Active recruitment efforts will commence. Every sophomore class will be visited by a recruiting professor. The side effect should be better consciousness of what English values: plenty of critical reading and reflection on the reading through writing.
- See above. In addition: - ENGL 2333 students will be presented, again, the departmental list of literary terms. This, we expect, will provide the 2333 students the tools to discuss their readings in reflective writing. - Visits to Newton Gresham Library and the Academic Success Center will be encouraged. Struggling writers will be sent to the Academic Success Center to get additional help in developing better sophomore-level essays early during the semester.
- All sophomores will be given the departmental essay criteria, and these will be discussed in detail during the beginning of the semester. - A meeting will be organized with all sophomore professors to discuss means of raising the level of students writing. The perils of grade inflation will be addressed.
|