SACSCOC Fifth Year Interim Report
Spacer
Assessment : 2011 - 2012 : Educational Programs :
English BA
English BA
3 Goals 4 Objectives 4 Indicators 4 Criteria 4 Findings 4 Actions
| GOAL: Literature And Literary Theory (4000-level) |
| Objective |
|
||||
| Students will be able to use various approaches and methodologies presented in analyzing literary texts and demonstrate the ability to interpret texts by communicating their understanding of those texts in analytic essays. |
| Indicator |
|
|||
|
Reading and writing are part and parcel of each other. Essays written to analyze and/or apply literary texts suggest the depth and quality of the students' reading, as well as their understanding of the assignment. Thus, we will collect writing samples of English majors from various 4000-level (senior-level) classes and examine them to ascertain the effectiveness of reading that they evince. Our goal is to read 25 percent of the essays, chosen at random, written by English majors in 4000-level literature courses. We anticipate an enrollment of some 105 students in any given long semester and so should expect to read 26 to 30 essays. Two experienced English professors agree that 70 percent of the students write at college level. College-level writing is defined as fluent, coherent, nearly error-free writing. For the purpose of evaluation, a rubric (see below) was developed. |
| Criterion |
|
|||
|
The chosen essays will be assessed by a primary trait scoring done by Department of English faculty. The traits to be assessed will include plot summary vs. analysis and effective use of secondary sources. A score equal to or greater than 5 will be deemed acceptable. One weakness evident last year was that we did not receive enough essays for the results to be meaningful and reliable. We will rectify that weakness this year. Seventy percent of the sample of collected 4000-level essays satisfies the requirements of mature academic BA-level writing as assessed holistically by two scoring professors. Students write fluent, coherent, and nearly error-free analytical essays which show sophistication in literary analysis that goes beyond mere superficial plot summaries, and their essays have a point. We were concerned last year with whether our process was reliable. We are taking steps to ensure reliability of the process. |
| Finding |
|
|||
|
During Spring 2012, 204 students were enrolled in English 4000-level classes. Using a random sampling method (an essay collected from every fifth student on the rosters), thirty-eight (38) essays were collected by instructors. This sampling constitutes 19 percent of the reference population. It is a bit lower than the anticipated 25 percent but still representative. On May 9, 2012, nine (9) full-time English faculty volunteered for four hours to read and evaluate the essays. Each essay was read by two professors, and evaluation was carried out based on a rubric developed for this particular occasion. A score of 4 or lower meant that the essay did not meet the requirements of BA-level English major's writing; a score of 5 or higher was acceptable. The range of possible scores was 2-8. The results are the following: Score # of students 2 8% ( 3) 3 13% ( 5) 4 13% ( 5) 5 18% ( 7) 6 29% (11) 7 11% ( 4) 8 8% ( 3) Total 100% (38) The scores for thirteen (13) students were lower than acceptable; twenty-five (25) students had acceptable scores. This means that 66 percent of students are writing at the level that English professors deem acceptable BA-level English-major writing. The inter-rater reliability was very high; only 5 of the 38 essays had to be read by a third reader. The group pondered possible reasons for the fact that 34 percent of the students were deemed not to be writing at an acceptable level. One reason could have been disparate writing tasks; some essays had been written in class, where the students did not have time to develop the paper through drafting and revision. Often the thesis was not sophisticated, the students had clear difficulty in developing the thesis, and sentence problems were massive. This applies to those 34 percent of the papers that received the added score of 4 or lower from two readers. |
Actions for Objective:
| Action |
|
|||
|
It was noticed that disparate assignments may have caused problems for comparison. Some essays were developed at home and were longer; some were shorter essays, written in class as drafts. Next year, we will ask for similar assignments. Another issue that was noticed was the importance of clear instructions. If the instructions were detailed (yet not too verbose), the students were addressing the task more accurately. The faculty will be informed about the fact that not all 4000-level students perform yet at an acceptable level; they fail to write at college level. All professors teaching senior-level classes will be asked to pay special attention to student writing. In the fall of 2012, professors will be given the rubric that the volunteers used, and they will be asked to share this rubric with their students in the senior classes. |
| GOAL: Gaining Knowledge In World And Multicultural Literature (2000-level) |
| Objective |
|
||||
|
Students will demonstrate understanding of basic literary terms and a basic knowledge of important writers. |
| Indicator |
|
|||
|
During Fall 2011, an objective test for the core English sophomore course was developed by professors who teach World Literature II (ENGL 2342). Seventy percent of sophomores are expected to pass the posttest with 70 percent correct answers. Comparing the pretest, given in the beginning of the semester, to the posttest in May will indicate whether any learning took place or not. |
| Criterion |
|
|||
|
Seventy percent of the sample of 2000-level posttest results will show that students have necessary rudimentary knowledge (score of 70 percent or higher) of literature after having taken a sophomore World Literature course. This basic knowledge is necessary before continuing to junior- and senior-level English classes. |
| Finding |
|
|||
|
An objective test consisting of forty-two multiple-choice questions was developed by a committee of English faculty regularly teaching World Literature II during the fall semester of 2011. The test was administered in all the ENGL 2342 classes in January (pretest). The same test was administered in May after the course had been taught (posttest). 234 students took the pretest in January 2012; 190 students took the posttest. The attrition rate was 18.8 percent. The average score on the test in January was 58/100. For May, the rate had gone up to 63/100. In other words, learning had taken place. The following table indicates the distribution of scores from pretest to posttest: Table 1. Distribution of scores from pre- to posttest. PRETEST POSTTEST N % N % 90-100% 2 ( 1) 2 ( 1) 80-89% 11 ( 5) 30 (16) 70-79% 47 (20) 45 (24) 60-69% 66 (28) 46 (24) 0-59% 108 (46) 67 (35) TOTAL STUDENTS 234 (100) 190 (100) Students scoring 70 percent and higher constituted 26 percent of all students in the pretest. In the posttest, students scoring 70 percent or higher constituted 41 percent. The percentage of students scoring under 70 percent in this objective test had gone from 74 percent in the pretest to 59 percent in the posttest. This means that either, while learning had taken place, it had not been sufficient, or that the students come into the sophomore class with such a low level of knowledge that one semester is not enough to bring them all to the level of our goal. |
Actions for Objective:
| Action |
|
|||
| In the fall of 2012, the results of the objective pre- and posttests will be disseminated to the faculty who teach sophomore courses, and a meeting will be held to discuss the reasons for the somewhat low performance by sophomores and to brainstorm for remedies. In a core course, it is to be expected that the result cannot be stellar, but knowing about the student performance across the board will help faculty to focus on teaching that terminology and those concepts that help students evaluate literary texts, analyze them, and build synthesis. |
| GOAL: Gaining Knowledge In World And Multicultural Literature (2000-level) |
| Objective |
|
||||
| 2000-level students are able to write about literature. |
| Indicator |
|
|||
| Two English professors assess 70 percent of ENGL 2331 essays as acceptable. Acceptable is defined as a score 5 on an scale 2-8. |
| Criterion |
|
|||
| Only 30 percent or fewer of the essays are classified as not fulfilling requirements for acceptable academic writing. |
| Finding |
|
|||
|
341 students were enrolled in ENGL 2331 (World Literature I). 42 essays (12.3%) from 10 sections were collected randomly. Nine experienced professors evaluated the essays to see whether they met the preset criteria (a rubric was developed specifically for this occasion; see attachment). The following table shows the distribution of holistic scores: Score Percentage of essays falling in this category 2 19% ( 8) 3 14% ( 6) 4 17% ( 7) 5 21% ( 9) 6 17% ( 7) 7 7% ( 3) 8 5% ( 2) Total 100& (42) Fifty percent (21) of the 42 essays met the criterion of competent and acceptable academic writing at the sophomore level. |
Actions for Objective:
| Action |
|
|||
|
It was noticed that disparate assignments may have caused problems for comparision. Some essays were developed at home and were longer; some were shorter essays, written in class as drafts. Next year, we will ask for similar assignments. |
| GOAL: Secondary English Education Certification |
| Objective |
|
||||
| Students seeking teacher certification will demonstrate knowledge and skills to teach English to secondary students. |
| Indicator |
|
|||
| Secondary English Education students will be prepared to pass the TExES English content area exam in their final semester or shortly after graduating. The State Board for Educator Certification (SBEC) developed standards for Texas educators that delineate what the beginning educator should know and be able to do. These standards, which are based on the state-required curriculum for students, the Texas Essential Knowledge and Skills (TEKS), form the basis for the Texas Examinations of Educator Standards (TExES). The TExES test is a criterion-referenced examination designed to measure the knowledge and skills required in English language and literature teaching. A score of 240 is the minimum level of competency required over all of the domains. A student may fail a domain but pass the test. The following are the areas tested: Domain I: Integrated Language Arts, Diverse Learners, and the Study of English; Domain II: Literature, Reading Processes, and Skills for Reading Literary and Nonliterary Texts; Domain III: Written Communication; Domain IV: Oral Communication and Media Literacy. |
| Criterion |
|
|||
| At least 75 percent of students taking the TExES English content area exam will obtain passing scores in each domain. Although last year, 100 percent of students who took the test passed, two areas emerged as weaknesses, II. Literature, Reading Processes, and Skills for Reading Literary and Nonliterary Texts; and III. Written Communication. We are anxious to determine if our interventions have been successful in raising these particular scores. |
| Finding |
|
|||
|
Eighteen English majors seeking secondary English Language Arts and Reading (8-12 ELAR) certification took the TExES certification exam during the assessment period. (Since one of these students failed the test and then retook it, and passed, there were 19 testing instances.) Seventeen of the 19 instances resulted in passing scores, for an overall passing rate of 88.9, exceeding the objective of 75%. Seventeen of eighteen individual students passed the exam during the assessment period, for an effective passing rate of 94.7. Passing rates for the individual domains were as follows: Domain I: 57.0% Domain II: 78.9% Domain III: 94.7% Domain IV: 89.5% Last year’s identified weaknesses were in Domain II (Literature, Reading Processes, and Skills for Reading Literary and Nonliterary Texts) and Domain III (Written Communication). While our students’ overall results in these domains brought us into successful territory, their scores fell off alarmingly in Domain I (Integrated Language Arts, Diverse Learners, and the Study of English). While Domain I is weighted less heavily than Domains II and III (and is therefore “less important”), this result demonstrates our need to focus attention on the concepts of integrating the language arts, diverse learners, and the study of English. |
Actions for Objective:
| Action |
|
|||
|
While our overall passing rates on the 8-12 ELAR TExES are exceptional, and while we made the desired gains on Domain II and Domain III scores, we clearly need to turn equal attention to the domain we seem to have neglected. Domain I spans much of our curriculum. Concepts of language arts integration are presented primarily in our two methods courses—English 4364 (Methods of Teaching Secondary English) and newly-revised English 3375 (Teaching Writing and Literature). Diverse learner issues are encountered primarily in the methods course, while the study of English is presented in our required grammar and linguistics courses (as well as the methods course). We will call this deficiency to the attention of those specific instructors and will formulate specific strategies for increasing our scores in Domain I for the coming year, while continuing our successful efforts in the other domains (and in overall passing rates). |
| Closing the Loop |
|
During AY 2011-12, the department restructured its freshman and sophomore courses. Composition I was defined clearly as an introductory writing course; Composition II was restructured to include elements from ENG 266, i.e., elements of writing about literature were added to persuasive and argumentative writing. Sophomore courses were redefined chronologically (2331 as World Literature before 17th century; 2342 as World Literature after 17th century) in order to avoid overlaps. Evaluation of these courses has been based on a combination of objective tests and assessing essays. Objective evaluation and essay assessment both reveal that much work remains to be done to raise the standards. A realistic approach includes admitting that our students need the three core writing courses in order to internalize the basics of academic writing.
Last year, the plan was to reevaluate all junior survey courses. The faculty met and came to the conclusion that everyone is satisfied with how the junior-level surveys are structured. American Studies program was launched, and English taught the first "Introduction to American Studies" course. American Studies minor was thus added to the other English minors (secondary education, creative writing, and technical and professional writing). Chances are that departmental objective testing in Composition I and World Literature II will help to raise the consciousness of shared standards across heterogeneous classes. It will be interesting to see whether this consciousness leads to higher achievements or not. What is encouraging is the finding revealed in the comparison of the 2000- and 4000-level essay assessments. Above tables are combined here for comparison: 2000-level 4000-level results results 2 19% 8% 3 14% 13% 4 17% 13% 5 21% 18% 6 17% 29% 7 7% 11% 8 5% 8% Tot.100% 100% Even though only 50 percent of the 2000-level essays were deemed as acceptable (score 5 or better), during the same rating session, the same professors rated 66 percent of the 4000-level essays as fulfilling academic standards of that level of writing. This is most encouraging, but it would be desirable to boost up the numbers of academically acceptable essays at both levels. Learning happens from the sophomore level to senior level, and faculty and students becoming more conscious about standards should help in our quest for excellence. Regarding the certification standards and exam, we will call this deficiency to the attention of those specific instructors and will formulate specific strategies for increasing our scores in Domain I for the coming year, while continuing our successful efforts in the other domains (and in overall passing rates). |