SACSCOC Fifth Year Interim Report
Spacer
Assessment : 2013 - 2014 : Educational Programs :
Sociology MA
Sociology MA
2 Goals 2 Objectives 3 Indicators 3 Criteria 3 Findings 1 Action
| GOAL: Sociological Research Competency |
| Objective |
|
||||
|
M.A. Sociology students will demonstrate competence in advancing sociologically informed arguments by producing a research paper that achieves the following: identification of patterns of social behavior; discernment of the three major sociological theoretical paradigms (structural functionalism, conflict, and symbolic interactionism); utilization of one or both of the two major sociological methods (positivism and interpretive). |
| Indicator |
|
|||
|
All MA sociology students enrolling prior to spring 2014 were required to write the professional paper. M.A. Sociology students writing the professional paper enrolled in SOCI 6098 Professional Paper Practicum bringing with them a self-chosen candidate paper from a previously taken class. That paper was presented to the director of graduate students for certification of eligibility for use in SOCI 6098. Student papers deemed proficient by graduate director review were then made the official object of student SOCI 6098 participation and forwarded to the SOCI 6098 professor of record for the given semester. At the end of participation in SOCI 6098, students failing to demonstrate the required level of content and writing mastery were to be assigned one of two courses of action: (1) work independently to complete the professional paper; and (2) to re-enroll subsequently in SOCI 6098. |
| Criterion |
|
|||
|
All participating MA candidates will demonstrate proficiency as defined by the associated 4-point grading rubrics. Minimum accepatable proficiency is defined by scoring an overall average of 2.0 of all graded rubric items as determined by a committee of SOCI faculty. Committee members will not have been professors of record for SOCI 6098 in the semester under evaluation. |
| Finding |
|
|||
|
Over the 2013-2014 academic year, fifteen students enrolled in SOCI 6098 Professional Paper practicum; 13 students (86.7%) completed the Professional Paper. Subsequently, the papers were evaluated double blind by a panel of two SOCI faculty (neither of whom were professors of record for the students). The results of this review were as follows (reference attached data sheets): Sheet 1 shows overall reviewer ratings of the format (F) and quality of research (P) and averages of both reviewers’ scores. Sheet 2 shows the same scores but separated into the two paper categories: literature review and research. Three of six review papers failed to meet criteria for quality. Two of the seven research papers failed to meet criteria for quality. Sheet 3 shows inter-reviewer reliability scores. Reviewer responses were separated by .14 for format and by .12 for research content. These scores justified a high confidence in reviewer reliability. Sheet 4 shows scores for two selected indicators each for methods and theory. These scores showed that six of thirteen papers did not meet criteria for methods. Only one of thirteen did not meet criteria for theory. Sheet 5 shows the same data from Sheet 4 but separated by group: research papers and literature review papers. Four of the seven research papers did not meet the minimum criteria for methods, with an overall average below the 2.0 minimum. Two of the six review papers did not meet the minimum criteria for methods, although the average for this group did exceed the minimum. For theory, only one of the thirteen papers was deficient and the averages for both groups were well above the 2.0 minimum. The faculty conclude that students are well-versed in sociological theory. The exercise revealed a focused weakness in research methods, well beyond any other issue presented by the papers. |
Actions for Objective:
| Action |
|
|||
|
MA students are required to submit a complete draft of the professional paper the semester before students anticipate graduating. Students failing to comply with this requirement are not allowed to enroll in the Professional Paper Practicum class. Per the Plan for Continuous Improvement for 2014-2015, the department will respond to these findings by implementing comprehensive exams (excepting thesis students), adding a graduate qualitative methods class (SOCI 5313), and requiring successful undergraduate SOCI methods and statistics credits as prerequisite for admission. |
| GOAL: Comprehensive Exams |
| Objective |
|
||||
|
MA SOCI students will demonstrate competence in applying sociological theory, sociological research methodologies (postivism and interpretive analysis); and research literature review of a selected, substantive sociological field. |
| Indicator |
|
|||
|
M.A. Sociology degree candidates will complete a written comprehensive exam. The specialty area will be assessed by an in-depth exploration of sociological literature on a topic of the students choosing, written answers to questions pertaining to sociological theoretical perspectives and appropriate research methodologies. In each case, an examination committee will be convened to discuss the performance of the students. A panel of at least two graduate faculty per each of the three examination areas will assess the quality of the written examination to determine student sociological mastery. Students failing to demonstrate the required level of content mastery will be allowed, upon the advice of the evaluating committee, to either revise and resubmit the portion of the exam not satisfactorily completed or retake the entire exam. A second failure will result in termination from the program. |
| Criterion |
|
|||
|
Possible examination grading outcomes are: fail, conditional pass, pass. All M.A. candidates taking the comprehensive exam will pass all components of the examination. |
| Finding |
|
|||
|
Spring of 2014 was the first offering of comprehensive exams since prior to 2011 (see Finding of Goal 1: Sociological Research Competency). In spring 2014, eight students took the written comprehensive examinations. One student passed all sections of the exam on the first submission; five students were asked to revise and resubmit various questions; two students did not pass the exam and are in the process of preparing again for the program exit requirement. |
| Indicator |
|
|||
| Select SOCI MA candidates will be allowed to substitute successful completion of a thesis project for the comprehensive exam process. |
| Criterion |
|
|||
| A successful SOCI MA thesis will be not less than 50 and not more than 100 printed pages of referenced, original SOCI research or a meta-analysis of existing research literature including primary and/or secondary data analysis. |
| Finding |
|
|||
| The thesis option was first offered in January of 2014. Through 2013-2014 no students requested this option. |
Actions for Objective:
There are no actions for this objective.
There are no actions for this objective.
| Previous Cycle's "Plan for Continuous Improvement" |
|
Based upon what we learned in AY2013, the department will finalize the evaluation rubric for SOCI 6098 Professional Paper Practicum. In addition, the Director of Graduate Studies and the Graduate Program Issues committee will revamp the assessment tool for incoming graduate students.
|
| Update on Previous Cycle's "Plan for Continuous Improvement" |
|
Three rubrics were finalized for the SOCI6098 Professional Paper assessment (see attachments under Professional Paper - Finding above). The three rubrics are: Paper Format; Research Paper; and Review Paper. The Director of Graduate Studies and Graduate Committee conducted a survey of students enrolled in the SOCI MA Program to identify the level of knowledge regarding sociological theory, sociology research methods, and social statistics. Based on information gathered in the survey, the department decided to require undergraduate research methods and statstics as pre-requisites for admission to the program.
|
| Plan for Continuous Improvement |
|
Having now run the SOCI MA program in its present form since fall 2011, the SOCI chair, graduate director and graduate committee will begin preparations for a comprehensive program performance review slated for 2016-17. As a start to this process, the SOCI faculty will review student recruitment, entry credentials, and outcomes in relationship to each other.
For the academic year 2014-2015, the department will begin a phase-out of the professional paper project in favor of having all non-thesis students exit the program via comprehensive examination. While it is true that SOCI faculty successfully led students in necessary skill acquisition, after participation for 4 semesters in the professional paper project faculty became concerned that most students would be better served by breadth of knowledge than over-specialization. That minority of students seeking focused research experience will still be able to apply for the thesis track and/or prepare for doctoral SOCI studies. As reported above, the comprehensive exam process was piloted in spring of 2014. The department will build on that process by creating comps performance rubrics (in each of the three dimensions: theory, methods, and literature review/substantive area) to assist students in preparing for exams. Faculty will also receive feedback from the comps process to assist them in modifying curricular content and pedagogy in further aid to students. In addition, with the reinstution of the thesis track option the department will create and implement a thesis rubric specifically designed to detect thematic or recurrent weaknesses in student research capacities. The department will subsequently make continuous amendments to pedagogy and curriculum to address detected weaknesses. |