SACSCOC Fifth Year Interim Report
Spacer
Assessment : 2013 - 2014 : Educational Programs :
Criminal Justice BA/BS
Criminal Justice BA/BS
2 Goals 2 Objectives 2 Indicators 2 Criteria 2 Findings 2 Actions
| GOAL: Specific Knowledge Acquired By All Undergraduates |
| Objective |
|
||||
|
Graduating seniors will exhibit comprehensive knowledge of core criminal justice course work. |
| Indicator |
|
|||
|
Each semester, all graduating seniors are asked to complete an exit exam, through Survey Monkey, an online data collection/response validation website. This survey provides faculty a means of assessing students' comprehension and retention on key concepts provided to them during their educational experience. The 60-item Exit Exam, created by faculty, assesses learning in seven core areas. |
| Criterion |
|
|||
|
The average on each competency area: (1) Criminal Law; (2) Corrections; (3) Law Enforcement; (4) Courts; (5) Theory/Criminology (6) Research Methods/Statistics, and (7) Diversity within the Criminal Justice System will be at least 75% for graduating seniors taking the exit exam, except for the average on Theory/Criminology. Last year this average was below 70%. We hope to see this average rise this year. |
| Finding |
|
|||
|
A senior exit knowledge survey was conducted with the 2013-2014 graduates using Survey Monkey in an online capacity. Last year, 43 percent of graduates (191 of 442 students) responded to the exit survey; this year, 40 percent of graduates (144 of 362 students) responded. Although we aimed to increase our response rate this year, we failed to do so. Our criterion for this objective was a 70% average score across seven content areas. These areas and average student achievement was: Criminal Law – 67%; Law Enforcement – 66%; Courts – 67%; Corrections = 61%; Theory – 46%; Methods – 59%; Diversity – 62%. With dropping the lowest scoring question from each area achievement was: Criminal Law – 73%; Law Enforcement – 70%; Courts – 74%; Corrections = 65%; Theory – 51%; Methods – 66%; Diversity – 65%. |
Actions for Objective:
| Action |
|
|||
| Our undergraduate program committee will strategize this year to determine a more effective approach in garnering a greater than 50 percent response rate. Since our failure to achieve our stated benchmark in four of the seven areas (accounting for outlier questions), it is evident that a two pronged approach is needed: [1] Increased participation rates from our undergraduate students to an acceptable rate. This will increase the validity of our findings; and [2] a review of current course content and exit exam questions to ensure that question accurately reflect the most recent material. A second consideration for next year’s assessment is also needed: It is increasingly apparent that the exit survey targets courses offered at the 200 level. A reasonably large proportion of our students complete these courses at the community college level. Thus, we may not be measuring knowledge transfer occurring at SHSU. At minimum, we need to include questions on the exit survey that query whether the courses were completed within our Department or credit was transferred. It may also be more reasonable to include questions on the exit survey that target 300 and 400 level courses thereby ensuring a more accurate assessment of our program. |
| GOAL: Development Of Writing And Research Skills |
| Objective |
|
||||
| Students will demonstrate competence in their writing and research skills. |
| Indicator |
|
|||
|
Undergraduate criminal justice students will develop a scholarly research paper proposal in their core required Research Methods course (CRIJ 3378). Writing and research skills will be scored by a rubric agreed upon by a faculty committee. Each professor will utilize this rubric when grading the student research paper proposals. These guidelines include areas of organization, content, quality of proposed research, hypothesis construction and writing. |
| Criterion |
|
|||
| The majority of students will score at 75% or above on the research paper proposal scoring rubric. |
| Finding |
|
|||
|
A total of 115 students from four different sections of CRIJ 3378 Research Methods had their writing ability assessed by the common scoring rubric attached here. Student scores, which ranged from 1 (Poor) to 5 (Excellent), were averaged by each of the nine domains measured by the rubric within each instructor’s section. Further, the domain scores were averaged to determine the overall score achieved by this subset of Research Methods students on the reading and writing research paper review. The overall average score across sections was 3.56 of 5, or 71.2 percent, which was slightly below our previously stated goal of 75 percent or above achievement levels. Regarding the average scores across the nine domains, the “relevance to the field” and “whether the importance of the study is made obvious” of the paper garnered the highest scores. The seven remaining domains received relatively similar scores with the “contribution to the academic debate” and “appropriateness of the research/study method” domains having slightly lower scores than the rest. Specifically, the following averages were found for each domain in the rubric: Relevance to the field (4.64), contribution to the academic debate (3.46), structure/organization of the paper (3.52), writing and grammar (3.55), appropriateness of the research/study method (3.44), abstract information (3.50), construction of hypothesis (3.60), whether the importance of the study is made obvious (4.12), and adequately and correctly cited references (3.51). In comparison to findings from the prior academic assessment cycle, the domains related to broader, contextual skills such as selecting relevant topics and emphasizing the importance of the study improved; whereas, structural components of writing did not improve. |
Actions for Objective:
| Action |
|
|||
|
Similar to our prior assessment cycle, we will continue to encourage the assignment of writing projects across our criminal justice and criminology courses including our lower division classes. This approach will enable student growth throughout the academic careers. We will also consider whether our assessment efforts regarding writing should focus only on our research methods course. It is possible that many of our students in CRIJ 3378 are recent transfers into our 4 year program and have had only limited exposure to SHSU instruction. Perhaps another course will be more suitable for assessment. We will also provide specific feedback to all of our faculty and instructors regarding the strengths and weaknesses displayed by our students in their writing so that broad based efforts can be made by all faculty members to develop our student’s writing ability. |
| Previous Cycle's "Plan for Continuous Improvement" |
|
The College of Criminal Justice will continue to assess our student's comprehensive knowledge of the discipline. We will also assess our exit exam to determine appropriate items and improve the completion rate of the exam.
We will also continue to assess and support our undergraduates in the development of research and writing skills. |
| Update on Previous Cycle's "Plan for Continuous Improvement" |
|
All elements from the 2012-2013 cycle were implemented this past year. We plan to maintain these two primary assessment approaches to allow for the development of baseline student data for our program, especially as other organizational aspects of the department are established.
|
| Plan for Continuous Improvement |
|
As indicated in detail in our findings on comprehensive knowledge, this assessment cycle our Departmental faculty will need to reexamine the undergraduate curriculum and its alignment with assessment protocol. We have been utilizing the same survey questions for a number of years and must update both our process of assessment and likely the content of our assessment materials. Regarding the assessment of writing and research skills, our primary goal will be to capture a greater number of research methods sections taught by our faculty to more fully assess student ability. These steps will provide us better data upon which we can improve our undergraduate program performance.
Certainly a first step is to determine a better approach to assessment that will include a large subset of our undergraduate students. The current sample size is small by comparison and limited to four sections of a course that used a common assessment rubric. Additional faculty “buy in” will be needed to achieve this goal. The Department Chair will accept the responsibility for strongly encouraging faculty utilize this common rubric (or a redesigned version of the rubric) to grade a research paper. Further, faculty will provide consideration if perhaps assessing papers in senior level writing enhanced classes will increase the validity of our efforts. |