SACSCOC Fifth Year Interim Report
Spacer
Assessment : 2011 - 2012 : Educational Programs :
Computing Science BS
Computing Science BS
2 Goals 2 Objectives 2 Indicators 2 Criteria 2 Findings 2 Actions
| GOAL: Specialized Competencies |
| Objective |
|
||||
| Students will develop and demonstrate skills and knowledge in their concentration areas. The department offers three concentration areas: Computer Science, Information Systems and Information Assurance and Security. |
| Indicator |
|
|||
|
During department meetings faculty with expertise in the three concentration areas will discuss students’ performances in their concentration areas. We expect that faculty in each concentration area will evaluate student performance across the 18 hours course sequence for each concentration. |
| Criterion |
|
|||
|
In last year's review the faculty assessed more than 70% of the students were performing at or above expected levels. As a result the faculty set a higher expectation. We expect that the faculty will deem at least 72% or more of students’ performances as acceptable. |
| Finding |
|
|||
|
For the formal assessment and evaluation of the performance of CS students, CS faculty developed tools and rubrics that can address the new ABET-CAC (Computing Accreditation Commission) criteria. First, four Program Education Objectives (E01-E04) and eleven Student Outcomes ((a)-(k)) for the Computer Science (CS) department at SHSU were established. Program Education Objectives are broad statements that describe what graduates are expected to attain within a few years of graduation, while Student Outcomes describe what students are expected to know and be able to do by the time of graduation. Secondly, CS core courses are mapped to the Student Outcomes to find the minimal number of courses for which self-study can be performed. Thirdly, member of the CS Undergraduate Curriculum Committee (UCC) arranged teams to prepare survey forms and assessment questionnaires for self-study that can be directly applicable for the new ABET-CAC assessment. During Fall 2011, CS faculty prepared the following assessment tools: (1) ABET Objectives Survey form, (2) ABET Outcomes Survey form, (3) Exit Interview form, (4) Exit Survey form, (5) Alumni Survey form, and (6) Employer Survey form. Furthermore, questions in each survey are mapped to the corresponding ABET objectives and ABET outcomes. |
Actions for Objective:
| Action |
|
|||
|
CS determines to use formal assessment tools that can measure students’ performance in terms of the new ABET criteria. The tools (Survey forms) are developed and the assessment rubrics for the two courses (COSC4319 and COSC4349) are prepared as the self-study tools. The initial rubrics were applied at the end of Spring 2012 and the revised rubrics and tools for the two courses were used for Fall 2012. In order to provide accurate assessment and evaluation for students’ performance, the assessment results should be collected over multiple semesters and compared. In addition, all the faculty members will share the rubrics and the collected assessment results and improve the CS curriculum/program accordingly |
| GOAL: Ethical Principles, Technical Skills, And Management Skills (core) |
| Objective |
|
||||
| Students will develop and demonstrate knowledge of ethical principles, technical skills, and management skills relevant to the field of computer science. |
| Indicator |
|
|||
| All undergraduate students must complete CS 437 prior to graduation. Part of the course requirements for CS 437 is to prepare for and take the ICCP Examination. The Institute Certified Computer Professional exam is an exam adopted by the Institute for Certification of Computer Professional (ICCP), the national association in the field of computer science. The exam consists of three sections: ethical principles, technical skills, and management skills. The grading of the three sections is objective grading (i.e., there is no written component to the exam) |
| Criterion |
|
|||
| The associate level scoring on the exam is between 70% and 85% and professional level scoring is between 86% and 100%. We expect that 70% of graduating seniors taking the exam will pass at or above the associate level. |
| Finding |
|
|||
|
After deliberation the department determined that the ICCP exam was a poor tool for assessment and this indicator will be dropped in future years. All undergraduate CS students must complete COSC 4319 (Software Engineering) and COSC4349 (Professionalism and Ethics) prior to graduation. In addition, COSC4319, as a capstone course, covers the eight Student Outcomes (except the three outcomes (e), (g) and (h)) out of the eleven Student Outcomes and COSC4349 addresses the remaining three outcomes. Therefore, both the courses are selected for assessment. To perform a quantitative assessment, the following rubrics for the two courses are developed: (1) rubric for Project, (2) rubric for presentation evaluation, (3) rubric for group member evaluation, and (4) rubric for ethics topics. For COSC4319, faculty members observe students’ project presentation and directly evaluated students’ performance based upon the rubrics (1)-(3). One the other hand, for COSC4349, students’ performance is indirectly evaluated through the exam questions that address rubric (4). Each category is rated with the following scale values: (1) 1.0 (below expectations or unacceptable), (2) 2.0 (evolving or developing), (3) proficient (or competent), and (4) outstanding (or exemplary). Student’s performance on Student Outcomes is directly evaluated with each specific rubric, while the performance on objectives is indirectly measured by mapping between Student Outcomes and Program Objectives. After the course mapping to the eleven Student Outcomes criteria, CS faculty find the minimum number of courses that will address all the eleven Student Outcomes. As a result, COSC4319 (Software Engineering) and COSC4349 (Professionalism and Ethics) are selected. The assessment result (out of the maximum score of 4) of Program Education Objectives ranges from 2.29 (with std of 1.11) to 3.15 (with std of 0.82) and the assessment score of Student Outcomes ranges from 2.38 (with std of 1.05) to 2.98 (with std of 0.90). |
Actions for Objective:
| Action |
|
|||
|
To improve confidence and significance of analysis results, number of students, different weights for each outcome, and other factors should be considered. For example, if a class has more than 20 (or 25) students, a random sample of 20 (or 25) students could be selected for assessment. Since COSC4319 and COSC4349 do not have the same number of students, the difference in students' needs to be weighted properly. COSC4319 is assessed directly by CS faculty members during students' project presentations based upon the prepared assessment rubrics, while COSC4349 is evaluated indirectly (but objectively) through test/exam questions. Therefore, to have the consistent assessment, the objective assessment for COSC4319 is developed and will be employed from Fall 2012. |
| Closing the Loop |
|
The first assessment scores on both Program Outcomes and Education Objectives for Spring 2012 is not enough to draw conclusions and/or suggest specific changes to the current curriculum. Therefore, the CS department will obtain more data during Fall 2012 and Spring 2013. Then, based on the analysis result collected at least over two or three consecutive semesters, the current tools and the rubrics will be re-evaluated to customize the curriculum to improve students' performance as well as to meet the ABET-CAC criteria.
In the future, the Program Outcomes and Objectives will be collected with the all the six survey forms and the rubrics for more core CS courses, also including other course activities such as quizzes, assignments, presentations, and exams. Based on the result, the CS undergraduate curriculum committee will plan to share their findings and recommendations to the CS department. Furthermore, CS faculty will discuss those issues with the course instructor(s) for the consistent, repeatable assessment plan and the effective change of the CS curriculum accordingly. |