SACSCOC Fifth Year Interim Report
Spacer
Assessment : 2013 - 2014 : Educational Programs :
Computing Science BS
Computing Science BS
2 Goals 2 Objectives 2 Indicators 2 Criteria 2 Findings 2 Actions
| GOAL: Specialized Competencies |
| Objective |
|
||||
| Students will develop and demonstrate skills and knowledge in their concentration areas. The department offers three concentration areas: Computer Science, Information Systems, and Information Assurance. |
| Indicator |
|
|||
|
During department meetings faculty with expertise in the three concentration areas will discuss students’ performances in their concentration areas. We expect that faculty in each concentration area will evaluate the students' performances across the 18 hours course sequence for each concentration. |
| Criterion |
|
|||
|
In last year's review the faculty assessed more than 70% of the students were performing at or above expected levels. As a result, the faculty set a higher expectation. We expect that the faculty will deem at least 72% or more of students’ performances as acceptable. |
| Finding |
|
|||
|
During the academic year (2011-2012), Computer Science (CS) faculty developed indirect program assessment tools and rubrics to address the new ABET-CAC (Computing Accreditation Commission) criteria, which includes eleven Student Outcomes (SOs) ((a)-(k)). In particular, CS Department set up four Program Education Objectives (PEOs) (E01-E04). For the quantitative assessment and continuous improvement of the program, the tools and rubrics have been continuously used since the 2011-2012 academic year. The students in two courses, COSC4319 (Software Engineering) and COSC4349 (Professionalism and Ethics) participated in this indirect assessment every fall and spring semester since Spring 2012. In addition, another assessment tool, TASO (Test for Assessing Student Outcomes), was implemented for the direct assessment of the students’ outcomes. This direct assessment was first performed in Spring 2013 and is also applied for this academic year as well. The ABET assessment team, which visited to the CS department in October 2012 for 2012-2013 Accreditation Cycle, reported their assessment result. Two deficiencies, two weaknesses, and one concern were cited in the Draft Statement by the ABET Visiting Team. After the department’s response to the initial report, , one deficiency was resolved and the second was cited as a weakness. Both weaknesses from the Draft Statement were cited as concerns. The concern was elevated to a weakness. CS faculty reviewed the ABET assessment team’s report and recognized the ABET team’s findings. During the current academic year (2013-2014), the assessment results have been reviewed and discussed. A number of suggestions and revisions have been made to properly address the issues raised in the ABET assessment team’s report as well as to continuously improve the program. |
Actions for Objective:
| Action |
|
|||
|
The aforementioned indirect and direct assessment tools and rubrics were performed and resulted in five consecutive semesters’ indirect assessment and three direct assessment data. The TASO examination was first administrated in Spring 2013; then, it has been applied in this academic year. After its first application, the TASO exam has been revised by the CS Undergraduate Curriculum Committee (UCC) during this academic year, specifically in the area of software engineering (Outcomes (i)-(k)) in order to more closely reflect the contents that are currently taught. Additionally, to address the findings from the TASO exams as well as the ABET team’s report the followings have been changed: (1) the coverage of the IEEE 754 floating-point standard has been moved from COSC 3327 to COSC 2329; (2) COSC 2329 has been revised with additional architecture and digital logic modules; (3) COSC 2347 has been updated to teach a scripting language, Python, particularly for IA and IS students; and (4) the UCC re-arranged course contents, revised curriculum, and discussed the possibility of introducing two new courses to act as a capstone courses. The CS UCC will monitor the effect of these changes as students in these classes approach graduation in 2015-16. The process of periodic review and revision of PEOs and SOs has been initiated with the rubrics and instruments through students’ presentation (at COSC4319), students’ response (at COSC4349), and TASO. |
| GOAL: Ethical Principles, Technical Skills, And Management Skills (core) |
| Objective |
|
||||
| Students will develop and demonstrate knowledge of ethical principles, technical skills, and management skills relevant to the field of computer science. |
| Indicator |
|
|||
| All undergraduate CS students must complete COSC 4319 (Software Engineering) and COSC4349 (Professionalism and Ethics) prior to graduation. In addition, COSC4319, as a capstone course, covers the eight Student Outcomes (except the three outcomes (e), (g) and (h)) out of the eleven Student Outcomes and COSC4349 addresses the remaining three outcomes. Therefore, both the courses are selected for assessment. Additionally, TASO (Test for Assessing Student Outcomes) is implemented to quantitatively measure the eleven SOs. It is required for senior students to take it in senior-level Software Engineering course, prior to graduation regardless of their program concentration. |
| Criterion |
|
|||
|
To perform a quantitative assessment, the following rubrics for the two courses are developed: (1) rubric for Project, (2) rubric for presentation evaluation, (3) rubric for group member evaluation, and (4) rubric for ethics topics. For COSC4319, faculty members observe students’ project presentation and directly evaluate students’ performance based upon the rubrics (1)-(3). One the other hand, for COSC4349, students’ performance is indirectly evaluated through the exam questions that address rubric (4). Each category is rated with the following scale values: (1) 1.0 (below expectations or unacceptable), (2) 2.0 (evolving or developing), (3) proficient (or competent), and (4) outstanding (or exemplary). Student’s performance on Student Outcomes is directly evaluated with each specific rubric, while the performance on objectives is indirectly measured by mapping between Student Outcomes and Program Objectives. Questions on TASO are based on topics from a number of required core courses (including COSC 1436, 1437, 2329, 3318, 3319, 4318, 4319, and 4349). Currently, it contains 26 multiple choice questions and 1 short answer question. Each multiple choice question has five choices, including the last choice of “I don’t know”. The percentage of students who select the correct answer to each given question will be computed and evaluated. |
| Finding |
|
|||
|
Student performance on the capstone project in COSC 4319 addresses 8 of the 11 ABET/CAC Student Outcomes. Fall 2013 and Spring 2014 results indicate somewhat improved performance in six of the eight student outcomes, with only one outcome(teamwork) showing levels below 2012/2013. The Ethics Test covers the three outcome that are not addressed by the Project assessment. One of the three student outcomes (Ethical responsibility) had a lower level than the previous academic year's level. The reamining outcomes (communication and evaluating local and global impacts) showed marked improvement. A copy of the comparison chart and the ABET/CAC Student Outcomes are attached for reference. |
Actions for Objective:
| Action |
|
|||
|
To improve confidence and significance of the analysis results, number of students, different weights for each outcome, and other factors should be considered. COSC4319 and COSC4349 do not have the same number of students, the difference in students’ needs to be weighted properly. In particular, the UCC raised a concern on the number of students in each project, since it is difficult to measure each student’s contribution, work, and learning. As a one solution, potential implementation of capstone courses was discussed. The level of attainment was evaluated by the CS UCC for each of the Student Outcomes (a)-(k) and the corresponding Program Education Objectives. Then, if needed, specific course contents have been revised to address the weaknesses. The TASO exam questions specifically on the topics of Software Engineering (Outcomes (i)-(k)) have been revised so as to more closely reflect the contents that are currently taught. |
| Previous Cycle's "Plan for Continuous Improvement" |
|
In the current (2012-2013) academic year, the department has focused on developing assessment instruments, which meet the new ABET CAC criteria, and also initiated the application of the developed instruments. However, as the ABET assessment team found, the PEOs (and/or SOs) had not been fully documented (or at least not documented) by all of the program’s constituencies (i.e., the students and the Advisory Board). Therefore, students’ exit survey and the Advisory Board review/survey will be documented and the results will be utilized into the continuous improvement process.
Accordingly, in the next academic year, the UCC will focus on the following items so as to demonstrate the continuous improvement effort with the periodic review and revision: (1) Documentation of the review and revision plan, (2) Documentation of full details of the TASO exam, its implementation, and its utilization as input into the continuous improvement process, (3) Further assessment of student outcomes and analysis of the extent to which they are attained, (4) Details of the utilization of the results and findings for program improvement, and (5) Continuing collection and analysis of the whole process and assessment of the implementation and effectiveness of the process. |
| Update on Previous Cycle's "Plan for Continuous Improvement" |
|
For this academic year, our focus was mostly on the revision of the TASO exam, making changes on the course contents accordingly, and responding to the ABET final report. Accordingly, docummentations, reports to the ABET CAC, continuing assessment, and revision has been properly performed.
One of the main items listed as this year’s goal was to document the students’ exit survey and the advisory board review/survey to utilize those for continuous improvement. However, we were not able to document the surveys for the students who are leaving the program and for the advisory board. The CS UCC has determined to review the Program Education Objectives (PEOs) on a bi-annual basis during the Fall Semester of even-numbered years usding the data collected from the Inderstrial Advisory Board and students' exit servey. The first review of the PEOs is scheduled for Fall Semester 2014. |
| Plan for Continuous Improvement |
|
As for periodic review and revision for continuous improvement of the program, the followings are planned for the next academic year:
(1) Continuing documentation of the review and revision plan, (2) Continuing documentation of the utilization and revision of TASO exam as input into the continuous improvement process (3) Assessment of student outcomes specific to each concentration (i.e., each of CS, IS, and IA concentrations), (4) Continuing collection and analysis of the whole process and assessment of the implementation and effectiveness of the process. (5) Initiation of the assessment to the newly-approved programs (e.g., Software Engineering Technology and Electronics and Computer Engineering Technology) |