SACSCOC Fifth Year Interim Report
Spacer
Assessment : 2012 - 2013 : Educational Programs :
Philosophy BA
Philosophy BA
3 Goals 4 Objectives 6 Indicators 8 Criteria 9 Findings 5 Actions
| GOAL: Improving Critical Thinking And Analytic Reasoning |
| Objective |
|
||||
| Critical thinking skills are an essential component of philosophical work. Students will be able to analyze arguments and draw conclusions from available information. |
| Indicator |
|
|||
| All students who take PHIL 2303 will be tested on their critical thinking skills. All faculty who teach PHIL 2303 will administer the Texas Assessment of Critical Thinking Skill (TACTS)s, an externally validated test of critical thinking skills, in a pre-test/post-test format. The TACTS is a broad-based assessment of critical thinking skills that goes beyond the current scope of PHIL 2303. This will allow the faculty to determine areas that may be added to our current curriculum in the future. In addition, it allows for substantial flexibility in what is taught; thereby ensuring academic freedom for instructors to design individual sections around their own expertise and interests. A copy of the current TACTS is attached. A copy of the credited responses is attached. The Philosophy Program Coordinator, currently Dr. Fair, will be responsible for ensuring that all faculty who teach PHIL 2303 effectively administer the pre- and post-tests in every section of their course. One faculty member, currently Dr. Sanford, will be responsible for gathering pre- and post-test data from the faculty members who teach PHL 2303. |
| Criterion |
|
|||
| A paired two-sample t-test will be performed on the scores of all students who take the pre-test and the post-test. The philosophy program expects to see a statistically significant improvement from the pre-test to the post-test. |
| Finding |
|
|||
| A paired two-sample t-test on our sample of 488 student scores, covering only those students who took both the pre-test and post-test, demonstrated a statistically significant improvement in their scores. Data and basic analysis are attached. |
| Criterion |
|
|||
| The data from the team that developed the TACTS show that a knowledge gap exists with respect to decision making when an outcome depends upon the conjunction of two probabilistic events. Their data show that less than 20% of those tested correctly answered the following question: "George is waiting for two of his customers, Fuzzy Logic Computers, Inc. and Stalking Horse Designs, to pay their bills. If either of them pays before the end of the month, then George can pay his supplier. But if neither of them pays, then George will have to take out a bank loan. George estimates that the chance that Fuzzy Logic will pay in time is 70% and the chance Stalking Horse Designs will pay in time is 60%. Assuming that his estimates are correct and that the two events are independent, what is the chance that George will have to take out a bank loan? (a) 12% (b) 40% (c) 65% (d) 42% (e) 88%"" 2012-2013 will be the fourth year that the Philosophy Program will expect all faculty to evaluate this type of reasoning as part of the critical thinking course. We will consider this effort successful if there is at least a 75% improvement on this type of question from the pre-test to the post-test. |
| Finding |
|
|||
| The sample is limited to only those students who took both the pre-test and post-test. For students whose pre-test score was reported, 87 out of 488 (17.8%) correctly answered the question. On the post-test, 197 out of 488 (40.4%) correctly answered the question. This represents a 127.0%improvement in the percentage of students who successfully answered the probability questions from the pre-test to the post-test. This is similar to the 126.2% improvement seen in 2011-2012, though both the pre-test and post-test rates were higher in 2012-2013. The 2012-2013 results compare favorably to the 85.6% improvement seen in 2009-2010 and 106.5% seen in 2010-2011; yet, it remains a challenge to improve even further.The sample is limited to only those students who took both the pre-test and post-test. |
Actions for Objective:
| Action |
|
|||
| Although our data showed a substantial improvement in calculating probabilities, it is expected that our faculty can improve on our performance. All Philosophy Program faculty who taught PHIL 2303 during 2012-2013 will be invited to share their experiences with those who will teach PHIL 2303 during 2013-2014. Of particular note is Dr. Sanford's dissatisfaction with his choice to adopt a new textbook for 2012-2013, as this decision affects a large portion of the total sample. He has already begun sharing his experience with others in the Program. In addition, input will be sought from other faculty who have experience teaching probability-based content. The goal will be to identify ways in which pedagogy in this area can be further improved without cutting back on the successful core of critical thinking skills we have regularly taught. It is expected that faculty will implement and evaluate new approaches during 2013-2014. |
| GOAL: Understanding Of General Philosophical Concepts |
| Objective |
|
||||
| As students progress through the Philosophy BA, they will acquire a basic understanding of metaphysics, epistemology, and moral theory. This basic information, provided by our introductory courses serves as the foundation for student success in upper-division courses. |
| Indicator |
|
|||
| All students in PHIL 2361 and PHIL 2603 will be tested on their knowledge of basic concepts in metaphysics, epistemology, and moral theory using a locally standardized pre-test and post-test for each course. Following a review of best practices for the teaching of these courses, a group of Program faculty chose the questions for the assessment. The questions asked cover the range of concepts that are taught in peer departments. Instruction on these concepts promotes a basic competence in metaphysics, epistemology, and moral theory. The attached documents provide the assessment instruments for PHIL 2361 and PHL 2603 as well as the credited responses for each. |
| Criterion |
|
|||
| A paired two-sample t-test will be performed on the scores of all students who take the pre-test and the post-test. Students in both courses will demonstrate a statistically significant improvement from the pre-test to the post-test. |
| Finding |
|
|||
| PHL 2361 students demonstrated a statistically significant improvement from the pre-test (M=6.4) to the post-test (M=10.4). Although statistically significant, the post-test mean is only 52% with 20 items. See the attached data. This is consistent with the 53.5% during 2011-2012, 54.5% during 2010-2011, and 51% during 2009-2010 on the post-test. Additional innovative strategies for teaching basic concepts in metaphysics, epistemology, and moral theory are needed to ensure deep learning. The number of reported data points is significantly lower than expected due to missing data from both of Dr. Cording's PHIL 2361 sections during Spring 2013. Dr. Cording had serious medical issues that disrupted end-of-esemester activities in his course and interfered with data collection. |
| Finding |
|
|||
| PHIL 2603 students demonstrated a statistically significant improvement from the pre-test (M=10.2) to the post-test (15.0). While statistically significant, the post-test mean is only 60.0% of 25 items. These results are consistent with the 60.3% improvement in 2011-2012. See the attached data. Hence, basic concepts in metaphysics, epistemology, and moral theory must be addressed in inventive ways so that deep learning versus shallow learning is accomplished. |
| Indicator |
|
|||
| Students will demonstrate increased understanding of Immanuel Kant's philosophy. Questions 10 and 12 on the pre-test and post-test were chosen to measure our Program faculty's ability to improve this targeted area. |
| Criterion |
|
|||
|
After comparing students' pre-test and post-test performance on questions 10 and 12 of those tests, the Program will consider this effort successful if the data indicate at least a 75% improvement in student performance on each question. Anything less will be taken as an indication that the Program must improve its performance in this area. Regardless of performance, the 2012-2013 data will serve as a baseline for measuring future performance. |
| Finding |
|
|||
| 23.8% of students chose the correct answer for question 10 on the pre-test. This improved to 44.1% on the post-test. This represented an 85.3% improvement. Likewise, 26.3% of students chose the correct answer for question 12 on the pre-test. This improved to 49.1% on the post-test. This represented an 86.7% improvement Though students demonstrated improved performance on both questions, the Program is not satisfied that this is the best that we can do in this area. |
| Indicator |
|
|||
| Students will demonstrate increased understanding of arguments related to the death penalty. Questions 19 and 20 on the pre-test and post-test were chosen to measure our Program faculty's ability to improve this targeted area. |
| Criterion |
|
|||
| After comparing students' pre-test and post-test performance on questions 19 and 20 of those tests, the Program will consider this effort successful if the data indicate at least a 75% improvement in student performance on each question. Anything less will be taken as an indication that the Program must improve its performance in this area. Regardless of performance, the 2012-2013 data will serve as a baseline for measuring future performance. |
| Finding |
|
|||
| 18.8% of students chose the correct answer for question 19 on the pre-test. This improved to 36.1% on the post-test. This represented an 92% improvement. Likewise, 27.3% of students chose the correct answer for question 20 on the pre-test. This improved to 52.2% on the post-test. This represented an 91.2% improvement. Though students demonstrated improved performance on both questions, the Program is not satisfied that this is the best that we can do in this area. |
Actions for Objective:
| Action |
|
|||
| The Program will continue its targeted instructional effort aimed at improving student learning. In PHIL 2603, this effort will focus on improving students' knowledge of arguments related to the death penalty. In PHIL 2361, it will continue the focus on Kantian philosophy. Of particular interest will be demonstration of improved performance over the 2012-2013 baseline data. Additionally, as the Program will seek data from evaluation of the PHIL 3364/3365 sections with an eye toward identifying additional opportunities to improve the overall presentation of general philosophical concepts to our students. |
| GOAL: Understanding Of General Philosophical Concepts |
| Objective |
|
||||
| Well-educated philosophy students will demonstrate appreciation for the arguments and positions of earlier thinkers. Because so much of what is written in philosophy is a reaction to the metaphysical and epistemological presuppositions of earlier thinking, it is the core of well-rounded philosophical education. |
| Indicator |
|
|||
| All students in PHL 3364 and PHL 3365 will be tested on their knowledge of general concepts in the history of philosophy. All faculty who teach these courses will administer a pre-test and post-test to all students. All Philosophy BA students are required to take PHL 364 (Ancient and Medieval Philosophy) and PHL 365 (Modern Philosophy). Together, these courses provide students with upper-level instruction covering the history of metaphysics and epistemology. Following a review of best practices for the teaching of these courses, a group of Program faculty chose the questions for the assessment. The questions cover the range of concepts that are taught in peer departments. Instruction on these concepts promotes a well-rounded understanding of the history of philosophy. |
| Criterion |
|
|||
| A paired two-sample t-test will be performed on the scores of all students who take the pre-test and the post-test. Students in both courses will demonstrate a statistically significant improvement from the pre-test to the post-test. |
| Finding |
|
|||
| PHIL 3365 students demonstrated a statistically significant improvement from the pre-test (M=3.4) to the post-test (14.3). Average scores on the post test were 333% higher than the pre-test. While statistically significant, the post-test mean is only 43.3% of the 33 items. As these results were obtained from a single section, the Program feels it is too soon to draw any conclusions, except that the students did demonstrate significant improvement. These results will serve as a baseline for judging future performance. |
| Criterion |
|
|||
| A paired two-sample t-test will be performed on the scores of all students who take the pre-test and the post-test. Students in both courses will demonstrate a statistically significant improvement from the pre-test to the post-test. |
| Finding |
|
|||
| No data is available for the Fall 2012 section of PHIL 3364, the only section of PHIL 3364 that was offered during 2012-2013, due the the faculty member losing the folder containing the pre-tests and post-tests before reporting the data. |
Actions for Objective:
| Action |
|
|||
| The Program will ensure that Dr. Fair, who is teaching PHIL 3364 during Fall 2013 records and reports pre-test/post-test data. Dr. Fair expects no difficulty, and has committed to take special precautions to ensure reporting of the data. |
| Action |
|
|||
| The Program will continue to monitor student learning in PHIL 3364/3365. It is essential that the Program record baselne data for the revised PHIL 3364 curriculum during 2013-2014 as was done for PHIL 3365 during 2012-2013. This will provide a means of assessing ongoing efforts to improve student learning. |
| Objective Supports Another 's Goals |
| Objective |
|
||||
|
Student's understanding of formal logic provides a foundation for rigorous analysis of arguments. All philosophy majors will demonstrate competence in propositional logic and predicate logic. Supports : Philosophy BA |
| Indicator |
|
|||
| All students who take PHIL 3362 (Introduction to Contemporary Logic) will be tested on knowledge of basic logical concepts, propositional logic, and predicate logic. All faculty who teach PHIL 3362 will administer a pre-test and post-test. These will serve as an indicator of student competence with the subject matter. Following a review of best practices for the teaching of this course, a group of Program faculty chose the questions for the assessment. The questions asked cover the range of concepts that are taught in peer departments. Instruction on these concepts promotes a an introduction to the concepts of formal logic, propositional logic, and predicate logic. The attached documents provide the sample questions similar to those used in the embedded assessments and the credited responses. Because some of these questions, particularly the logical proofs, have answers that could be memorized if the questions were known in advance, exam security requires that the actual questions not be posted in a public document. The Program faculty will use questions that are essentially similar but sufficiently distinct to limit the potential for academic dishonesty. A group of Program faculty has reviewed the particular questions that will be embedded each semester. |
| Criterion |
|
|||
| A paired two-sample t-test will be performed on the scores of all students who take the pre-test and the post-test. Students in both courses will demonstrate a statistically significant improvement from the pre-test to the post-test. |
| Finding |
|
|||
| PHIL 3362 students demonstrated a statistically significant improvement from the pre-test (M=.2) to the post-test (M=4.9). Although statistically significant, the post-test mean is only 35% with 14 items. See the attached data. |
Actions for Objective:
| Action |
|
|||
| As a result of a retirement, beginning in Fall 2013, the Program will employ two new faculty members to teach PHIL 3362. As part of this change, the Program is expecting both new members to work together to develop new stategies for improving overall student learning outcomes with respect to formal logic, propositional logic, and predicate logic. The faculty members teaching PHIL 3362 have been charged with developing an embedded assessment process that will satisfy both SACS and Core Curriculum assessment requirements. During Fall 2013, they will develop a preliminary assessment that will be used across all sections of PHIL 3362 during Spring 2014. Then, during Summer 2014, the Program will use the Spring 2014 results to fine tune the assessment process before full implementation during Fall 2014. |
| Previous Cycle's "Plan for Continuous Improvement" |
|
The Program will focus on improving deep learning by looking for ways to improve the links between PHL 2361/2603 and PHL 3364/3365. The goal will be to provide greater continuity between the introductory and advanced courses with an eye toward improving students' long-term outcomes as they proceed through the Program's curriculum. By linking the content and presentation of materials in the lower-level courses to what is expected in the upper-level courses, the Program expects to improve student outcomes and deep learning as measured by improved performance by students in PHL 3364/3365. As part of this ongoing effort, the Program plans to implement targeted effort at improving student learning outcomes in PHIL 2361/2603 and undertake a full review/revision of PHIL 3364/3365 for 2012-2013. It is expected that these changes will take approximately two years to begin showing significant results in the upper-level courses.
|
| Update on Previous Cycle's "Plan for Continuous Improvement" |
|
Elements implemented:
The Program successfully implemented revised curricula in PHIL 3364/3365. Dr. Fair delivered a revised PHIL 3364 curriculum during Fall 2012, and Dr. Gurley successfully delivered a revised PHIL 3365 curriculum during Spring 2013. The Program successfully implemented its targeted effort to improve student knowledge of the death penalty debate upon completion of PHIL 2306. The Program successfully implemented its targeted effort to improve student knowledge of the Kantian philosophy upon completion of PHIL 2361. Elements not implemented: The Program failed to evaluate student performance in PHIL 3364 owing to the data on student outcomes being lost before reporting. |
| Plan for Continuous Improvement |
|
The Program will continue its efforts to improve the links between PHL 2361/2603 and PHL 3364/3365. The goal will be to provide greater continuity between the introductory and advanced courses with an eye toward improving students' long-term outcomes as they proceed through the Program's curriculum. By linking the content and presentation of materials in the lower-level courses to what is expected in the upper-level courses, the Program expects to improve student outcomes and deep learning as measured by improved performance by students in PHL 3364/3365. Once the Program has data on the success of our new efforts in PHIL 3364/3365, we should be in a position to identify areas for improvement.
Revision of the PHIL 3362 curriculum and its assessment process is expected to be completed during 2013-2014. Due to the influx of new faculty members and the concomitant expansion of perspectives concerning contemporary moral issues, the Program has convened a committee of faculty members to revise the expectations and assessment instrument of PHIL 2306. This committee has its first meeting scheduled for September 2013, and is expected to issue its findings in time for Fall 2014 implementation. |