Evaluation of Courses and Clerkships

Introduction

The Evaluation Subcommittee of the Curriculum Committee is charged with the qualitative evaluation of all courses in the College of Medicine. To do this, we ask that all course directors submit the course syllabus and other information that the subcommittee can use in its evaluation. The course directors’ thoughts about the course are invaluable so we ask for a detailed analysis in addition to factual data. The reports will be used to: allocate educational funds equitably, support desired educational outcomes, promote innovation, and reward outstanding teachers, courses and clerkships. Recommendations of the committee are forwarded to the Curriculum Committee and to the Senior Associate Dean of Educational Affairs. Recommendations for providing resources to courses and clerkships are then forwarded to them by COMEC.

The annual report consists of three parts:

I. The course or clerkship syllabus given to the students
II. The written report prepared by the director with accompanying appendixes
III. Data supplied by the Office of Medical Education (OME)

In order to facilitate evaluation, course/clerkship directors should be sure to include all material requested in the exact form and organization requested. Each section of the syllabus and report should be clearly delineated and identified to correspond with these instructions.

Each section of the syllabus and report are evaluated longitudinally by members of the subcommittee. Four parameters are used to assess quality of a course or clerkship syllabus and four parameters to assess the written report. A reviewer will assess the same section in all reports, i.e. like an NIH grant evaluation. Therefore, each section should stand alone. For example: if a deficiency is identified in the section on student evaluation of instruction and a strategy for improvement implemented, the improvement should appear in that section in addition to the separate section on plans for improvement. Do not refer a reviewer to another section.

An example of an exemplary submission (course report / syllabus, clerkship report / syllabus) and the evaluation form used by the subcommittee is provided. It is strongly recommended that these documents be reviewed during preparation of materials.

I. Course / Clerkship Syllabus

Given to the students. In order, it must contain:

A. Overview

This section should contain the four following elements at a minimum:

  1. A brief description of the course/clerkship
  2. The administrative structure
    1. Department chair with contact information
    2. Course/clerkship director with contact information
    3. Course/clerkship administrative assistant/secretary with contact information
  3. An overview of the clerkship content
    1. Length, hours, units of study as appropriate
    2. Sites for instruction
    3. Schedule for a typical week
  4. The educational philosophy
    1. Course / clerkship
    2. Department – if there is one

B. Learning Objectives and Performance Criteria

These must be in the specific format requested (see representative course and clerkship syllabus template)

5. The competency categories to be addressed
6. The measurable learning objectives for each competency category
7. The learning/instructional activity used to achieve each learning objective
8. The evaluation methods used to assess student achievement of each learning objective.

Representative instruments should be included.

9. Describe the system for giving formative feedback. Representative forms should included, but alone are not sufficient.
10. Describe the system for determining summative feedback if methods besides exams are used to assess performance. Representative forms should be included.

II. Director’s Written Report

There are five sections to the directors’ report.

Representative course and clerkship written report templates.

  1. Identification of the Teaching Faculty for the Course or Clerkship1. Director(s): Total FTE
    2. Administrative support: Total FTE
    3. Departmental Teaching Faculty: Total FTE
    4. Non-departmental Teaching Faculty/Staff/Students: Total FTE
  2. Specific Changes in the Course or Clerkship5. Compare to last year and the rationale or basis for any changes.
    6. Address remediation of areas of weakness or deficiency cited in the previous report.
  3. Evaluation of Student Performance7. List the Course Objectives and identify types of evaluations used to assess student levels of achievement for each objective. Comment on the appropriateness of them. If MCQ’s are used, include the Table on item analysis provided by the Office of Medical Education.
    8. Describe how students performed on internal or external (e.g., NBME subject examinations) instruments relative to the course/clerkship learning objectives.
    9. Based upon the data from the Office of Medical Education, comment on the distribution of competency scoring and final grade distribution of student performances. Do you believe that the course evaluation system identified students who were truly exceptional and those that may need additional assistance? Comment on the course’s grade distribution in relation to the Curriculum Committees document, “Evaluation Student Performance in the UFCOM Competency-Based Curriculum (Exhibit 7, Effective July 1, 2006).Number of student performances scored in the following competency and grade categories.
    Competency Category Needs Remediation Acceptable Very Good Exemplary
    1. Professionalism
    2. Patient Care
    3. Medical knowledge
    4. Practice-based learning
    5. Interpersonal and communication skills
    6. Systems-based practice
    Year D or F C C+ B B+ A
    Current Year
  4. Feedback to Students10. Indicate what types of formative feedback are used. Identify the student rating to the on-line course question about formative mid-course feedback.
    11. Discuss the legitimacy of student ratings.
  5. Student Evaluation of the Course/Clerkship and Director12. What is your opinion of how well the course succeeded with reference to its goals. Indicate what you think the strengths and weaknesses are and any plans for improvement.
    13. Provide a synopsis/summary of student on-line evaluations of the course/clerkship and discuss the student ratings. Numerical data alone is not sufficient.
    14. Provide a synopsis/summary of the student debriefings. This should include analysis and comment by the director of student concerns. Minutes alone are not sufficient. Discuss the legitimacy of the student ratings.
    15. Provide a synopsis/summary of student evaluation of the Director, and administrative support, if applicable. Discuss the legitimacy of the student ratings.
  6. Evaluation of Instruction1. Provide a synopsis/summary of student evaluation (on-line and debriefings) of the overall instruction by faculty, residents and staff, as applicable.
    2. Critique the overall teaching performance of faculty, residents, and staff from the Director’s perspective. Identify and comment on strengths and areas needing improvement.
    3. Identify any plans for faculty development.
  7. Evaluation of Course/Clerkship1. Discuss the timing and appropriateness of the course/clerkship to the curriculum’s general professional education program including content and instructional methods.
    2. Discuss the strengths and weaknesses of the course/clerkship and describe plans for improvement. Include a description of present and/or future innovative programs.
    3. Discuss the long-term plans for continued development and new directions for the course/clerkship.
    4. Describe the resources needed from the Office of Medical Education to improve the course/clerkship.

III. Course / Clerkship and Student Numerical Data Supplied by the Office of Medical Education (OME)

  1. Course/Clerkship Evaluation – Student year end evaluations
  2. Course/Clerkship Instruction – Student year-end evaluations
  3. Course/Clerkship Director – Student year-end evaluations
  4. Student performance
    1. NBME subject examinations
    2. USMLE Steps 2 and Steps 2, if applicable.