|
| Original Creations |
|
Mitchell J. Lipp, D.D.S.
Dr. Lipp is Clinical Assistant Professor, Director of Predoctoral Courses in Clinical Orthodontics, and Coordinator of Curriculum for Undergraduate Orthodontics, Department of Orthodontics, New York University College of Dentistry. Direct correspondence and requests for reprints to him at Department of Orthodontics, New York University, College of Dentistry, 345 East 24th Street, New York, NY 10010; 212-998-9512 phone; 212-995-4241 fax; mjl3@nyu.edu. Key words: dental education, orthodontics, competency-based education, credentialing Submitted for publication 9/4/07; accepted 2/13/08 Predoctoral orthodontic education is in flux. The American Dental Association’s Accreditation Standards for Dental Education Programs have become less specific in this area.1,2 There seems to be no consensus as to how institutions should reach their orthodontic educational objectives. Research has shown great variability in the focus,3-5 assessment,5,6 and timing3,5 of didactic and clinical orthodontic training among educational programs, both in the United States and abroad. Surveys show that most dentists do not feel prepared by their predoctoral education to apply orthodontic knowledge to clinical situations and that more time devoted to this area is needed.7-9 One especially troubling article compared the orthodontic knowledge of first- and fourth-year dental students and found no significant improvement in clinical diagnostic skills.10 Another study discovered that a number of malocclusions encountered by recent graduates in general practice were not recognized or referred for specialist treatment.11 Perhaps a reflection of inadequate predoctoral orthodontic training is the increased demand for orthodontic continuing education classes.8,12 Clinical diagnostic skills in the recognition and management of malocclusion and skeletal problems are necessary for general dentists, who asprimary care providers are the gatekeepers for accessing care. An objective competency-based course in this area has the potential to validate the product of education, create consistency in predoctoral orthodontic education, provide practitioners with the necessary skills to manage patients, and ensure that the public welfare is paramount in training health care providers. In 2000, the New York University College of Dentistry (NYUCD) developed a curriculum that was comprehensive, integrated, and competency-based. The Department of Orthodontics was responsible for competencies in the management of malocclusion and skeletal problems. A course was created to provide third-year dental students with instructional and experiential opportunities to attain competence while accommodating variation in individual learning styles. Clear and specific end points were established to define the competencies that could be objectively assessed in the course. Case-based clinical simulations were the basis for instruction and assessment. This article describes the course and principles fundamental to its design and presents data based on student performance measures and surveys/course evaluations that suggest it is an effective component in a strategy to objectify instruction and assessment of clinical competence.
Defining Competence and the Concept of Critical Errors Often, the subject of orthodontics is taught at the predoctoral level from the specialist’s perspective. In the process of developing our course, the central question was the following: "What skills must a general dentist have to properly manage his or her patients with malocclusion and/or skeletal problems?" Suddenly, with this question, there was a shift in our approach. This course now focuses on the ability of the generalist to recognize patients with occlusal or skeletal conditions that would benefit from treatment or intervention with comprehensive orthodontic treatment, limited orthodontic treatment, growth modification, orthognathic surgery, and other adjunctive services. The course recognizes that most patients who benefit from orthodontic care (or orthognathic surgery) do not have disease in the traditional sense. There is seldom pain, pathology, or dysfunction, and the benefit from treatment is largely esthetic. The dentist’s responsibility is both to counsel patients who have esthetic concerns and to recognize those who possess risk factors or conditions that may lead to other problems that compromise health or function. A list of occlusal, functional, and skeletal conditions that the graduate must be able to recognize was developed (Figure 1). The threshold for competence for each condition was specifically defined. For example, in the skill of determining crowding in the arch, the threshold for competence is recognition of severe dental crowding, defined as being greater than 5 mm. Failure to identify severe dental crowding, or describing mild (1-2 mm) or moderate (3-5 mm) crowding as severe, is termed a critical error. The concept of critical errors provides the backbone for standardizing and objectifying the assessment of competence. By our definition, if a student commits a single critical error when evaluating a patient, competence has not been demonstrated. When establishing the list of critical errors, the department referred back to the clinical situation to justify the thresholds. Again, using the example of crowding, at what point does dental crowding move beyond an esthetic concern and become a risk factor for dental or periodontal disease? Will severe crowding compromise the ability of a patient to maintain a plaque-free environment? When should a dentist step in and advise the patient about the condition? This sort of dialogue occurred for each criterion listed, for there needs to be a reason to justify why this is a critical issue for patient management. The evaluation criteria that arose from this process became fundamental to both instructional and evaluative aspects of the course. The criteria are referred to in each session, during the competency assessment, and as a reference guide in the clinics, providing students with a well-defined list of the conditions they are expected to recognize.
The Path Toward Competence: Course Design, Implementation, and Evaluation Generally, attaining and assessing the clinical skill of diagnosis and treatment planning are accomplished at the clinical level, where student-faculty instruction is one on one. The class size needed to be controlled for the program to be effective. For each group of approximately thirty-five students, there is one lead faculty and three additional instructors (similar to the student-faculty ratios in the general practice clinics). To control the conditions for assessment of competence, the course incorporates clinical simulation cases. These cases consist of the following images: facial photographs (frontal, profile), intra-oral photographs in maximum intercuspation (frontal, right side, left side), intra-oral occlusal photographs, a panoramic radiograph, and a lateral cephalometric radiograph. The clinical simulation cases also include a brief patient summary, identifying the patient’s age, significant dental history, significant medical history, habits or abnormal functions, and esthetic concerns. Formative Experiences The course consists of six weekly consecutive two-hour sessions and a self-study component. It reinforces didactic knowledge presented in the first- and second-year curricula, as diagnostic and treatment planning concepts are reviewed. These concepts are then applied as students participate in formative exercises, working directly and individually on clinical simulation material. Faculty members evaluate the exercises as they would in the clinics: on a one-on-one basis.
 Figure 1. Orthodontic competency evaluation criteria
Session 2: Cephalometric Evaluation. Students are randomly given one of three cephalometric tracings representing a variety of dentofacial problems. They are required to locate cephalometric landmarks and reference lines, measure cephalometric values, and interpret their findings compared to normative values. Using this information, the student develops a problem list, with each significant finding supported by at least one cephalometric value. The problem lists are then reviewed and discussed with a faculty member. Later, students present their findings for the various tracings in a group discussion moderated by a faculty member, so that students are exposed to a greater variety of skeletal conditions and comprehend the relationships between malocclusion and skeletal problems. This exercise encourages students to think about skeletal problems in the sagittal and vertical planes of space, along with problems related to anterior dental inclinations. It also introduces the idea of presenting evidence from the diagnostic records to support the findings in the problem list. Session 3: Diagnostic Cast Analysis. Students are randomly assigned one of five sets of diagnostic casts trimmed in maximum intercuspation, representing a variety of malocclusions at different stages of dental development. Using the competency evaluation criteria, students construct problem lists that are reviewed and discussed with a faculty member. Concepts of normal and abnormal relationships are explored, and critical errors are identified using the evaluation criteria. Later, students present their findings for the various casts in a group discussion moderated by a faculty member, so that students are exposed to a greater variety of malocclusions. Sessions 4 and 5: Mock Competency Assessments. Students are given clinical simulation cases and are required to construct a problem list, a list of treatment objectives, and a sequential treatment plan under conditions similar to the final competency assessment.After students complete the assessments, they are reviewed and discussed in class. Session 6: Final Competency Assessment. The final competency assessment, consisting of four clinical case simulations, is given at the end of the course. Students construct a problem list, a list of treatment objectives, and a sequential treatment plan based on the clinical case simulations. The assessment is set at the level of competence for a general practitioner. The assessment emphasizes the problem list since the generalist should be able to recognize conditions that would benefit from treatment. The threshold of critical underscores the necessity of a generalist to counsel the patient to seek treatment. The generalist should be able to create a treatment plan that is within his or her skill set. Since training in clinical orthodontic treatment is often beyond the scope of the predoctoral curriculum, the generalist should be able to utilize the services of specialists to manage his or her patients. The treatment plan is a sequential list of dental and nondental services required to address the patient’s problems. The order of the list should be rational, addressing oral/dental disease and dysfunction and the patient’s needs/concerns. Before initiating orthodontic treatment, the patient should be cleared of etiology and/or conditions specifically related to pain, pathology, and dysfunction. It is therefore necessary to begin a sequential treatment plan by obtaining medical, dental, and periodontal clearance when indicated. If orthodontics is required to address malocclusion or a problem of tooth malposition, the student must be able to decide the level of treatment: limited or comprehensive. If there is a skeletal problem (defined as a condition that affects malocclusion and/or facial attractiveness), the student must determine if growth modification or orthognathic surgery is indicated. Finally, the student must be able to determine other adjunctive services (prosthodontics, implants, esthetic dentistry, oral surgery, etc.) that may be required to adequately address the patient’s condition. Since competence is an all or none event, the minimum standard to demonstrate competence is defined as zero critical errors in one clinical case simulation. The number of critical errors is irrelevant in the determination of competence. One critical error is sufficient to not be competent. The assessment score increases as the student consistently demonstrates competence, with zero critical errors in all four clinical case simulations earning the highest score. Self-Study: Problem-Based Learning-Like Experiences Classroom experiences are supplemented with a self-study component of the course. Seventeen clinical simulation cases are posted on the NYUCD academic intranet. Students are asked to go through the process of constructing problem lists, treatment objectives, and treatment plans for each of these clinical simulation cases. Students are encouraged to team up with others in the group outside of the classroom to discuss, debate, and justify their findings. Since some of these clinical simulation cases could appear on the final competency assessment, students are motivated to embrace this aspect of the course. During session five of the course, students can discuss and ask questions directly to faculty members concerning these clinical simulation cases. This component of the course was guided by ideas that have been popularized in the problem-based learning model: encouraging active learning, problem-solving, and collaboration. Problem-based learning has grown in popularity as a teaching technique, with some authors trying and/or recommending variations of this learning style13,14 and case-based instruction15 in predoctoral orthodontic education. The entire course promotes participation and an active ongoing process of questioning. The formative classroom experiences and supplementary self-study cases present opportunities for discussion and inquiry. Promoting Standardization in the Course In planning this course, great effort was made to create and define a common and standardized vocabulary, including every term, modifier, and quantifier. For example, while there may be no universal agreement about what constitutes severe mandibular dental crowding, within the context of the course there is no ambiguity. Severe crowding is defined as greater than 5 mm of interdental crowding per arch. The dental arch is defined as the arch perimeter mesial to the first permanent molars. Techniques for establishing the arch perimeter and estimating dental crowding are also described during the course. With a standardized language and well-defined criteria to describe abnormal conditions, students can more efficiently grasp concepts. The course itself serves as a means for effective faculty standardization and calibration. All instructors take the course and the competency assessment (and meet the requirements for competence) prior to teaching. This allows for consistency in instruction and assessment. At the conclusion of each cycle of the course, the course director reviews with the instructors the results of the competency assessments and student course evaluations, in order to identify patterns of critical errors and students’ concerns to better focus instructional and educational methodologies. These post-course meetings are a valuable feedback mechanism that helps to reinforce the standardization and calibration process. All assessments are graded by the course director based on the established competency evaluation criteria as defined in the course, ensuring consistency in grading. Course Policies: Attendance, Grading, Tutorials Attendance at each of the five sessions preceding the competency assessment is encouraged and is factored into the course grade as shown in Table 1.
 Table 1. Grading mechanism for course
If a student fails the assessment, he or she receives a conditional grade and is required to retake it. The grade of record is based on the last assessment taken. If a student feels the assessment did not accurately reflect his or her abilities, the assessment can be retaken. If however, the student fails the next assessment, the student will receive a conditional grade and will be required to retake it. The maximum number of assessments allowed is three (twelve clinical case simulations in total), after which the student is required to remediate or repeat the course. Students who fail the assessment are required to meet individually with the course director. During a thirty-minute conference session, the student and course director review the assessment, identify problem areas, and develop an individualized strategy to prepare for the next assessment. Summary of Results: Academic Year 2003-04 The effectiveness of the course is measured in terms of students’ outcomes and perceptions. Student outcomes are measured in two ways: the number of trials on assessments until the student demonstrates competence (as defined by the criteria outlined in the course), and the final grade, which considers the effect of formative experiences (as measured by attendance points) as well as performance on the assessment. The data pertaining to course grades and student performance on competency assessments for academic year 2003-04 are presented in Figures 2 and 3.
 Figure 2. Academic year 2003-04 final course grades
 Figure 4. Academic year 2003-04 statement 1 by groups 1-9
Discussion and Conclusions Although clinical simulation cases provide a valuable tool for learning and assessment, competence can only be validated in the rigorous application of these guidelines to patients in the context of comprehensive diagnosis and treatment planning. The course described is perceived to be a starting point in a process of refining the methodology for training, assessing, and credentialing clinical competence. Future studies will investigate the effect of group size, student-faculty ratios, absenteeism, and curriculum clock hours among other variables that affect student performance outcomes. Future investigations should also attempt to validate the performance bar for competence, including asking: should the threshold for competence (zero critical errors in one or more clinical simulation cases) be adjusted? As educational programs develop objectified competency assessments, these measures should be clinically validated. This matter is being addressed at our college as we have recently developed a D4 clinical orthodontic rotation in which skills taught and assessed in the D3 course are applied to patients. With this new component in place, we envision a second stage in credentialing for clinical competence. We intend to track student performance in the D3 clinical simulation course and the D4 clinical rotation to refine our methodology for assessment of competence. Increasingly, society demands institutional accountability concerning clinical competence since it directly impacts the well-being of the public. The idea of objectifying clinical competence is a necessary one. Objectifying clinical competencies and proficiencies in staged training and testing exercises so that institutions will be able to set clinical training goals that ultimately achieve 100 percent attainment has widespread implications for all health care institutions. Additionally, the clarity we have achieved by defining and assessing competence has facilitated the learning process. Faculty members better understand the educational goals of the course, and students know what performance measures they are expected to achieve.
Acknowledgments The author gratefully acknowledges the following for their assistance in this project: Rahul Gulati, D.D.S., a first-year orthodontic resident at New York University College of Dentistry; Prospero E. Matos, a third-year dental student at New York University College of Dentistry; and Randy J. Weinstein, D.D.S., who graduated from the New York University College of Dentistry Orthodontic Postdoctoral Program in 2005. REFERENCES
|
