The following recommendations are from the MPPDA Board Pass Task Force. This group was designed to evaluate risk factors for board failure for Med-Peds residents and to provide effective strategies for Med-Peds program directors to track and monitor progress.
Useful Strategies and Resources for Board Preparation
In-Training Exam (ITE) – The task force recommended that all residents take the in-training exams in both internal medicine and pediatrics on a yearly basis. These exams can provide information about specific knowledge deficits as well as general information about test-taking difficulties. The results of the exams will provide regular feedback to the resident and the program director regarding the individual resident’s readiness for the certification exams. The semi-annual meetings with the program director should include a discussion of the in-training exam results, review of data indicating likelihood of board passage (when available), specific areas of knowledge deficits, and a plan to address these areas.
Reading Program – The task force recommended that each program establish clear expectations regarding ongoing reading by residents. It is recognized that this will vary by the needs of the individual resident, the expectations of individual programs, and the level of training. The task force felt that being very clear about the expectations was critical, as it is not realistic to expect that residents will be able to pass boards based on their clinical and didactic training alone. Instead, residents should read textbooks pertaining to issues in both specialties, and make use of resources such as Pediatrics in Review or the ACP’s MKSAP on an ongoing basis throughout residency.
Ongoing Preparation Throughout Residency – Programs should also set expectations for residents regarding ongoing preparation for Boards throughout residency. In some programs, this may consist of a regularly scheduled board review program (often developed by the categorical programs). A number of programs recommend that residents work on board review type questions, often beginning during the last two years of training. Completing 5 to 10 questions per week will allow residents to get through a significant number of questions prior to completing residency.
Board Preparation Resources – Many resources are available to help residents prepare for their certification exams. Different resources exist for internal medicine and pediatrics, and specific recommendations should be tailored to individual residents and programs. Some resources that are used by many programs include:
- Reading MKSAP and Pediatrics in Review with each rotation
- Completing 5 years of PREP questions (5 per week)
- Reading/reviewing key texts such as Zitelli (for pictures), AAP Redbook (for immunization information and other infectious diseases), and Smith (for common genetic conditions)
A sample of board preparation recommendations are provided in Appendix 1. There are also many board review courses available. These include: on-site programs, web-based programs, and videotaped programs. Some program directors recommend that residents participate in one of these courses during their third-year, especially if they have had difficulty with exams. In the future, program directors may want to consider requiring these courses for struggling students. Alternatively, residents may choose to take a course after their residency, just prior to taking the certifying exam. Again, this recommendation needs to be tailored to the needs of the individual resident.
Explore educational resources at your institution/community to help individuals with test-taking difficulties. Such resources may include: neuropsychologists, educational specialists in the medical school, employee assistance programs, and reading specialists.
Identifying Residents at Risk
Many program directors use the in-training exam as a marker for those who may be at risk of not passing their board exams. While programs generally use the first-year exam as an initial assessment of medical or pediatric knowledge, low performance on this exam could also flag residents as having a more general difficulty with taking tests. Programs use various cut-points for identifying residents at risk, ranging between the 25th percentile and 50th percentile of those taking the exam nationally. Obviously, using a higher cut-point may result in more residents being identified as “at-risk,” but can also be a motivator for some residents.
Many, but not all, residents who have test-taking difficulties will have a pattern of difficulties on exams going back to the USMLE exams. Data generated from a recent MPPDA survey demonstrated that residents who had failed either board exam were highly likely to have had a USMLE Step 1 score of less than 220. USMLE 2 was somewhat less predictive, as some residents who had failed a board exam and had a USMLE 2 score of greater than 220 often had a USMLE 1 score of less than 220. Thus, if a first-year resident has a low ITE score and also scored less than 220 on the USMLE exam, program directors may want to begin to address this even earlier.
Once residents are identified as being “at-risk,” using program-specific criteria as outlined above, each program needs to determine how best to assist the individual resident. Most programs select a combination of both formal and informal assessment strategies. Program directors should keep in mind that test-taking difficulties are often multi-factorial.
Several programs use informal assessments of learning styles to help evaluate their residents. One such tool is the Visual-Aural-Reading-Kinesthetic (VARK) Learning instrument (http://www.vark-learn.com/). Other programs have residents reflect on what worked well for them when studying for exams in the past. Informal assessments may be most useful for residents who have not been reading and studying sufficiently because they can prompt residents to recognize that they need to dedicate more time and energy to this process.
A more formal assessment should be considered under the following circumstances 1) the resident has been diligent about studying, yet still seems to have difficulty retaining information (may occur in clinical situations in addition to exam performance); 2) if sufficient time is being spent on studying, but improvements are not occurring over time; or 3) if the resident has a history of test-taking difficulties in the past.
More formal assessments can include neurocognitive testing, assessments of reading speed, or evaluation for subtle attention problems or test-taking anxiety. This kind of assessment is actually a medical evaluation that is best managed outside of the residency program. For example, many institutions have an employee assistance program that can facilitate such an evaluation. However, institutions do vary on how much support they can provide for residents who need these formal assessments. Neurocognitive testing is expensive. In some places, there are institutional funds to support testing; in other places, the cost may need to be borne by the resident or the resident’s insurance plan. Program directors should be familiar with their local resources, particularly with regard to personnel who are best able to assess medical professionals. Medical schools may also be able to identify resources for residents.
Regardless of what type of evaluation is done, remediation will be needed for residents who are struggling with poor performance on board exams. This remediation is best accomplished with a faculty mentor or “coach” — in other words, a dedicated person whose charge it is to ensure the success of the struggling resident. Each individual program should decide whether the program director should serve this role; sometime having the program director serve as both the “coach” and the person who has to develop consequences may prove to be a conflict. There should be regular meetings between the coach and the resident, with specific goals to be accomplished between meetings. Activities that can be used for remediation include:
- Answer a specific number of questions each week, with the goal of getting 70-75% correct.
- Attend a formal board review course, not only to increase knowledge, but also to expose the resident to the breadth of content that is needed to pass the exams
- Complete a series of practice questions on Pedialink using “test mode,” which provides the opportunity to simulate an exam, since residents cannot see the answers after each question.
- Make notecards to represent weaker areas of content knowledge on the ITE, and write out answers to those content specifications.
- Require attendance at local board review sessions
- Utilize a series of worksheets Appendix 2 in which residents have to identify differentiating features from among several conditions that could present similarly.
- It is important to be sensitive to the fact that there is a stigma associated with going through remediation plan, as residents who are experiencing difficulty are often already overwhelmed and inefficient. Scrutiny associated with remediation may worsen these issues.
- Most programs do not allow residents who are in a remediation plan to moonlight or to attend outside conferences.
- Consideration should be given to having residents take their ABIM and ABP exams in separate years, especially when they are struggling.
Materials from the presentation “Improving Medical Knowledge and Board Performance for Med-Peds Residents” from the National MPPDA Meeting April 10, 2013. Sam Borden, Caroline Mueller, and Russ Kolarik are available for MPPDA Members though AAIM Connect at this link.