It’s Time to Change the MOC Process

Everyone knows the problems with the maintenance of certification testing. The tests, no matter what specialty, ask arcane details of obscure issues in medicine, information that easily could be – and is — looked up. It doesn’t test most physicians’ stock-in-trade: decision-making. The score has little to do with performance in practice.

However, to totally forgo ongoing assessment and certification turns the quality control of medical practice over to payers, including the government. In addition to it affecting our income, do we really want our abilities to be represented by how well we check off boxes in the electronic health record?

The simple issue is this: Testing recall of information has little correlation with actual practice, especially in this environment in which information changes so quickly. Ongoing physician assessment must assure that board-certified physicians can make decisions based on the best, currently-available evidence that should inform their practice.

The process of making decisions goes way beyond simply recalling previously-learned information. Daniel Kahneman, in his popular book Thinking Fast and Slow, describes two distinct ways that humans make decisions.

In System 1 Thinking, we rely mainly on our memory and recent experience to make decisions almost automatically without deliberate, conscious thought. For example, most primary care physicians, when faced with treating a cat bite, will immediately recall the most common organisms likely to be causing the infection and the appropriate first-line treatment.

The other type of thinking, System 2 Thinking, is a slow, deliberate approach, often accompanied by a realization that new information is needed to solve the problem. If a patient is allergic to the first-line treatment of a cat bite, many physicians will need to find, using information tools, appropriate second-line options. System 2 thinking requires two steps: 1) recognition that we don’t know the right answer; and, 2) using information sources to find the information quickly and accurately.

System 2 Thinking requires physicians to manage their “information inventory” to determine whether they can make decisions based on their current knowledge or whether they need to use high quality information tools to find the appropriate information at the time the information is needed.

Current testing, even that which is cased-based, evaluates recall of information and previously-developed “illness scripts.” While this knowledge is important, it is also important to test whether physicians two additional components of decision-making: 1) knowing when to question one’s existing knowledge; and 2) where to obtain new information when a patient case requires one to go “off script.”

Ongoing certification processes of physicians should focus more on clinical decision-making ability, assessing when current knowledge may not be sufficient or when it is not based on the best current evidence. It should also test the ability to use commonly-available, evidence-based tools to quickly find the answers to questions. In other words, tests should reflect the real world of medicine.

Some boards are considering “open book” assessments. This is just a start. As in the real world of clinical practice, there has to be an element of time involved in the assessment. Testing in the future should be more closely aligned with clinical practice, in which physicians ask themselves: 1) Am I confident enough in my decision to go forward; and, 2) If not, where can I quickly find the best evidence? In this way, the new exam will test examinees’ ability and practice of keeping up with changes in medicine, as demonstrated by their ability to determine when their existing knowledge might not be sufficient, triggering their use of information tools to determine or confirm their initial decision, and the ability to use tools to find the best currently-available evidence.

These changes will do two things. First, it will reflect actual practice, and will feel like what most physicians do day to day (or should be doing). Second, it will assess lifelong learning skills. In this way it should be more predictive of performance in practices in which information is constantly evolving and comes from a wide variety of sources.

This approach will not be easy – the “right” answer on a test question may change quickly as evidence evolves. Still, this approach, rather than asking physicians to commit to their short-term memory various facts they no longer use, while not giving them credit for the decision-making and information retrieval skills they’ve developed in their practice, will give all of us a more confidence in the process.