The Forest From the Trees

In 1995, researchers at the University of Pittsburgh Medical Center (UPMC) published findings from a study of 772,000 patient admissions, across 78 hospitals nationwide between July 1987 and December 1988. Their study sought to deduce probabilistic models for priority triaging of patients admitted with pneumonia who were at the highest risk of mortality from the infection.

The researchers assessed a variety of models—logistic regression, neural networks, Bayesian and non-Bayesian belief networks, and rule-based systems—for their appropriateness in producing a high predictive value for whether or not to treat the patients as inpatients (more severe cases) or outpatients (less severe cases).

As described in a 2015 analysis of that 1995 study published by researchers at Columbia University and Microsoft, the most accurate models that could be trained from the 1995 dataset were multitask neural nets. Despite this, these models were considered to be too risky for use on real patients, as they were not as intelligible (interpretable by humans) as the rules-based models. The 2015 analysis continued—

On one of the pneumonia datasets, the rule-based system learned the rule “HasAsthma(x) ⇒ LowerRisk(x)”, i.e., that patients with pneumonia who have a history of asthma have lower risk of dying from pneumonia than the general population. Needless to say, this rule is counterintuitive. But it reflected a true pattern in the training data: patients with a history of asthma who presented with pneumonia usually were admitted not only to the hospital but directly to the Intensive Care Unit.

Such conclusions, based upon decontextualized analysis, could be operationalized into illogical rules such as the three implemented in the Corticon rule sheet in figure 1. The implementation of these three basic rules likewise illustrates that the visual, readable quality of a rules-based model can better enable implementers of clinical informatics tooling to spot errors in logic.

Indeed, while the less intelligible model was more accurate, the very fact that it was less intelligible made it vulnerable to deriving misleading conclusions due to the absence of necessary context, whether clinical or otherwise. Intelligibility is thus a distinct requirement for useful analyses of clinical data for decision support, as well as for process automation within healthcare more generally.

Figure 1

For use cases where it’s less immediately obvious to users that the business logic they are tasked with implementing appear counterproductive or outright harmful, the value in leveraging proven, code-free tools like Corticon is more pronounced.

In contrast to other decision automation tools that also provide an entirely intelligible, code-free interface, Corticon doesn’t require that rule modelers catch logical issues like this, nor wait until the testing phase of development to reveal undesired outcomes of the rules. Corticon Studio, the design-time environment, automatically identifies problems with the logic of rules that users create—for example, in the same example above, the Conflict Checker will reveal the conflict between rules 1 and 3, while the Completeness Checker will show that the rule modeler has not defined any action for cases where a patient is neither asthmatic nor pneumonic. As demonstrated by the example shown with Corticon, intelligibility should bolster—not come at the expense of— implementer productivity and ease of use.

Amid the widespread evolution in healthcare IT towards the development of standards for the communication of healthcare data between applications through FHIR, this blog series explores some of the overarching value propositions derived from the standard. As with other domain-specific yet openly available technical standards, FHIR leverages software design concepts not specific to healthcare, such as intelligibility.

Through examples and walkthroughs built with Corticon and other Progress tooling, we’ll capture how these design concepts are implemented in other domains, thereby informing and driving innovation within healthcare.

Learn more about Corticon