A recent Proceedings of the National Academy of Sciences study revealed the possibility of combining knowledge representation models and collective intelligence (CI) for complex decision-making tasks. This approach was tested on general medical diagnostics.
Study: Automating hybrid collective intelligence in open-ended medical diagnostics. Image Credit: raker/Shutterstock.com
Background
The importance of collective intelligence in making a proper decision has been observed in many domains, including geopolitical forecasting, investment decisions, and medical diagnostics.
The approach has been applied to simple decision-making tasks based on well-defined answer sets. Application of CI for a complex task, such as general medical diagnostics, that involves larger answer sets has been challenging.
Accurate diagnosis has been one of the leading causes of death in the US. An error in diagnosis leads to incorrect treatment, loss of life, morbidity, and inefficient use of scarce resources. Misdiagnosis also reduces trust in the healthcare system.
A complete reliance on algorithmic processes to solve complex decision-making tasks has been associated with a couple of limitations. For instance, human decision-makers are reluctant to completely rely on algorithmic solutions that are generated based on open-ended decision tasks.
Secondly, the computational complexity and the vast space required for the problem could be challenging for domain-agnostic algorithms. As a result, it requires human intervention for optimal solutions.
For high-dimensional problem spaces, humans require a well-guided search process to narrow down the possible solutions. Knowledge engineering approaches help models to structure various solutions in a hierarchical manner.
CI is explored as an opportunity to reduce diagnostic errors based on the intelligence of multiple diagnosticians instead of a single diagnostician. CI can be established by combining independent decisions of decision-makers, group discussions, or market mechanisms, and this approach can boost the accurate diagnosis of a disease.
Not many studies are available regarding the effectiveness of combining general medical diagnostics to make the diagnosis from a very large number of possible diagnoses.
About the study
The study examined whether the collective intelligence approach can accurately diagnose a disease from a large number of potential diagnoses, adopting the wisdom of the crowd approach for medical diagnostics. This approach offers decisions from diagnosticians worldwide without requiring coordination efforts in time or space.
A large dataset on general medical diagnostics was obtained from the Human Diagnosis Project (Human Dx). Human Dx is an online collaborative effort to assist clinicians in making diagnoses by enabling access to the wisdom of the global medical community.
It is an online platform where medical experts can submit and solve patient cases, and it contains general patients’ information, such as their age, gender, and general symptoms, along with clinical findings (e.g., results from physical and laboratory tests).
Cases are removed from the platform if they lack quality or clarity. Human Dx contains cases of varied medical specialties, such as cardiology, endocrinology, dermatology, neurology, and others. Specialists across the world are invited to register on this platform and diagnose the uploaded medical cases. Users receive the correct solution to an uploaded case.
Study findings
The current study analyzed all cases between May 7, 2014, and October 5, 2016 - around 1,572 cases. For each case, ten experienced diagnosticians were randomly sampled.
An automated reproducible, and scalable method was developed. This approach was designed based on a combination of semantic knowledge graphs and the natural language processing (NLP) method. The combination was integrated into a publicly available medical ontology, namely, the SNOMED Clinical Terms ontology (SNOMED CT).
The current study presents an automated method for disease diagnosis using CI. For disease diagnosis, the wisdom of independent medical experts was considered. This CI application for medical diagnostics is beyond binary or multiclass classification or numeric estimation tasks.
A semantic knowledge graph based on SNOMED CT was generated, which enabled automatic aggregation of diagnoses from multiple users. The aggregation of independent responses from multiple users led to a significant improvement in diagnostic accuracy across medical specialties, chief complaints, and tenure levels of users.
This system is fully automated and does not require manual mapping of free-text inputs by experts. It can automatically identify symptoms, unlike previous methods. As a result, this approach overcomes the shortcomings of previous methods and biases, particularly in terms of accuracy, quick processing time, and cost efficiency.
Conclusions
A key strength of the newly developed aggregation and evaluation procedures is that it is completely automated and does not require human intervention. Since this automated method can be implemented in real-time clinical settings, it helps clinicians make accurate diagnoses.
The current study has some limitations, including its representative nature of design. Although a large number of cases were analyzed, these were selected by an expert panel of Human Dx.
Another limitation of the study is that all results are provided in textual data in the English language only. Despite the limitations, this study revealed the significance of the application of CI in the global medical community to reduce misdiagnosis and improve treatment accuracy.
Study reveals AI's potential and pitfalls in medical diagnosis