Clinical decision support system
Clinical decision support system (CDSS) is an interactive Expert system Computer Software, which is designed to assist physicians and other health professionals with decision making tasks, such as determining diagnosis of patient data. A working definition has been proposed by Robert Hayward of the Centre for Health Evidence; "Clinical Decision Support systems link health observations with health knowledge to influence health choices by clinicians for improved health care". This definition has the advantage of simplifying Clinical Decision Support to a functional concept. It is a major topic of artificial intelligence in medicine.
- 1 Role & Characteristics
- 2 Effectiveness
- 3 Current U.S. Regulations
- 4 Challenges to Adoption
- 5 Electronic Health Records and CDSS
- 6 Methodological Basis of CDSS
- 7 Examples of CDSS
- 8 See also
- 9 References
- 10 External links
Role & Characteristics
A clinical decision support system has been coined as an “active knowledge systems, which use two or more items of patient data to generate case-specific advice.” This implies that a CDSS is simply a DSS that is focused on using knowledge management in such a way to achieve clinical advice for patient care based on some number of items of patient data.
The main purpose of modern CDSS is to assist clinicians at the point of care. This means that a clinician would interact with a CDSS to help determine diagnosis, analysis, etc. of patient data. Previous theories of CDSS were to use the CDSS to literally make decisions for the clinician. The clinician would input the information and wait for the CDSS to output the “right” choice and the clinician would simply act on that output. The new methodology of using CDSS to assist forces the clinician to interact with the CDSS utilizing both the clinician’s knowledge and the CDSS to make a better analysis of the patients data than either human or CDSS could make on their own. Typically the CDSS would make suggestions of outputs or a set of outputs for the clinician to look through and the clinician officially picks useful information and removes erroneous CDSS suggestions.
There are two main types of CDSS:
An example of how a CDSS might be used by a clinician comes from the subset of CDSS (Clinical Decision Support System), DDSS (Diagnosis Decision Support Systems). A DDSS would take the patients data and propose a set of appropriate diagnoses. The doctor then takes the output of the DDSS and figures out which diagnoses are relevant and which are not.
Another important classification of a CDSS is based on the timing of its use. Doctors use these systems at point of care to help them as they are dealing with a patient, with the timing of use as either pre-diagnoses, during diagnoses, or post diagnoses. Pre-diagnoses CDSS systems are used to help the physician prepare the diagnoses. CDSS used during diagnoses help review and filter the physician’s preliminary diagnostic choices to improve their final results. And post-diagnoses CDSS systems are used to mine data to derive connections between patients and their past medical history and clinical research to predict future events. It has been claimed that decision support will begin to replace clinicians in common tasks in the future.
Features of a Knowledge-Based CDSS
Most CDSS consist of three parts, the knowledge base, inference engine, and mechanism to communicate. The knowledge base contains the rules and associations of compiled data which most often take the form of IF-THEN rules. If this was a system for determining drug interactions, then a rule might be that IF drug X is taken AND drug Y is taken THEN alert user. Using another interface, an advanced user could edit the knowledge base to keep it up to date with new drugs. The inference engine combines the rules from the knowledge base with the patient’s data. The communication mechanism will allow the system to show the results to the user as well as have input into the system.
Features of a non-Knowledge-Based CDSS
CDSS’s that do not use a knowledge base use a form of artificial intelligence called machine learning, which allow computers to learn from past experiences and/or find patterns in clinical data. Two types of non-knowledge-based systems are artificial neural networks and genetic algorithms.
Artificial neural networks or more generally neural networks use nodes and weighted connections between them to analyze the patterns found in the patient data to derive the associations between the symptoms and a diagnosis. This eliminates the need for writing rules and for expert input. However since the system cannot explain the reason it uses the data the way it does, most clinicians don’t use them for reliability and accountability reasons.
Genetic Algorithms are based on simplified evolutionary processes using directed selection to achieve optimal CDSS results. The selection algorithms evaluate components of random sets of solutions to a problem. The solutions that come out on top are then recombined and mutated and run through the process again. This happens over and over until the proper solution is discovered. They are the same as neural networks in that they derive their knowledge from patient data. Non-knowledge-based networks often focus on a narrow list of symptoms like ones for a single disease as opposed to the knowledge based approach which cover many different diseases to diagnosis.
A 2005 systematic review by Garg et al. of 100 studies concluded that CDSSs improved practitioner performance in 64% of the studies. The CDSSs improved patient outcomes in 13% of the studies. Sustainable CDSSs features associated with improved practitioner performance include the following:
- automatic electronic prompts rather than requiring user activation of the system
Garg et al. concluded that the number and methodologic quality of studies have improved from 1973 through 2004.
Another 2005 systematic review (quantitative analysis) of 70 studies by Kawamoto et al. found... "Decision support systems significantly improved clinical practice in 68% of trials." The CDSS features associated with success include the following:
- the CDSS is integrated into the clinical workflow rather than as a separate log-in or screen.
- the CDSS is electronic rather than paper-based templates.
- the CDSS provides decision support at the time and location of care rather than prior to or after the patient encounter.
- the CDSS provides (active voice) recommendations for care, not just assessments.
However, other systematic reviews are less optimistic about the effects of CDSS. Black et al. concludes that "There is a large gap between the postulated and empirically demonstrated benefits of [CDSS and other] eHealth technologies ... their cost-effectiveness has yet to be demonstrated".
Current U.S. Regulations
With the enactment of the American Recovery and Reinvestment Act of 2009 (ARRA), there is a push for widespread adoption of health information technology through the Health Information Technology for Economic and Clinical Health Act (HITECH). Through these initiatives, more hospitals and clinics are integrating Electronic Medical Records (EMRs) and Computerized physician order entry (CPOE) within their health information processing and storage. Consequently, the Institute of Medicine (IOM) promoted usage of health information technology including Clinical Decision Support Systems to advance quality of patient care. The IOM had published a startling report which focused on patient safety crisis in the United States pointing to the incredibly high number of deaths. This statistic gained great attention to the quality of patient care.
With the recent enactment of the HITECH Act included in the ARRA, encouraging the adoption of health IT, more detailed case laws for CDSS and EMRs are still being defined by the Office of National Coordinator for Health Information Technology (ONC) and approved by Department of Health and Human Services (HHS). “Meaningful use” definition is yet to be polished.
Despite the absence of laws, the CDSS vendors would almost certainly be viewed as having a legal duty of care to both the patients who may adversely be affected due to CDSS usage and the clinicians who may use the technology for patient care. Therefore, the duties of care legal regulations are not explicitly defined yet.
With recent effective legislations related to performance shift payment incentives, CDSS are appealing as more attractive.
Challenges to Adoption
Much effort has been put forth by medical institutions and software companies to produce viable CDSSs to cover all aspects of clinical tasks. However, with the complexity of clinical workflows and the demands on staff time high, care must be taken by the institution deploying the support system to ensure that the system becomes a fluid and integral part of the workflow. To this end CDSSs have met with varying amounts of success, while others suffer from common problems preventing or reducing successful adoption and acceptance.
Two sectors of the healthcare domain in which CDSSs have had a large impact are the pharmacy and billing sectors. Pharmacy and prescription ordering systems now do batch-based checking of orders for negative drug interactions and report warnings to the ordering professional. Such systems commonly exist both in clinical settings as well as in more commercial settings, such as in the software used by local or chain pharmacy stores. Another sector of success for CDSS is in billing and claims filing. Since many hospitals rely on Medicare reimbursements to maintain their operational status, systems have been created to help examine both a proposed treatment plan and the current rules of Medicare in order to suggest a plan that attempts to maximize both the care of the patient and the financial needs of the institution.
Other CDSSs that are aimed at the diagnostic tasks have found success but are often very limited in deployment and scope. The Leeds Abdominal Pain System went operational in 1971 for the University of Leeds hospital, and found fantastic levels of success where the CDSS produced a correct diagnosis 91.8% of cases compared to the clinicians’ rating of 79.6%.
Despite the wide range of efforts by institutions to produce and use these systems, widespread adoption and acceptance has still not yet been achieved for most offerings. One large roadblock to acceptance is workflow integration. A tendency to focus only on the functional decision making core of the CDSS exists, causing a deficiency in planning for how the clinician will actually use the product in situ. Often these systems are stand-alone applications, requiring the clinician to cease working on their current report system, switch to the CDSS, input the necessary data, and receive the information. These additional steps break the flow from the clinician’s perspective and cost precious time.
Technical Challenges & Barriers to Implementation
Clinical decision support systems face steep technical challenges in a number of areas. Biological systems are profoundly complicated, and a clinical decision may utilize an enormous range of potentially relevant data. For example, an electronic evidence-based medicine system may potentially consider a patient’s symptoms, medical history, family history and genetics, as well as historical and geographical trends of disease occurrence, and published clinical data on medicinal effectiveness when recommending a patient’s course of treatment.
Clinically, a large deterrent to CDSS acceptance is workflow integration. Inclination to focus only on functional decision making core of the CDSS causes a deficient plan on how the clinician will actually utilize the system in situations. Generally extra steps are required of the clinician which then causes a disruption in workflow affecting efficiency. Generally these systems are stand-alone applications which are not integrated with existing healthcare systems, the clinical user must stop work on the current system, switch to the CDSS, and reenter data necessary into the CDSS that may already exist in another electronic system.
Another source of contention with many medical support systems produces mass amounts of alert. When systems produce high volume of warnings (especially those that do not require escalation), aside from the annoyance, clinicians may pay less attention to warnings, causing potentially critical alerts to be missed.
One of the core challenges facing CDSS is difficulty in incorporating the extensive quantity of clinical research being published on an ongoing basis. In a given year, tens of thousands of clinical trials are published. Currently, each one of these studies must be manually read, evaluated for scientific legitimacy, and incorporated into the CDSS in an accurate way.
In addition to being laborious, integration of new data can sometimes be difficult to quantify or incorporate into the existing decision support schema, particularly in instances where different clinical papers may appear conflicting. Properly resolving these sorts of discrepancies is often the subject of clinical papers itself (see meta-analysis), which often take months to complete.
In order for a CDSS to offer value, it must demonstrably improve clinical workflow or outcome. Evaluation of CDSS is the process of quantifying its value to improve a system’s quality and measure its effectiveness. Because different CDSSs serve different purposes, there is no generic metric which applies to all such systems; however, attributes such as consistency (with itself, and with experts) often apply across a wide spectrum of systems.
The evaluation benchmark for a CDSS depends on the system’s goal: for example, a diagnostic decision support system may be rated based upon the consistency and accuracy of its classification of disease (as compared to physicians or other decision support systems). An evidence-based medicine system might be rated based upon a high incidence of patient improvement, or higher financial reimbursement for care providers.
Electronic Health Records and CDSS
Implementing Electronic Health Records (EHR) was always going to be an inevitable challenge. The reasons behind this challenge is that it is a relatively uncharted area as it is something that has never been done before, thus there is; and will be many issues and complications during the implementation phase of an EHR. This can be seen throughout the numerous studies that have been undertaken. Challenges in implementing electronic health records (EHRs) have received some attention, but less is known about the process of transitioning from legacy EHRs to newer systems. With all of this said, electronic health records are the way of the future for healthcare industry. It is a way to capture and utilise real-time data to provide high-quality patient care, ensuring efficiency and effective use of time and resources. By incorporating EHR and CDSS it has the potential to change the way medicine has been taught and practiced. As it is said that, “the highest level of the EHR is a CDSS”.
Since “clinical decision support systems (CDSS) are computer systems designed to impact clinician decision making about individual patients at the point in time that these decisions are made”, the reasons can be seen why it would be beneficial to have a fully integrated CDSS and EHR.
Even though the benefits can be seen, to fully implement a CDSS within an EHR, it will require significant planning by the healthcare facility/organisation, in order for the purpose of the CDSS to be successful and effective. The success and effectiveness can be measured by the increase in patient care being delivered and reduced adverse events occurring. In addition to this, there would be a saving of time, resources, autonomy and financial benefits to the healthcare facility/organisation 
Benefits of CDSS and EHR
There has always been errors that occur within the healthcare industry, thus trying to minimise them as much as possible in order to provide quality patient care. Four areas that can be addressed with the implementation of CDSS and Electronic Health Records (EHRs), are:
- Medical error
- Medication error
- Adverse drug events
- Lessen error
CDSS will be most beneficial once the healthcare facility is 100% electronic thus simplifying the number of modifications that have to occur to ensure that all the systems are up to date. However, the measurable benefits of clinical decision support systems on physician performance and patient outcomes remain the subject of ongoing research. Systematic reviews of the literature have yielded differing correlations to date.
Barriers to CDSS and EHR
Implementing electronic health records (EHR) in healthcare settings incurs challenges; none more important than maintaining efficiency and safety during rollout  but in order for the implementation process to occur effectively, an understanding of the EHR users’ perspectives is key to the success of EHR implementation projects. In addition to, adoption needs to be actively fostered through a bottom-up, clinical- needs-first approach. This can be said for CDSS too. The main barriers associated with CDSS and EHRs consist of feasibility (cost), poor usability/ integration, uniformity, clinician non-acceptance, alert desensitisation, as well as the key fields of data entry that need to be addressed when implementing a CDSS to avoid potential adverse events from occurring. These include:
→ Correct data is being used
→ All the data has been implemented
→ Current best practice
→ Evidence based
The main areas of concern with moving into a fully integrated EHR system are:
4. Document accuracy and completeness
8. Alert desensitisation
Current stage of progress with EHR especially in Australia, majority of the healthcare facilities is still completely paper-based form, and some are in the transition phase of a form of EHR with either already implemented scanned-EHR or are in the process of converting to the scanned EHRs. The process of gathering clinical data and medical knowledge and putting them into a form that computers can manipulate to assist in clinical decision-support is still in its infancy. Nonetheless there is great potential once EHRs are implemented, taking on board the key areas of concern and the associated barriers, it will allow for successful integration of CDSS and EHR to provide best practice, high quality care to the patient, which is the ultimate goal of healthcare. In saying this, Victoria has attempted to implement EHR across the state with the HealthSMART program, but due to financial costs it has cancelled the project.
South Australia (SA) however is slightly more successful then Victoria in the implementation of an EHR. This maybe due to all public healthcare organisations being centrally run. SA is in the process of implementing “Enterprise patient administration system (EPAS)”. This system is the foundation for all public hospitals and health care sites for an EHR within SA and it is expected that the end of 2014 will have all facilities connected. This will allow for successful integration of CDSS into SA and increase the benefits of the EHR.
Methodological Basis of CDSS
There are many different methodologies that can be used by a CDSS in order to provide support to the health care professional.
The basic components of a CDSS include a dynamic (medical) knowledge base and an inferencing mechanism (usually a set of rules derived from the experts and evidence-based medicine) and implemented through medical logic modules based on a language such as Arden syntax. It could be based on Expert systems or artificial neural networks or both (connectionist expert systems).
The Bayesian network is a knowledge-based graphical representation that shows a set of variables and their probabilistic relationships between diseases and symptoms. They are based on conditional probabilities, the probability of an event given the occurrence of another event, such as the interpretation of diagnostic tests. Bayes’ rule helps us compute the probability of an event with the help of some more readily available information and it consistently processes options as new evidence is presented. In the context of CDSS, the Bayesian network can be used to compute the probabilities of the presence of the possible diseases given their symptoms.
Some of the advantages of Bayesian Network include the knowledge and conclusions of experts in the form of probabilities, assistance in decision making as new information is available and are based on unbiased probabilities that are applicable to many models.
Some of the disadvantages of Bayesian Network include the difficulty to get the probability knowledge for possible diagnosis and not being practical for large complex systems given multiple symptoms. The Bayesian calculations on multiple simultaneous symptoms could be overwhelming for users.
Example of a Bayesian network in the CDSS context is the Iliad system which makes use of Bayesian reasoning to calculate posterior probabilities of possible diagnoses depending on the symptoms provided. The system now covers about 1500 diagnoses based on thousands of findings.
Another example is the DXplain system that uses a modified form of the Bayesian logic. This CDSS produces a list of ranked diagnoses associated with the symptoms.
A third example is SimulConsult, which began in the area of neurogenetics. By the end of 2010 it covered ~2,600 diseases in neurology and genetics, or roughly 25% of known diagnoses. It addresses the core issue of Bayesian systems, that of a scalable way to input data and calculate probabilities, by focusing specialty by specialty and achieving completeness. Such completeness allows the system to calculate the relative probabilities, rather than the person inputting the data. Using the peer-reviewed medical literature as its source, and applying two levels of peer-review to the data entries, SimulConsult can add a disease with less than a total of four hours of clinician time. It is widely used by pediatric neurologists today in the US and in 85 countries around the world.
Artificial Neural Networks (ANN) is a nonknowledge-based adaptive CDSS that uses a form of artificial intelligence, also known as machine learning, that allows the systems to learn from past experiences / examples and recognizes patterns in clinical information. It consists of nodes called neuron and weighted connections that transmit signals between the neurons in a forward or looped fashion. An ANN consists of 3 main layers: Input (data receiver or findings), Output (communicates results or possible diseases) and Hidden (processes data). The system becomes more efficient with known results for large amounts of data.
The advantages of ANN include the elimination of needing to program the systems and providing input from experts. The ANN CDSS can process incomplete data by making educated guesses about missing data and improves with every use due to its adaptive system learning. Additionally, ANN systems do not require large databases to store outcome data with its associated probabilities. Some of the disadvantages are that the training process may be time consuming leading users to not make use of the systems effectively. The ANN systems derive their own formulas for weighting and combining data based on the statistical recognition patterns over time which may be difficult to interpret and doubt the system’s reliability.
Examples include the diagnosis of appendicitis, back pain, myocardial infarction, psychiatric emergencies and skin disorders. The ANN’s diagnostic predictions of pulmonary embolisms were in some cases even better than physician’s predictions. Additionally, ANN based applications have been useful in the analysis of ECG (A.K.A. EKG) waveforms.
A Genetic Algorithm (GA) is a nonknowledge-based method developed in the 1940s at the Massachusetts Institute of Technology based on Darwin’s evolutionary theories that dealt with the survival of the fittest. These algorithms rearrange to form different re-combinations that are better than the previous solutions. Similar to neural networks, the genetic algorithms derive their information from patient data.
An advantage of genetic algorithms is these systems go through an iterative process to produce an optimal solution. The fitness function determines the good solutions and the solutions that can be eliminated. A disadvantage is the lack of transparency in the reasoning involved for the decision support systems making it undesirable for physicians. The main challenge in using genetic algorithms is in defining the fitness criteria. In order to use a genetic algorithm, there must be many components such as multiple drugs, symptoms, treatment therapy and so on available in order to solve a problem. Genetic algorithms have proved to be useful in the diagnosis of female urinary incontinence.
A rule-based expert system attempts to capture knowledge of domain experts into expressions that can be evaluated known as rules; an example rule might read, "If the patient has high blood pressure, he or she is at risk for a stroke." Once enough of these rules have been compiled into a rule base, the current working knowledge will be evaluated against the rule base by chaining rules together until a conclusion is reached. Some of the advantages of a rule-based expert system are the fact that it makes it easy to store a large amount of information, and coming up with the rules will help to clarify the logic used in the decision-making process. However, it can be difficult for an expert to transfer their knowledge into distinct rules, and many rules can be required for a system to be effective.
Rule-based systems can aid physicians in many different areas, including diagnosis and treatment. An example of a rule-based expert system in the clinical setting is MYCIN. Developed at Stanford University by Edward Shortliffe in the 1970s, MYCIN was based on around 600 rules and was used to help identify the type of bacteria causing an infection. While useful, MYCIN can help to demonstrate the magnitude of these types of systems by comparing the size of the rule base (600) to the narrow scope of the problem space.
The Stanford AI group subsequently developed ONCOCIN, another rules-based expert system coded in Lisp in the early 1980s. The system was intended to reduce the number of clinical trial protocol violations, and reduce the time required to make decisions about the timing and dosing of chemotherapy in late phase clinical trials. As with MYCIN, the domain of medical knowledge addressed by ONCOCIN was limited in scope and consisted of a series of eligibility criteria, laboratory values, and diagnostic testing and chemotherapy treatment protocols that could be translated into unambiguous rules. Oncocin was put into production in the Stanford Oncology Clinic.
The methodology behind logical condition is fairly simplistic; given a variable and a bound, check to see if the variable is within or outside of the bounds and take action based on the result. An example statement might be "Is the patient's heart rate less than 50 BPM?" It is possible to link multiple statements together to form more complex conditions. Technology such as a decision table can be used to provide an easy to analyze representation of these statements.
In the clinical setting, logical conditions are primarily used to provide alerts and reminders to individuals across the care domain. For example, an alert may warn an anesthesiologist that their patient's heart rate is too low; a reminder could tell a nurse to isolate a patient based on their health condition; finally, another reminder could tell a doctor to make sure he discusses smoking cessation with his patient. Alerts and reminders have been shown to help increase physician compliance with many different guidelines; however, the risk exists that creating too many alerts and reminders could overwhelm doctors, nurses, and other staff and cause them to ignore the alerts altogether.
Causal Probabilistic Network
The primary basis behind the causal network methodology is cause and effect. In a clinical causal probabilistic network, nodes are used to represent items such as symptoms, patient states or disease categories. Connections between nodes indicate a cause and effect relationship. A system based on this logic will attempt to trace a path from symptom nodes all the way to disease classification nodes, using probability to determine which path is the best fit. Some of the advantages of this approach are the fact that it helps to model the progression of a disease over time and the interaction between diseases; however, it is not always the case that medical knowledge knows exactly what causes certain symptoms, and it can be difficult to choose what level of detail to build the model to.
The first clinical decision support system to use a causal probabilistic network was CASNET, used to assist in the diagnosis of glaucoma. CASNET featured a hierarchical representation of knowledge, splitting all of its nodes into one of three separate tiers:symptoms, states and diseases.
Examples of CDSS
- Clinical Rules
- [Litmusdx ]
- Clinical Informatics
- Electronic medical record (EMR)
- Gello Expression Language
- Health Informatics Society of Australia
- International Health Terminology Standards Development Organisation
- Personal Health Information Protection Act
- "Decision support systems ." 26 July 2005. 17 Feb. 2009 <http://www.openclinical.org/dss.html>.
- Berner, Eta S., ed. Clinical Decision Support Systems. New York, NY: Springer, 2007.
- Khosla, Vinod (4 December 2012). "Technology will replace 80% of what doctors do". CNN. Retrieved 25 April 2013.
- Garg AX, Adhikari NK, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J et al. (2005). "Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review.". JAMA 293 (10): 1223–38. doi:10.1001/jama.293.10.1223. PMID 15755945.
- Kensaku Kawamoto, Caitlin A Houlihan, E Andrew Balas, David F Lobach. (2005). "Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success.". BMJ 330 (7494): 765. doi:10.1136/bmj.38398.500764.8F. PMC 555881. PMID 15767266.
- Black, A.D., J. Car, C. Pagliari, C. Anandan, K. Cresswell, T. Bokun, B. McKinstry, R. Procter, A. Majeed, and A. Sheikh. (2011). "The impact of ehealth on the quality and safety of health care: A systematic overview.". PLoS Medicine 8 (1). doi:10.1371/journal.pmed.1000387.
- Gluud C, Nikolova D (2007). "Likely country of origin in publications on randomised controlled trials and controlled clinical trials during the last 60 years.". Trials 8: 7. doi:10.1186/1745-6215-8-7. PMC 1808475. PMID 17326823.
- Wagholikar, K; Kathy L. MacLaughlin; Thomas M Kastner; Petra M Casey; Michael Henry; Robert A Greenes; Hongfang Liu; Rajeev Chaudhry (2013). "Formative evaluation of the accuracy of a clinical decision support system for cervical cancer screening". Journal of American Medical Informatics Association. pp. 747–759. Retrieved 6 March 2013.
- Zandieh, Stephanie O.; Kahyun Yoon-Flannery; Gilad J. Kuperman; Daniel J. Langsam; Daniel Hyman; Rainu Kaushal (2008). "Challenges to EHR Implementation in Electronic- Versus Paper-based Office Practices". Journal of Global Information Management: 755–761.
- Berner, Eta S.; Tonya J.La Lande (2007). "1". Clinical Decision Support Systems: Theory and Practice (2 ed.). New York: Springer Science and Business Media. pp. 3–22.
- Rothman, Brian; Joan. C. Leonard; Michael. M. Vigoda (2012). "Future of electronic health records: implications for decision support". Mount Sinai Journal of Medicine 79 (6): 757–768.
- Sambasivan, Murali; Pouyan Esmaeilzadeh; Naresh Kumar and Hossein Nezakati (2012). "Intention to adopt clinical decision support systems in a developing country: effect of Physician’s perceived professional autonomy, involvement and belief: a cross-sectional study". BMC Medical Informatics and Decision Making 12: 142–150.
- "TechnologyAdvice EHR Buyer's Guide". TechnologyAdvice. Retrieved May 25, 2014.
- Spellman Kennebeck, Stephanie; Nathan Timm; Michael K Farrell; S Andrew Spooner (2012). "Impact of electronic health record implementation on patient flow metrics in a pediatric emergency department". Journal of the American Medical Informatics Association: 443–447.
- McGinn, Carrie A; Marie-Pierre Gagnon, Nicola Shaw, Claude Sicotte, Luc Mathieu, Yvan Leduc, Sonya Grenier, Julie Duplantie, Anis B Abdeljelil and France Légaré (2012). "Users' perspectives of key factors to implementing electronic health records in Canada: a Delphi study". BMC Medical Informatics & Decision Making 12: 105–118.
- Rozenblum, Ronen; Yeona Jang; Eyal Zimlichman; Claudia Salzberg; Melissa Tamblyn; David Buckeridge; Alan Forster; David W. Bates and Robyn Tamblyn (2011). "A qualitative study of Canada's experience with the implementation of electronic health information technology". Canadian Medical Association Journal: 281–288.
- Berner, Eta S.; Tonya J.La Lande (2007). "4". Clinical Decision Support Systems: Theory and Practice (2 ed.). New York: Springer Science and Business Media. pp. 64–98.
- Gardner, Reed M (April 2004). "Computerized Clinical Decision-Support in Respiratory Care". Respiratory Care 49: 378–388.
- King, Gerry; Catherine O’Donnell, David Boddy, Fiona Smith, David Heaney and Frances S Mair (2012). "Boundaries and e-health implementation in health and social care". BMC Medical Informatics and Decision Making 2012: 100–111.
- Charette, Robert N. "Troubled HealthSMART System Finally Cancelled in Victoria Australia". Retrieved 18 May 2013.
- South Australian Health. "EPAS program update". South Australian Health. Retrieved 15 May 2013.
- Wagholikar, K. "Modeling Paradigms for Medical Diagnostic Decision Support: A Survey and Future Directions". Journal of Medical Systems. Retrieved 27 November 2013.
- ONCOCIN: An expert system for oncology protocol management E. H. Shortliffe, A. C. Scott, M. B. Bischoff, A. B. Campbell, W. V. Melle, C. D. Jacobs Seventh International Joint Conference on Artificial Intelligence, Vancouver, B.C.. Published in 1981.
- Madhumita, Prabhakar (1 April 2013). "The opportunity in affordable healthcare". thesmartceo.in. Retrieved 22 March 2014.
- CDSS from Dr Y Health Informatics Knowledge Base
- Decision support chapter from Coiera's Guide to Health Informatics
- OpenClinical maintains an extensive archive of Artificial Intelligence systems in routine clinical use.
- Robert Trowbridge/ Scott Weingarten. Chapter 53. Clinical Decision Support Systems
- Stanford CDSS