Evaluation of an architecture for intelligent query and exploration of time-oriented clinical data

Susana B. Martins, Yuval Shahar, Dina Goren-Bar, Maya Galperin, Herbert Kaizer, Lawrence V. Basso, Deborah McNaughton, Mary K. Goldstein

Research output: Contribution to journalArticlepeer-review

43 Scopus citations


Objective: Evaluate KNAVE-II, a knowledge-based framework for visualization, interpretation, and exploration of longitudinal clinical data, clinical concepts and patterns. KNAVE-II mediates queries to a distributed temporal-abstraction architecture (IDAN), which uses a knowledge-based problem-solving method specializing in on-the-fly computation of clinical queries. Methods: A two-phase, balanced cross-over study to compare efficiency and satisfaction of a group of clinicians when answering queries of variable complexity about time-oriented clinical data, typical for oncology protocols, using KNAVE-II, versus standard methods: both paper charts and a popular electronic spreadsheet (ESS) in Phase I; an ESS in Phase II. The measurements included the time required to answer and the correctness of answer for each query and each complexity category, and for all queries, assessed versus a predetermined gold standard set by a domain expert. User satisfaction was assessed by the Standard Usability Score (SUS) tool-specific questionnaire and by a "Usability of Tool Comparison" comparative questionnaire developed for this study. Results: In both evaluations, subjects answered higher-complexity queries significantly faster using KNAVE-II than when using paper charts or an ESS up to a mean of 255 s difference per query versus the ESS for hard queries (p = 0.0003) in the second evaluation. Average correctness scores when using KNAVE-II versus paper charts, in the first phase, and the ESS, in the second phase, were significantly higher over all queries. In the second evaluation, 91.6% (110/120) of all of the questions asked within queries of all levels produced correct answers using KNAVE-II, opposed to only 57.5% (69/120) using the ESS (p < 0.0001). User satisfaction with KNAVE-II was significantly superior compared to using either a paper chart or the ESS (p = 0.006). Clinicians ranked KNAVE-II superior to both paper and the ESS. Conclusions: An evaluation of the functionality and usability of KNAVE-II and its supporting knowledge-based temporal-mediation architecture has produced highly encouraging results regarding saving of physician time, enhancement of accuracy of clinical assessment, and user satisfaction.

Original languageEnglish
Pages (from-to)17-34
Number of pages18
JournalArtificial Intelligence in Medicine
Issue number1
StatePublished - 1 May 2008


  • Clinical decision-support systems
  • Human-computer interface
  • Information-visualization
  • Intelligent user interfaces
  • Knowledge-based systems
  • Medical informatics
  • Temporal reasoning

ASJC Scopus subject areas

  • Medicine (miscellaneous)
  • Artificial Intelligence


Dive into the research topics of 'Evaluation of an architecture for intelligent query and exploration of time-oriented clinical data'. Together they form a unique fingerprint.

Cite this