2023, Jacob, Christine, Lindeque, Johan Paul, Klein, Alexander, Ivory, Chris, Heuss, Sabina, Peter, Marc K.

Background: Technological advancements have opened the path for many technology providers to easily develop and introduce eHealth tools to the public. The use of these tools is increasingly recognized as a critical quality driver in health care; however, choosing a quality tool from the myriad of tools available for a specific health need does not come without challenges. Objective: This review aimed to systematically investigate the literature to understand the different approaches and criteria used to assess the quality and impact of eHealth tools by considering sociotechnical factors (from technical, social, and organizational perspectives). Methods: A structured search was completed following the participants, intervention, comparators, and outcomes framework. We searched the PubMed, Cochrane, Web of Science, Scopus, and ProQuest databases for studies published between January 2012 and January 2022 in English, which yielded 675 results, of which 40 (5.9%) studies met the inclusion criteria. The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines and the Cochrane Handbook for Systematic Reviews of Interventions were followed to ensure a systematic process. Extracted data were analyzed using NVivo (QSR International), with a thematic analysis and narrative synthesis of emergent themes. Results: Similar measures from the different papers, frameworks, and initiatives were aggregated into 36 unique criteria grouped into 13 clusters. Using the sociotechnical approach, we classified the relevant criteria into technical, social, and organizational assessment criteria. Technical assessment criteria were grouped into 5 clusters: technical aspects, functionality, content, data management, and design. Social assessment criteria were grouped into 4 clusters: human centricity, health outcomes, visible popularity metrics, and social aspects. Organizational assessment criteria were grouped into 4 clusters: sustainability and scalability, health care organization, health care context, and developer. Conclusions: This review builds on the growing body of research that investigates the criteria used to assess the quality and impact of eHealth tools and highlights the complexity and challenges facing these initiatives. It demonstrates that there is no single framework that is used uniformly to assess the quality and impact of eHealth tools. It also highlights the need for a more comprehensive approach that balances the social, organizational, and technical assessment criteria in a way that reflects the complexity and interdependence of the health care ecosystem and is aligned with the factors affecting users’ adoption to ensure uptake and adherence in the long term.

2020, Schmalz, Oliver, Jacob, Christine, Ammann, Johannes, Liss, Blasius, Iivanainen, Sanna, Kammermann, Manuel, Koivunen, Jussi, Klein, Alexander, Popescu, Razvan Andrei

Background: Cancer immunotherapy (CIT), as a monotherapy or in combination with chemotherapy, has been shown to extend overall survival in patients with locally advanced or metastatic non-small cell lung cancer (NSCLC). However, patients experience treatment-related symptoms that they are required to recall between hospital visits. Digital patient monitoring and management (DPMM) tools may improve clinical practice by allowing real-time symptom reporting. Objective: This proof-of-concept pilot study assessed patient and health care professional (HCP) adoption of our DPMM tool, which was designed specifically for patients with advanced or metastatic NSCLC treated with CIT, and the tool’s impact on clinical care. Methods: Four advisory boards were assembled in order to co-develop a drug- and indication-specific CIT (CIT+) module, based on a generic CIT DPMM tool from Kaiku Health, Helsinki, Finland. A total of 45 patients treated with second-line single-agent CIT (ie, atezolizumab or otherwise) for advanced or metastatic NSCLC, as well as HCPs, whose exact number was decided by the clinics, were recruited from 10 clinics in Germany, Finland, and Switzerland between February and May 2019. All clinics were provided with the Kaiku Health generic CIT DPMM tool, including our CIT+ module. Data on user experience, overall satisfaction, and impact of the tool on clinical practice were collected using anonymized surveys—answers ranged from 1 (low agreement) to 5 (high agreement)—and HCP interviews; surveys and interviews consisted of closed-ended Likert scales and open-ended questions, respectively. The first survey was conducted after 2 months of DPMM use, and a second survey and HCP interviews were conducted at study end (ie, after ≥3 months of DPMM use); only a subgroup of HCPs from each clinic responded to the surveys and interviews. Survey data were analyzed quantitatively; interviews were recorded, transcribed verbatim, and translated into English, where applicable, for coding and qualitative thematic analysis. Results: Among interim survey respondents (N=51: 13 [25%] nurses, 11 [22%] physicians, and 27 [53%] patients), mean rankings of the tool’s seven usability attributes ranged from 3.2 to 4.4 (nurses), 3.7 to 4.5 (physicians), and 3.7 to 4.2 (patients). At the end-of-study survey (N=48: 19 [40%] nurses, 8 [17%] physicians, and 21 [44%] patients), most respondents agreed that the tool facilitated more efficient and focused discussions between patients and HCPs (nurses and patients: mean rating 4.2, SD 0.8; physicians: mean rating 4.4, SD 0.8) and allowed HCPs to tailor discussions with patients (mean rating 4.35, SD 0.65). The standalone tool was well integrated into HCP daily clinical workflow (mean rating 3.80, SD 0.75), enabled workflow optimization between physicians and nurses (mean rating 3.75, SD 0.80), and saved time by decreasing phone consultations (mean rating 3.75, SD 1.00) and patient visits (mean rating 3.45, SD 1.20). Workload was the most common challenge of tool use among respondents (12/19, 63%). Conclusions: Our results demonstrate high user satisfaction and acceptance of DPMM tools by HCPs and patients, and highlight the improvements to clinical care in patients with advanced or metastatic NSCLC treated with CIT monotherapy. However, further integration of the tool into the clinical information technology data flow is required. Future studies or registries using our DPMM tool may provide insights into significant effects on patient quality of life or health-economic benefits.