Information science

The challenge now is how to implement artificial intelligence technology safely into clinical practice Artificial intelligence is a branch of computer science that, in broad terms, deals with either decision making or classification. The aim of artificial intelligence is to surpass human cognitive functioning such that automated decisions can be made. Machine learning — an application of artificial intelligence — is commonly used in image recognition. In general, the machine, or algorithm, learns from exposure to a large dataset. Once learning has taken place, the algorithm can be applied to unseen data. The potential advantages of this approach in health care are clear: machines can learn from very large datasets in relatively short time frames and can apply themselves to new data without fatigue or intra‐observer replication error. Machine learning has recently demonstrated remarkable performance in image‐based diagnosis across various medical fields, including ophthalmology, radiology, pathology and dermatology. In dermatology, the primary focus has been on developing machine learning systems that facilitate classification and decision support for skin cancer management. Skin cancer (including melanocytic and keratinocytic malignancy) is the most common cancer in Australia and among Caucasian populations worldwide. Melanoma is responsible for the majority of skin cancer deaths in Australia and has various presentations.1,2 While dermoscopy has improved the accuracy of melanoma diagnosis, significant variability occurs and is largely a function of clinical expertise. Recent studies show that machine learning algorithms have the potential to surpass the diagnostic performance of experts, and the challenge now is how to implement this new technology safely into clinical practice. Although there are a number of machine learning algorithms that could be used in the dermatology setting, convolutional neural networks (CNNs) are the most promising. This is largely because they learn from data without any feature specification, and they are known to exhibit superior performance for image recognition in comparison with other machine learning algorithms.3 The aim of the CNN is to generalise its previously learned knowledge on unseen images beyond the training dataset. There are numerous parameters within a CNN that can be tweaked to maximise algorithm performance. Most of these parameters are adjusted automatically by the algorithm, without user input. Therefore, very little can be known, in principle, about why and how the algorithm reaches any particular decision. Currently, there are efforts underway to reduce the “black box” effect of CNNs. Some commercial software programs coupled to imaging devices will provide the user with comparable lesions to justify the algorithm's output and improve transparency. However, this retrieval system may fail for rare or unseen cases and does not provide a decision‐making process. While the black box phenomenon remains, there are two potentially negative implications for clinical practice: first, clinicians may have difficulty upskilling by following the algorithms’ outputs; and second, there exists the potential for deskilling and underperforming due to an over‐reliance on technology.4,5 The effect of a faulty system has been explored by manipulating a previously trusted algorithm to generate incorrect classifications and found that doctors of all experience levels were susceptible to being misled by the recommendation.5 Algorithm performance is dependent on both the size and quality of the training image dataset and on whether the algorithm is used in situations for which it was intended. Depending on the training set, the device may be limited in its ability to diagnose specific lesions (eg, non‐pigmented), or lesions in certain skin types (eg, darker skin) or sites (eg, scalp or acral). Retrospective image databases used to train algorithms may be associated with bias. In addition, artefacts (eg, hair, dermoscopic gel, air bubbles, rulers, pen markings, reflections) can distract from key features. However, if a CNN is trained on a large enough cohort, it can learn to deal with potential artefacts. Nonetheless, unbiased lesion selection and standardised image capture would invariably improve algorithm performance, and recent advances in three‐dimensional (3D) imaging modalities will enable this.6 Several studies have now shown that CNNs trained on retrospective image data collected at a single time point are capable of classifying skin cancer with sensitivities and specificities equal or superior to that of dermatologists (Box 1),5,7,8,9,11 and clinicians with less experience gain most from AI support under experimental conditions.5 Hypomelanotic and acral melanoma can be more challenging to diagnose clinically,1 and this could potentially present a challenge for automated classification. However, CNNs have achieved greater accuracy for hypopigmented and acral lesions in comparison with human experts, at least in silica.9,11 In addition to clinical images, CNNs have been applied to histopathological images of melanoma and benign naevi with promising results.10 The ground truth for lesion diagnosis The gold standard for melanoma diagnosis is histopathological assessment. However, there exists significant inter‐ and intra‐observer variability in histological diagnostic labels attributed to atypical melanocytic lesions.12 The existence of such variability in diagnoses poses the dilemma of whether the CNN has learnt from the correct set of diagnoses. Consensus diagnoses, if practical, may help overcome this problem. Molecular biomarkers may assist in establishing a diagnosis13 and identifying high risk biology,14 but they require extensive validation before clinical use. Pathologists and clinicians also rely on metadata (age, personal and family history, lesion symptoms, recent change), which may influence diagnostic likelihoods. Importantly, it is possible to incorporate different data types, including metadata, sequential image data coupled with histopathology, to train future CNN algorithms and improve diagnostic discrimination of borderline lesions (Box 2). Use of artificial intelligence for melanoma screening It is well known that the incidence of invasive melanoma in Australia has increased over the past 40 years. In addition, there has been a striking increase in incidence of in situ melanoma over the past decade, from 32 cases per 100 000 population in 2004 to 80 per 100 000 population in 2019, with age‐standardised mortality remaining fairly stable.2 The potential causes for the increase in incidence are complex, and involve a true increase, driven by poor sun exposure practices of individuals born before the SunSmart era, combined with increased awareness, excessive screening, and overdiagnosis. It has recently been estimated that 54% of melanomas (15% of invasive melanomas) are overdiagnosed.15 Artificial intelligence‐assisted targeted screening of high risk individuals is likely to be a more effective strategy to save lives than the current opportunistic approach. With sequential whole‐body image datasets linked to metadata, molecular biomarkers and clinical outcomes, our ability to identify lesions associated with sinister biological potential will improve (Box 2), thereby reducing unnecessary biopsies, minimising overdiagnosis and other potential harms associated with screening. Use of artificial intelligence in clinical practice There are advantages and disadvantages of introducing artificial intelligence at different points in the patient care pathway.16 An artificial intelligence system used as a triaging tool before clinician assessment would enable automated risk stratification of individuals and/or lesions (Box 2). This approach could dramatically improve clinician workload and timely access to specialist care for people requiring urgent attention. Alternatively, artificial intelligence consulted following an examination by the clinician may act as a second opinion to improve diagnostic sensitivity and reduce unnecessary biopsies.5 The latter is more closely aligned with current clinical workflows and therefore likely to be preferred while the field matures. There is potential for over‐reliance on artificial intelligence systems in both scenarios. A secondary support system may provide the clinician with a diagnosis or a management decision. Doctors are more likely to change their minds if they are uncertain of a diagnosis and an algorithm provides a conflicting result.5 It is thus important to consider how an algorithm might convey uncertainty to avoid false guidance. For example, a decision‐support output (eg, excise, monitor or reassure) avoids the diagnostic dilemma of differentiating between melanoma and dysplastic naevi. However, the problem is complex and arguments exist as to why, in many situations, a diagnostic probability output might be more desirable. Safe implementation of new technologies The Therapeutic Goods Administration (TGA) has developed an action plan to improve the processes by which new devices are approved for use in Australia, strengthen monitoring and follow‐up, and provide more information to consumers about the devices they use.17 International collaborations also exist with groups, such as the International Medical Device Regulators Forum, to establish better processes for medical device regulation globally. If software is classified as a medical device (ie, it is intended for diagnosis, prevention, monitoring, treatment or alleviation of disease), it must be registered on the Australian Register of Therapeutic Goods following TGA approval and before distribution within Australia. Consumers and clinicians need to be aware of the intended use of an application or device. There are several smartphone applications available to the general public, with functionality ranging from education to monitoring and tracking to skin lesion classification. Some of these provide skin lesion risk assessment, although they may state that they are not intended to be used as a diagnostic device. There is concern that, if this is not immediately obvious to the consumer, unregistered applications may be used in lieu of seeking medical advice. Unsupervised consumer‐operated diagnostic devices would require careful testing before they can be recommended. Conclusion As clinicians, we need to be aware of the limitations of any diagnostic tool and interpret outputs accordingly. Although the performance of artificial intelligence to date is promising, it remains to be seen how diagnostic devices in dermatology will influence decision making in the clinic and affect patient outcomes. Regardless of the specialty, any new technologies need to be rigorously tested before implementation and monitored after implementation. Ultimately, responsibility for patient care remains with the clinician and, as such, a high level of clinical acumen must be maintained. Nonetheless, artificial intelligence in dermatology is primed to become a powerful tool in skin cancer assessment. Box 1 – Comparison of skin cancer classification tasks by artificial intelligence (AI) systems and dermatologists/pathologists Study AI architecture Images Classification task Training dataset size Test dataset size AI Dermatologists/pathologists Sensitivity Specificity AUC/overall accuracy Sensitivity Specificity AUC/overall accuracy Tschandl5 ResNet34 CNN Clinical (dermoscopic) Benign v malignant v non‐neoplastic skin lesions 10 015 1412 0.81 (0.79–0.83)* 0.92 (0.90–0.93)* 0.73†(0.70–0.76)* 0.80 (0.78–0.83)* 0.80 (0.77‐0.82)* 0.60† (0.57–0.63)*,‡ 0.86 (0.84–0.88§)* 0.88 (0.87–0.90§)* 0.74† (0.71–0.77§)* Esteva7 GoogleNet Inception v3 CNN Clinical (macroscopic, dermoscopic) Benign v malignant v non‐neoplastic skin lesions 129 450 1942 na na 72.1%¶ ± 0.9% na na 66.0%¶ Haenssle8 GoogleNet Inception v4 CNN Clinical (macroscopic, dermoscopic) Benign melanocytic naevi v melanoma > 100 000 100 86.6%** 82.5%** 0.86** 86.6%** 71.3%** 0.79** 88.9%†† 82.5%†† 0.86†† 88.9%†† 75.7%†† 0.82†† Tschandl9 GoogleNet Inception v3 CNN Clinical (macroscopic, dermoscopic) Benign v malignant hypo‐pigmented lesions 13 724 2072 81% 53.5% 0.73 78% 51.3% 0.68 Hekler10 ResNet50 CNN Histopathology Benign naevus v melanoma 595 100 76% 60% na 51.8%‡‡ 66.5%‡‡ na Fujisawa11 GoogleLeNet DCNN Clinical (macroscopic) Benign v malignant skin lesions§§ 4867 1142 96.3% 89.5% 92.4%¶ ± 2.1% na na 85.3%¶ ± 3.7% AUC = area under the curve; na = not applicable. * 95% CI. † Youden statistic. ‡ Clinicians with varied experience and training. § Clinician accuracy with multiclass probabilistic AI support. ¶ Overall accuracy. ** Level I: AI and human readers provided with dermoscopic images only. †† Level II: AI provided with dermoscopic images only, human readers provided with dermoscopic images, macroscopic images and additional clinical information. ‡‡ Pathologist. §§ 52.6% of melanomas in this study were acral. Box 2 – Incorporation of different data types to train future convolutional neural network (CNN) algorithms and improve diagnostic discrimination of borderline lesions AI = artificial intelligence.

Miki Wada · ZongYuan Ge · Stephen J Gilmore · Victoria J Mar

Enhancing clinical trial recruitment through eConsent has potential but needs more evidence of use Consent for medical interventions or clinical research participation currently relies on the use of printed information combined with a conversation with a health care professional, which is largely undocumented. Studies have shown that few participants are truly informed at all using these traditional means, and have demonstrated that recall or comprehension of what was disclosed is poor.1,2,3 Attempts to develop standardised participant information and consent forms (PICFs) that meet ethical requirements have often resulted in longer and more complex documents. While consumers have been engaged to assist with these programs, the purpose of PICFs is still too heavily weighted toward satisfying regulatory requirements rather than patient information needs. Unsurprisingly, data show that, as PICFs get longer, they are less well understood,4,5 and there is evidence that this is one of the reasons why patients do not agree to participate in clinical research.6 eConsent is not simply a conversion of a paper PICF into an electronically delivered version. It also holds the promise of improving participant engagement in clinical trials through a variety of features that include: the use of multimedia tools to enhance comprehension; ready conversion into multiple languages; a means to track consent in a highly portable manner; and the opportunity to provide information in a more convenient way to persons with an inability to attend clinics. The use of eConsent does not replace the opportunity for participants to ask direct questions to their doctor or the investigators. Moreover, in most instances, participants will still be required to make a physical visit to a clinic to receive their treatment, whereupon they can ask questions and confirm their willingness to participate. There are relatively few studies using eConsent. In an early randomised controlled study, there was a preference for eConsent as well as improved comprehension and intention to participate in people assigned to use computer terminals rather than paper to receive information.7 In a more recent study involving people infected with human immunodeficiency virus,8 eConsent was found to be acceptable and had some advantages over paper information sheets. There were a majority of males included in the study (75%), and more than half were African American, with a mix of sexual orientation. Health literacy of participants was the only factor that emerged as having an impact on comprehension; however, the number of participants (n = 20) is too small to draw statistically sound conclusions. A 2013 study tested comprehension and satisfaction when using iPads to deliver information for a neuropathy in chemotherapy study.9 Importantly, the investigators presented the same information in both formats, but the iPad had an initial video outlining the main features of the study. They found that of the 55 patients who took part in the randomised study, there was a statistically significant association with increased comprehension in the group assigned to the iPad. The sample sizes were too small for definitive findings, but of interest was that use of the iPad did not increase likely participation rates (it was slightly lower). All participants advised that the information provided was still too complex regardless of the media used, and that simplified text, diagrams, animations and other ways to enhance comprehension are needed. A recent study reported on the TransCelerate eConsent Initiative, which employed a large survey of 3045 participants and a number of smaller stakeholder consultations.10 While there was general support by potential participants for the use of eConsent, the survey revealed that people living in the European Union had the greatest level of discomfort with it. In this survey, they also found that people were concerned that eConsent might eliminate site/participant discussion regarding participation, even though this is not the case where it has actually been used. In Australia, there has not been widespread use of eConsent to date. To better understand the Australian context, Clinical Trials: Impact and Quality (CT:IQ) — a cooperative funded by MTPConnect, an Australian Government Industry Growth Centres Initiative, using funds from the federal government's Medical Research Future Fund (MRFF) — set out to investigate stakeholder perceptions of eConsent and, therefore, to identify potential actionable insights. Chrysalis Advisory developed a survey that was sent via email to the members of CT:IQ for distribution to the wider clinical trial sector in the first quarter of 2019. A total of 179 participants completed the survey and as we used a snowball methodology, there is no denominator of persons polled. In addition, there were 19 semi‐structured interviews conducted drawn from the CT:IQ membership. The majority of respondents (68%) were women, 75% were aged 40 years or over, and 80% had more than 10 years of working in trials, demonstrating considerable experience in the sector. The full report is available on the website,11 with the questions presented on pages 58–59 of the report. The key findings are summarised in the Box. We specifically surveyed those deploying eConsent at this stage and not the end users because we wished to understand what the sector was already doing and what the perceived barriers and opportunities were. Although only 29.2% of respondents indicated that they had any direct experience with eConsent, our survey revealed that they were overall cautiously positive toward the use of eConsent. An important finding was that there was optimism that use of electronic formats would enable participants to drive the information‐seeking process in a way that best suited their needs. The physical infrastructure, particularly in some public hospitals, was widely held as not being adequate to support eConsent uptake. Wi‐Fi blind spots within hospitals were cited as a major reason for this, as well as difficulties achieving infrastructure updates within the public health system. Respondents recommended that approaches to eConsent should employ technologies that do not rely on expensive infrastructure delivered by health services. In addition, respondents indicated that, ideally, there should be a sector‐wide standard for site information technology infrastructure requirements combined with clear guidance for sponsors to standardise their approaches. A number of interviewees who had worked on trials with eConsent where sponsors had provided devices noted that the devices were clunky and prone to malfunction, which increased overall study time and burdened trial staff. Clinical trial sites often experienced sponsors insisting on their own standards, resulting in unnecessary duplication or incompatibility of instrumentation at sites. Many respondents cited that differences in the use of eConsent platforms and inconsistencies between organisations regarding eConsent compliance (eg, whether participants would be required to sign electronically, or would be able to consent by using technologies such as face recognition, fingerprint identification etc) made it difficult to adjust to the use of eConsent. Greater industry engagement and collaboration may mitigate this barrier by providing stakeholders with frameworks and support to implement eConsent. Furthermore, setting some national guidelines will facilitate the design, regulatory approval and implementation of strategies to adopt eConsent. While some stakeholders identified data security as a risk associated with eConsent, others did not believe security threats were any greater than similar threats to existing digital technologies in use throughout clinical trials and the medical field more broadly. They suggested that when appropriate security systems are in place and data governance risks are managed, stakeholders were not likely to be concerned about data governance risks for eConsent. Using eConsent does not automatically mean that participants will have the ability to provide consent offsite, simply that they have access to the information offsite. This is no different from participants providing wet ink signatures offsite in terms of risk and the fact that a person comes to a clinic and accepts the study treatments is a clear demonstration of consent. Two‐factor authentication processes enabled by eConsent may provide a more robust means to authenticate consent than current paper‐based processes. It was not surprising that eConsent was considered to add a cost burden over and above a paper‐based approach. However, few of the respondents considered the cost savings made through enabling prior reading of relevant documentation and, in particular, the major cost savings for the site and for the participants this could potentially deliver. A respondent from a large cancer centre articulated the potential benefits by outlining how participants from anywhere outside of a 50 km radius of the tertiary centre could avoid additional time needed in the clinic through being able to use eConsent. This centre is piloting a tele‐trial model to deliver trials in non‐tertiary settings and recognises that eConsent is pivotal to enabling this model, which promises to reduce the burden on patients through reducing their need to travel and to ensure that clinical trial participation is more available beyond metropolitan centres. It appears from our survey that Australia is willing but only partially ready to implement eConsent. The pathway forward will require proactive planning, leading and managing organisational change with the creation of practical demonstration cases of the development, delivery and use of eConsent in the clinical trial setting vital to support wider adoption. CT:IQ is now looking at a program to undertake these pilot projects as part of its initiatives to enhance clinical trial capability across Australia and in other jurisdictions. Box – Key findings of the eConsent survey Barrier Finding Problems with using paper‐based information sheets and consent forms 38% of respondents thought paper consent forms were not a problem, 71.5% thought they were too long, and 62% found them too complex 37.4% of respondents thought paper‐based consent impaired participant comprehension 67% of respondents believed eConsent would improve comprehension, although they did not believe that this would necessarily translate into greater recruitment 59.2% of respondents believed there was a significant issue with providing adequate information to people from culturally and linguistically diverse populations and saw eConsent as a solution to this Perception that regulators, HRECs and hospital governance offices will not accept eConsent 40.8% of respondents believed that ethics committees would not approve use of eConsent, 26.8% were unsure 90.5% of respondents found it necessary to have guidelines for use by both researchers and HRECs Patients will not be sufficiently proficient with technology or have access to suitable devices Certain demographics (eg, older people) were considered likely to struggle with eConsent eConsent was likely to be well received by younger generations Health services lack the infrastructure to deliver eConsent 82.7% of respondents identified a lack of IT infrastructure as a critical barrier to overcome 59.2% indicated that the current infrastructure was inadequate, particularly within hospital sites Difficulties with authentication of individuals and data security 46.3% of respondents believed there would be issues with data governance, security and privacy, but 29% of respondents disagreed with this 59.2% of respondents felt that they would lose the ability to ensure that the person signing the eConsent was actually the participant, the remainder were undecided or felt this was not a problem Lack of consistent practice across the sector 67% of respondents identified a lack of standardised guidelines as a significant barrier to success 49.2% of respondents indicated that staff were able to manage eConsent despite the lack of training and standardised guidance eConsent will be more expensive 60.3% of respondents believed that there would be a significant initial cost, which might be a barrier to uptake HRECs = human research ethics committees; IT = information technology.

Nikolajs Zeps · Nicholas Northcott · Leanne Weekes

To the Editor: The coronavirus disease 2019 (COVID‐19) pandemic has required rapid adjustments in health service delivery.1 The Victorian COVID‐19 Cancer Network (VCCN) is a joint initiative of the Victorian Comprehensive Cancer Centre and Monash Partners Comprehensive Cancer Consortium. Through expert groups, the VCCN aims to provide support and advice to clinicians and health care services treating cancer patients during the pandemic. The VCCN Telehealth Expert Working Group conducted a survey to understand the barriers and enablers to the rapid adoption of telehealth in health services during the first week of April 2020. Seventeen cancer services from across metropolitan and regional Victoria and Tasmania responded. Notably, all respondent cancer services had implemented some form of telehealth since the pandemic. Healthdirect, the Victorian Department of Health and Human Services’ supported telehealth platform, was used in 40% of services, with 25% using phone only and others using platforms such as Skype, FaceTime and doxy.me. With the unprecedented increase in the uptake of telehealth,2 there is a tremendous opportunity to integrate telehealth into routine practice, potentially improving inequities and inefficiencies in the delivery of cancer care for suitably selected patients. Our survey results suggest several areas for attention to support telehealth, including the need for further investment in information technology infrastructure across health services and administrative support to facilitate changes in practice and workflow (Box). The survey results also highlight the educational and training needs of consumers and health professionals during telehealth implementation. Aboriginal and Torres Strait Islanders, people from culturally and linguistically diverse backgrounds and of lower socio‐economic status, and older patients may have greater needs and will require additional support from both government and relevant organisations to ensure equity of access to cancer care via telehealth. We strongly advocate the need to establish evidence‐based, patient‐centred and sustainable telehealth in cancer management. Research into the experience of patients and clinicians should be prioritised to ensure the consistent quality of telehealth consultation with face‐to‐face consultation in appropriate clinical circumstances. Box – Barriers to implementing telehealth: survey results

Zee Wan Wong · Hannah L Cross

To the Editor: In their recent article, Ioannidis and Boyack focused on the misuse of author‐ and journal‐based metrics.1 The “predatory and other easy journals” they allude to are becoming increasingly difficult to distinguish2 in a widening continuum of journal quality that is seeing some overlap between predatory journals and indexed (eg, in Web of Science, Scopus or PubMed) journals that are traditionally perceived to be of peer‐review quality and whose scholarly content has been editorially authenticated.3 This increasing overlap between predatory and indexed journals is accentuated by an increasing lack of reproducibility, often revealed through post‐publication peer review of indexed journals.4 Predatory journals may also seek scholarly validation by allowing citation of their papers to infiltrate supposedly reputable databases.5 However, the continued inability to identify such journals invalidates calls to ban such entities or to not cite papers from currently blacklisted predatory journals, as was recently suggested by the International Committee of Medical Journal Editors.6 Increasing retractions in the biomedical literature as a result of post‐publication peer review — which identifies errors and misuses such as the manipulation of citations discussed by Ioannidis and Boyack, including inflated and coercive self‐citation— affect author‐based metrics and journal‐based metrics differently. It is incumbent upon authors, editors and publishers to correct inflated, skewed or distorted author‐ and journal‐based metrics. To achieve this, retractions need to be destigmatised. Moreover, inflated author‐ and journal‐based metrics (eg, H‐index, Journal Impact Factor [Web of Science Group], CiteScore [Elsevier]) need to be adjusted with corrective, but not punitive, measures, to correct for imbalances and unfair rewards that may be associated with the attribution of citations of retracted (and thus potentially invalid) literature.7 Self‐citations that support stated claims are valid, independent of their number, and involve no ethical breaches. However, the misuse of self‐citations to manipulate author‐ and journal‐based metrics, such as citation cartels,8 raises ethical red flags. Independent of the possible ethical parameters of inflated or coercive self‐citation, such metrics can also be adjusted downwards to reflect the more balanced perspective of an author‐ or journal‐based metric.9 If the identity of predatory journals can be clearly determined and unanimously agreed upon, then the journal‐based metrics of valid, indexed scholarly journals that cite such journals should be adjusted accordingly.

Jaime A Teixeira da Silva

To the Editor: Before the coronavirus disease 2019 (COVID‐19) pandemic, we had been thinking about how best to re‐imagine our university medical program to enhance student experience and learning outcomes. Globally, questions have been raised regarding the utility and format of the pre‐clinical content taught in medical programs in the junior years,1 particularly lectures, which have increasingly low attendance rates. There is emerging evidence that blended approaches to education meet the connectivity, flexibility and interactivity expectations of learners,2 and have potential to combine the best of both online and face‐to‐face teaching. Packaging content in digestible chunks, combined with active learning activities online such as adaptive tutorials, discussions and reflections, results in more meaningful educational experiences for students than didactic lectures.3,4 The COVID‐19 pandemic forced a rapid transition to entirely online teaching for junior medical students. Even components of clinical teaching (other than physical examination) had to proceed in this format. Despite the pace of this transition, both formal and informal student feedback indicated that students have an extremely high level of satisfaction and engagement with online learning activities. The clinical training components of the program have, by necessity, also become more streamlined. COVID‐19 has forced us to examine all elements of our medical program. This is an opportunity to review the curriculum for future doctors, especially its alignment with the skills and capabilities they will need in their careers. Clearly, we need to facilitate the development of teamwork and communication skills, which will prepare students for effective patient care and multidisciplinary, interprofessional practice. Additionally, we have an obligation to support medical students in developing skills in reflection, adaptive problem solving, leadership and lifelong learning, all of which are needed to adapt to a rapidly changing health care environment.5 Some important aspects of university life, such as such as friendships, personal identity development, exposure to diversity and self‐care skills, will be much harder to achieve in a solely online environment, but as we develop plans to reintroduce elements of face‐to‐face teaching, we need to ensure that these are integrated with, and informed by, the advances made in medical education during the past few months.

Adrienne J Torda · Gary Velan · Vlado Perkovic

To the Editor: Australia's social distancing policies to contain the spread of coronavirus disease 2019 (COVID‐19), caused by the severe acute respiratory syndrome coronavirus 2 (SARS‐CoV‐2), have had social consequences. Social distancing and school disruptions have increased parental responsibilities. There has also been an increased opportunity for parents to use alcohol to cope with increased stress.1 Parents, especially mothers, have taken to social media to share “memes” about needing a drink to survive staying at home (Box). These posts are mostly shared with the aim of creating an online environment for peer support and stress relief, but they risk normalising the use of alcohol as a coping strategy and promoting the false belief that alcohol is good for mental health.2 Alcohol is a central nervous system depressant that may relieve stress in the short term, but regular drinking increases psychological distress and the risk of alcohol‐related harm.3 A review of recent advertising complaints indicated that some alcohol advertisers have been quick to capitalise on COVID‐19.4 An investigation of a social media account found an average of one alcohol advertisement every 35 seconds, with themes of easy access without leaving home (58%), buy more (35%), drink during COVID‐19 (24%), and drink to cope (16%).5 Australia has a regulation system for alcohol advertising, which most people mistakenly believe is government‐funded.6 It is in fact an industry‐funded quasi‐regulatory system that is activated by consumer complaints and lacks systematic independent monitoring.7 Further, regulations do not prevent certain social media platforms from being used by alcohol brands to post advertisements and engage with consumers.8 In light of the alcohol industry's opportunistic advertising through social media, it is questionable how well Australia's regulatory system protects parents and other targeted populations at risk from exposure to constant encouragements to drink during these challenging times. Box – Examples of parental drinking‐related memes during the coronavirus disease 2019 (COVID‐19) lockdown

Janni Leung · Jason Connor · Leanne Hides · Wayne D Hall

Our journal remains a source of trusted information in a world awash with wild beliefs and untrustworthy advice

Nicholas J Talley AC

Objective: To assess the cost‐effectiveness of a computer‐guided quality improvement intervention for primary health care management of cardiovascular disease (CVD) in people at high risk. Design: Modelled cost‐effectiveness analysis of the HealthTracker intervention and usual care for people with high CVD risk, based on TORPEDO trial data on prescribing patterns, changes in intermediate risk factors (low‐density lipoprotein cholesterol, systolic blood pressure), and Framingham risk scores. Participants: Hypothetical population of people with high CVD risk attending primary health care services in a New South Wales primary health network (PHN) of mean size. Intervention: HealthTracker, integrated into health care provider electronic health record systems, provides real time decision support, risk communication, a clinical audit tool, and a web portal for performance feedback. Main outcome measures: Incremental cost‐effectiveness ratios (ICERs): difference in costs of the intervention and usual care divided by number of CVD events averted with HealthTracker. Results: The estimated numbers of major CVD events over five years per 1000 patients at high CVD risk were lower in PHNs using HealthTracker, both for patients with prior CVD events (secondary prevention; 259 v 267 with usual care) and for those without prior events (primary prevention; 168 v 176). Medication costs were higher and hospitalisation costs lower with HealthTracker than with usual care for both primary and secondary prevention. The estimated ICER for one averted CVD event was $7406 for primary prevention and $17 988 for secondary prevention. Conclusion: Modelled cost‐effectiveness analyses provide information that can assist decisions about investing in health care quality improvement interventions. We estimate that HealthTracker could prevent major CVD events for less than $20 000 per event averted. Trial registration (TORPEDO): Australian New Zealand Clinical Trials Registry, ACTRN 12611000478910.

Bindu Patel · David P Peiris · Anushka Patel · Stephen Jan · Mark F Harris · Tim Usherwood · Kathryn Panaretto · Thomas Lung

A false impression of technological panacea may see much needed interventions overlooked and may introduce unintended consequences and risks In the face of coronavirus disease 2019 (COVID‐19) limiting free movement, experts are scrambling to mitigate the profound impact that the disease is having on our lives. For many countries, this approach involves increased testing, isolation, and education about hygiene practices until a vaccine is found. To varying degrees, without much evidence as to their efficacy, countries are turning to technology to solve some of the current challenges.1 Increasingly, smartphone applications (apps) are being contemplated for tracking proximity of people to determine possible sources of transmission, with elements of technological solutionism. Such technical solutions require trust, and without honest and clear information about the possibilities and limitations of technologies, an app's benefits may be undermined by low adoption, or conversely a false impression of a technological panacea may see much needed interventions overlooked. For example, the Australian Government's target of a 40% uptake of the COVIDSafe app may or may not be effective in helping to control the disease, while 60% uptake is supported by independent modelling from the United Kingdom.2 Furthermore, such summary statistics do not clarify to the public the wide range of other factors and assumptions that must be considered in predicting the app's efficacy. Much is being written about the different technological models and whether they trace, track and comply with privacy and human rights frameworks, including whether this information can, in fact, ever be anonymised.3 Fully effective anonymisation is unlikely when collecting data as granular as regular interaction with others in addition to age, gender and postcode demographics, as has been demonstrated by previous attempts to de‐anonymise data.4 If these data are accidentally or deliberately linked with other datasets, such as births in hospitals or the public Myki public transport dataset,5 anonymity is virtually impossible to guarantee. Successful uptake of new technologies requires trust. When adoption is insufficient, collective benefits are not guaranteed. Civil society in the United Kingdom called for clear and comprehensive primary legislation to regulate data processing in symptom tracking and digital contact tracing applications, including with a strict purpose, access and time limitations.6 Such regulation may improve trust. Technology embeds values Even when people are told of the limitations of technology, they may have magical thinking about its capabilities.7,8 In early May 2020, the Australian Government furthered this magical thinking by direct messaging Australians that downloading the COVIDSafe app would help to keep people safe and ease restrictions, linking the two directly and potentially conflating the capability of COVIDSafe. Contact tracing apps may assist in manual tracing, in turn slowing the virus’ spread, but usage of an app does not render the individual protected from infection nor does it guarantee successful tracking without intensive manual efforts. Yet statements by those in authority have made strained assertions about COVIDSafe, likening the use of the app to the use of sunscreen9 or a digital vaccine: “You could think about contact tracing as a digital vaccine with our contact data being the virtual antibodies”.10 Such statements are incorrect representations of the app's capabilities.11 Even the technical details of the app are not immune from false messaging. For example, the app records all Bluetooth contacts, not just those that last 15 minutes or that are within 1.5 m. The filtering occurs after contacts are uploaded. Furthermore, there are some inaccurate statements on the official COVIDSafe website; for example, the frequently asked questions section states that “all information that is stored on the phone is digitally encrypted;” however, metadata, such as the device make and model for each contact, are stored unencrypted.12 Communication must be fact‐based, transparent and consultative, any short term gains in support from the use of emotive and persuasive messaging may be undone when they are ultimately demonstrated to be false. Centralised versus decentralised data collection The fundamental difference between centralised versus decentralised tracking is in who learns what. In the centralised approach, the central authority learns who an infected person has interacted with, whereas this does not occur in the decentralised system. Decentralised systems are no more challenging to implement but they better protect privacy. In a centralised approach (Box 1), such as TraceTogether (Singapore) or COVIDSafe (Australia): encrypted identifiers are issued by the central authority to each device; devices broadcast the encrypted identifiers via Bluetooth, and nearby devices listen for such broadcasts and record any that they receive; if a person tests positive, they report to the central authority all the identifiers they have received within a predetermined timeframe; and the central authority decrypts the identifiers and maps them to the individuals they were issued to and duly notifies them if they are deemed to be at risk. The above is a very high level description and there are many technical challenges in implementing such a system securely.13 In a decentralised approach (Box 2), as proposed by decentralised privacy‐preserving proximity tracing (DP‐3T), Covid Watch, Apple and Google: devices generate random identifiers that are not linked to an individual; identifiers are broadcast via Bluetooth and recorded by nearby devices; a person who tests positive publishes a list of the identifiers they have broadcast; and all apps on user devices download such lists and check if they received positive identifiers so as to identify likely contacts. While there are variations in the details, in the decentralised approach, the central authority does not map identifiers to individuals. Although the distinction between centralised versus decentralised tracking may seem small, from a privacy perspective, there is a significant difference. In the case of COVIDSafe, the identifiers are generated and provided to the phone individually rather than as a daily batch: the central authority can monitor whether the app is being used in at least 2‐hourly increments, and possibly as frequently as every 9 minutes, due to regular checks for new identifiers. Models reflect differing societal priorities. In Germany, where there are legal protections for both individual and group privacy, the decentralised app has been chosen. In fact, it has been suggested that a decentralised smartphone contact tracing system — as contemplated by DP‐3T, Apple, Google, and governments across Europe — would be likely to comply with human rights and data protection laws. In contrast, a centralised smartphone system would pose a greater risk to fundamental rights and would require significantly greater justification to be lawful.6 Even when consent for central data collection has been sought, it is unclear what users are consenting to in the absence of fully open code that includes server‐side code, a clear regulatory framework, and with omissions, such as the COVIDSafe's Privacy Impact Assessment and Privacy Policy failing to mention the collection of the devices’ make and model.14 In comparison, Singapore's TraceTogether is based on the same codebase and its frequently asked questions section notifies of such data collection.15 Efficacy and risks of using Bluetooth Bluetooth Low Energy (BLE) is designed to be a low power communication technology, it was not designed to facilitate range finding. Accurately measuring the distance between two devices based only on the received signal strength is a challenge, with error margins often in the metres.16 The signal strength is relative not absolute, and thus, the scale of the reported values differ by manufacturer. Furthermore, the signal strength is influenced by many external factors, including the angle at which the device is held, whether it is in a pocket or a bag and any objects around or between it and the other device. Whether BLE can deliver the necessary accuracy remains an open question. While the use of Bluetooth avoids direct location tracking, many other risks remain. There are vast networks of Bluetooth beacons distributed around cities, which facilitate location tracking. Security advice is to disable Bluetooth when not in use. While the public might be expected to compromise for the common good, legislation could also move to limit Bluetooth beacons during the crisis. However, the Privacy Amendment (Public Health Contact Information) Act 202017 passed on 14 May provides no such protections.18 It provides an exemption to those accidentally collecting COVIDSafe data as part of a wider collection of non‐COVIDSafe data. This appears to be aimed at protecting commercial tracking, rather than protecting privacy. Legal and social implications are as important as the technical ones Given the many risks of using technology, the contemplation of any technological solutions to alleviate the impacts of COVID‐19 needs to be not only technical but also legal and social. Making the code open for audit provides some technical guard rails, much as providing open and transparent proof of test results ensures that no risks are overseen. But beyond technical questions there are also legal questions, including with whom the data may be shared. A recently published article refers to the multiple legal regimes potentially applicable to the app in Australia, as experts scramble to review the legal protections for individuals using COVIDSafe.19 Enacting emergency measures in the face of catastrophes is easy. Rolling back changes to technology, habits and even culture is far more difficult. If they are to be used, technological tracking solutions must have sunset clauses to ensure that human rights are protected. But even with sunset clauses, the large quantity of data collected are effectively out in the world, where they can be accessed and misused. Protections and limits for these data and their providers need to be contemplated before use, not only to protect individuals but also for group privacy. Increasingly, there is a risk of data being accessed by overseas agencies, which could have an impact on national security. It is vital that the technical, legal and social challenges are addressed in coordination. Any new legislation must be written within the context of existing technological practices, particularly around Bluetooth tracking. Likewise, where technical compromises are made, they must be justified to the public with clear, concise explanations, in a manner that is transparent and open to scrutiny. While many liberties have been curtailed during COVID‐19, all modifications to existing rights are required, under law, to be legal, necessary and proportionate. These same standards apply to the use of technology. Legal protections need to be in place to ensure that rights are protected, including the right to privacy. Without sound legal protections and safeguards, tracing apps will not only fail but will embed values that may not be those that represent the society we wish to be. Box 1 – The centralised approach of contact tracing wherein the central server learns user contact details Box 2 – The decentralised approach to contact tracing wherein no central authority learns user contact details

Kobi Leins · Christopher Culnane · Benjamin IP Rubinstein

The available tools are largely unregulated, and do not reliably guide people to the right care at the right time

Adam G Dunn

To the Editor: The advent of coronavirus disease 2019 (COVID‐19) has generated an unparalleled level of interest from the medical and non‐medical community. As clinician‐scientists, we watch in astonishment at the exponential growth of academic publications in journals. In January 2020, PubMed saw a sharp rise in the number of publications related to COVID‐19, which continues to grow (Box). We could not help but wonder if this has generated a race to publish. Of course, publishing is crucial to help confront one of the most devastating global health issues of the century. However, it is well recognised that external pressures to publish can muddle the intrinsic pursuit for scientific curiosity and excellence,1 and COVID‐19 has certainly provided the incentive for many clinicians and scientists alike to seek rapid publication. This may, unfortunately, fuel competition in the research/publishing field, which was exemplified by the concerning lack of research collaborations when humans were faced with natural disasters,2 including the 2003 severe acute respiratory syndrome coronavirus (SARS‐CoV) outbreak.3 The urgent nature of this situation means a number of preliminary studies and publications on COVID‐19 are fast‐tracked through the peer review process — or not at all — in the hope of rapidly publicising important findings, opinions and experiences. However, hastily penned observations may mislead and do more harm than good. A recent non‐peer‐reviewed publication on a preprint server likening SARS‐CoV‐2 structurally to the human immunodeficiency virus (HIV) was quickly retracted after the scientific community highlighted serious flaws in the study.4 Furthermore, a preliminary study5 supporting the use of hydroxychloroquine as a COVID‐19 treatment prompted a flurry of off‐label use and media attention. The study was later criticised as being too small and biased, and provided insufficient evidence to recommend its use.6 In summary, rapid publishing allows extensive dissemination of knowledge and sharing of experiences; yet the astute clinician needs to keep an open mind and analyse what is being published, for this cannot take the place of rigorous scientific evaluation and best clinical practice. This is a challenging time in the academic world and COVID‐19 will, no doubt, test our abilities to untangle the vast range of literature available. Box – Monthly and cumulative published articles on coronavirus disease 2019 (COVID‐19)* * We conducted an online search in PubMed and included all articles with the terms “coronavirus”, “COVID‐19”, “COVID” and/or “SARS‐CoV‐2”. The information is correct as of 30 April 2020.

Adrian YS Lee · Ming‐Wei Lin

In reply: Lee and Lin raise an important point about the need for caution in interpreting rapidly published articles in the era of coronavirus disease 2019 (COVID‐19). At the Medical Journal of Australia, we are acutely aware of the need to balance rapid dissemination of key data with the need to maintain our usual high standards of quality and accuracy. We have taken the view that in these unprecedented times, rapid sharing of information is critical, but we recognise the risk of errors this infers. In response, we have implemented a preprint and rapid review process for selected manuscripts of an urgent nature (Box). In order to minimise the risk of errors, all manuscripts are carefully reviewed by myself, our team of experienced and medically qualified editors and, where appropriate, our consultant biostatistician, before being selected for preprint in the MJA. Only where the editorial team have a high level of confidence in the validity and importance of the article will it be selected for rapid preprint publication. Before full acceptance of the manuscript to be published online and in print and, in selected cases, before we accept an article for preprint, we organise a rapid double blind peer review followed by revision in line with our usual stringent processes. In these circumstances, we endeavour to have this process completed within 7 days of preprint publication so that any errors can be quickly identified and corrected. We are very grateful to our reviewers who have been very generous in their assistance with this new process. One final check in our process on full publication is review and editing by our experienced scientific and structural editors, who meticulously check all articles for consistency, accuracy and referencing, while finessing them for readability and clarity of presentation — their expertise is invaluable in ensuring published manuscripts are presented accurately and in the best possible light. We acknowledge that contradiction and error may be inevitable during this rapidly evolving situation but would like to assure our readers that at the MJA, when errors occur, they will be rectified in a timely manner and with full transparency. While we are living in a world of rapid change, our commitment to providing Australian health and medical researchers, clinicians and policy makers with the world‐leading general medical journal they deserve stands strong. Box – MJA process for rapid publication of selected coronavirus disease 2019 (COVID‐19)‐related manuscripts* * Timing is indicative and may vary according to the complexity of the manuscript.

Nicholas J Talley

Objectives: To investigate the quality of diagnostic and triage advice provided by free website and mobile application symptom checkers (SCs) accessible in Australia. Design: 36 SCs providing medical diagnosis or triage advice were tested with 48 medical condition vignettes (1170 diagnosis vignette tests, 688 triage vignette tests). Main outcome measures: Correct diagnosis advice (provided in first, the top three or top ten diagnosis results); correct triage advice (appropriate triage category recommended). Results: The 27 diagnostic SCs listed the correct diagnosis first in 421 of 1170 SC vignette tests (36%; 95% CI, 31–42%), among the top three results in 606 tests (52%; 95% CI, 47–59%), and among the top ten results in 681 tests (58%; 95% CI, 53–65%). SCs using artificial intelligence algorithms listed the correct diagnosis first in 46% of tests (95% CI, 40–57%), compared with 32% (95% CI, 26–38%) for other SCs. The mean rate of first correct results for individual SCs ranged between 12% and 61%. The 19 triage SCs provided correct advice for 338 of 688 vignette tests (49%; 95% CI, 44–54%). Appropriate triage advice was more frequent for emergency care (63%; 95% CI, 52–71%) and urgent care vignette tests (56%; 95% CI, 52–75%) than for non‐urgent care (30%; 95% CI, 11–39%) and self‐care tests (40%; 95% CI, 26–49%). Conclusion: The quality of diagnostic advice varied between SCs, and triage advice was generally risk‐averse, often recommending more urgent care than appropriate.

Michella G Hill · Moira Sim · Brennen Mills

We need informative citation metrics that will be less prone to misuse and gaming

John PA Ioannidis · Kevin W Boyack

Patient-centred technology, delivered via mobile phone apps, can dramatically improve patient outcomes

Roy G Beran

The potential of modern mass communication techniques for managing chronic disease should be explored further

Yang Si · Xiaoqiang Xiao · Cai Xia · Jiang Guo · Qiukui Hao · Qianning Mo · Yulong Niu · Hongbin Sun

Chronic fatigue syndrome (CFS) is a prevalent condition affecting about one in 100 patients attending primary care. There is no diagnostic test, validated biomarker, clear pathophysiology or curative treatment. The core symptom of fatigue affects both physical and cognitive activities, and features a prolonged post‐activity exacerbation triggered by tasks previously achieved without difficulty. Although several different diagnostic criteria are proposed, for clinical purposes only three elements are required: recognition of the typical fatigue; history and physical examination to exclude other medical or psychiatric conditions which may explain the symptoms; and a restricted set of laboratory investigations. Studies of the underlying pathophysiology clearly implicate a range of different acute infections as a trigger for onset in a significant minority of cases, but no other medical or psychological factor has been reproducibly implicated. There have been numerous small case–control studies seeking to identify the biological basis of the condition. These studies have largely resolved what the condition is not: ongoing infection, immunological disorder, endocrine disorder, primary sleep disorder, or simply attributable to a psychiatric condition. A growing body of evidence suggests CFS arises from functional (non‐structural) changes in the brain, but of uncertain character and location. Further functional neuroimaging studies are needed. There is clear evidence for a genetic contribution to CFS from family and twin studies, suggesting that a large scale genome‐wide association study is warranted. Despite the many unknowns in relation to CFS, there is significant room for improvement in provision of the diagnosis and supportive care. This may be facilitated via clinician education.

Carolina X Sandler · Andrew R Lloyd

SmartStartAllergy faccilitates monitoring of infant feeding practices in primary care and parent-reported reactions to food

Michael O'Sullivan · Sandra Vale · Richard KS Loh · Jessica Metcalfe · Karin Orlemann · Sandra Salter · Ian Peters · Alan Leeb

The global expansion of wearable technology combined with smartphone access creates new questions and opportunities in the diagnosis and management of cardiac conditions

David Jin · Heath Adams · Anthony M Cocco · William G Martin · Sonny Palmer

Tempering pre-submission expectations and mastering the Coping and reLaxing Mechanisms of senior colleagues are important for junior researchers

Hui‐Chen Han · Anoop Ninan Koshy · Tina Lin · Matias Yudi · David Clark · Andrew W Teh · Omar Farouque

Unsurprisingly, Dr Arthur Conan Doyle included numerous medical allusions in his stories and novels

Jan D Kampmann

Australia needs to develop and implement policies to ensure that data are disaggregated, analysed and presented by sex and gender

The Sex and Gender Sensitive Research Call to Action Group

Despite the immense economic burden of the growing number of people with chronic and end-stage kidney disease, it is still not recognised as one of the five national priority areas in Australia

Wai H Lim · David W Johnson · Stephen P McDonald · Carmel Hawley · Philip A Clayton · Matthew D Jose · Germaine Wong

New health information technologies have the potential to transform the delivery of mental health care

Ian B Hickie

Machine learning has huge potential to enhance clinical decision making, but there are still many limitations

Ian A Scott · David Cook · Enrico W Coiera · Brent Richards