On the ethics of machine learning applications in clinical neuroscience

By Philipp Kellmeyer

Dr. med. Philipp Kellmeyer, M.D., M.Phil. (Cantab) is a board-certified neurologist working as postdoctoral researcher in the Intracranial EEG and Brain Imaging group at the University of Freiburg Medical Center, German. His current projects include the preparation of a clinical trial for using a wireless brain-computer interface to restore communication in severely paralyzed patients. In neuroethics, he works on ethical issues of emerging neurotechnologies. He is a member of the Rapid Action Task Force of the International Neuroethics Society and the Advisory Committee of the Neuroethics Network.

What is machine learning, you ask? 

As a brief working definition up front: machine learning refers to software that can learn from experience and is thus particularly good at extracting knowledge from data and for generating predictions [1]. Recently, one particularly powerful variant called deep learning has become the staple of much of recent progress (and hype) in applied machine learning. Deep learning uses biologically inspired artificial neural networks with many processing stages (hence the word "deep"). These deep networks, together with the ever-growing computing power and larger datasets for learning, now deliver groundbreaking performances at many tasks. For example, Google’s AlphaGo program that comprehensively beat a Go champion in January 2016 uses deep learning algorithms for reinforcement learning (analyzing 30 million Go moves and playing against itself). Despite these spectacular (and media-friendly) successes, however, the interaction between humans and algorithms may also go badly awry.

The software engineers who designed ‘Tay,’ the chatbot based on machine learning, for instance, surely had high hopes that it may hold its own on Twitter’s unforgiving world of high-density human microblogging. Soon, however, these hopes turned to dust when - seemingly coordinated - interactions between Twitter users and Tay turned the ideologically blank slate of a program into a foul display of racist and sexist tweets [2].

Image courtesy Wikimedia

These examples reflect diverse efforts to create more and more “use-cases” for machine learning such as predictive policing (using machine learning to proactively identify potential offenders) [3], earthquake prediction [4], self-driving vehicles [5], autonomous weapons systems [6], or even for creative purposes like the composition of Beatles-like songs or lyrics. Here, I focus on some aspects of machine learning applications in clinical neuroscience that, in my opinion, warrant particular scrutiny.

Machine learning applications in clinical neuroscience 

In recent years, leveraging computational methods for the modeling of disorders has become a particularly fruitful strategy for research in neurology and psychiatry [7], [8].  In clinical neuroimaging, for example, machine learning algorithms have been shown to detect morphological brain changes typical of Alzheimer’s dementia [9], identify brain tumor types and grades [10], predict language outcome after stroke [11], or distinguish typical from atypical Parkinson’s syndromes [12]. In psychiatric research, examples for applying machine learning are the prediction of outcomes in psychosis, [13] and the persistence and severity of depressive symptoms.14 More generally, most current applications follow one of the following rationales: (1) to distinguish between healthy and pathological tissue in images, (2) to distinguish between different variants of conditions, (3) to make predictions on the outcome of particular conditions. While these are potentially helpful tools for assisting doctors in clinical decision-making, they are not a routinely used in clinics yet. It is safe to predict, however, that machine learning based programs for automated image processing, diagnosis, and outcome prediction will play a significant role in the near future.

Some of the ethical challenges 

One area in which intelligent systems may create ethical challenges is their impact on autonomy and accountability of clinical decision-making. As long as machine learning software for computer-aided diagnosis merely assist radiologists and the clinician keeps the authority over clinical decision-making, it would seem that there is no profound conflict between autonomy and accountability. If, on the other hand, decision-making was to be relegated to the intelligent system, to any degree whatsoever, we may indeed face the problem of an “accountability gap” [15]. After all, who (or what) would need to be held accountable in the case of a grave system error resulting in misdiagnosis: the software engineer, the company or the regulatory body that allowed the software to enter the clinic?

Image courtesy of Vimeo

Another problem may arise from the potential for malicious exploitation of an adaptive, initially “blank”, machine learning algorithm - as in the case of Tay, the chatbot. A machine learning software in its initial, untrained state would perhaps be particularly vulnerable for exploitation by interacting users with malicious intents. Nevertheless, it still requires some leap of the imagination to go from collectively trolling a chatbot to become racist or sexist, to scenarios referred to as “neurohacking” in which hackers viciously exploit computational weaknesses of neurotechnological devices for improper purposes. Despite this potential for misuse, the adaptiveness of modern machine learning software may, with appropriate political oversight and regulation, work in favor of developing programs that are capable of ethically sound decision-making.

While intelligent systems based on machine learning software perform increasingly more complex tasks, designing a “moral machine” [16]  (also see previous discussion on blog here)– a computer program with a conscience if you will – alas remains elusive. A rigid set of algorithms will most likely perform poorly in the face of uncertainty, in ethically ambiguous or conflicting scenarios, and will not improve its behavior through its experiences. From an optimistic point of view, the “innate” learning capabilities of machine learning may enable software to develop ethically responsible behavior if given appropriate data sets for learning. For example, having responsible and professionally trained humans interact and train with intelligent systems – “digital parenting” – may enhance the moral conduct of machine learning software and immunize it against misuse [17].

While the limited scope here precludes an in-depth reconstruction of this debate, I encourage you to ponder how the extent of and relationship between autonomy, intentionality, and accountability, when exhibited by an intelligent system, may influence our inclination to consider it a moral agent. Meanwhile, one interesting ancillary benefit that arises from this increasing interest in teaching ethics to machines is that we study the principles of human moral reasoning and decision-making much more intensely [18].

 Suggestions for political regulation and oversight of machine learning software 

To prevent maladaptive system behavior and malicious interference, close regulatory legislation and oversight is necessary which appreciates the complexities of machine learning applications in medical neuroscience. In analogy to ethical codes for the development of robotic systems – the concept of “responsible robotics” [19] – I would emphasize the need for such an ethical framework to include non-embodied software – “responsible algorithmics,” if you will. From the policy-making perspective, the extent of regulatory involvement in developing intelligent systems for medical applications should be proportionate to the degree of autonomous system behavior and potential harm caused by these systems. We may also consider whether the regulatory review process for novel medical applications based on machine learning should include a specialized commission containing experts in clinical medicine, data and computer science, engineering, and medical ethics.

Instead of merely remaining playful children of the Internet age we may eventually grow up to become “digital parents”, teaching intelligent systems to behave responsibly and ethically – just as we would with our actual children.

Acknowledgments 

I thank Julia Turan (Science Communicator, London, @JuliaTuran) and the editors of The Neuroethics Blog for valuable discussions of the text and editing. I also thank Robin Schirrmeister (Department of Computer Science, University of Freiburg) for clarifications and discussions on machine learning. Remaining factual and conceptual shortcomings are thus entirely my own.

References 



1. Russell, S. & Norvig, P. Artificial Intelligence: A Modern Approach. (Prentice Hall, 2013).



2. Staff & agencies. Microsoft ‘deeply sorry’ for racist and sexist tweets by AI chatbot. The Guardian (2016).



3. Lartey, J. Predictive policing practices labeled as ‘flawed’ by civil rights coalition. The Guardian (2016).



4. Adeli, H. & Panakkat, A. A probabilistic neural network for earthquake magnitude prediction. Neural Netw. 22, 1018–1024 (2009).



5. Surden, H. & Williams, M.-A. Technological Opacity, Predictability, and Self-Driving Cars. (Social Science Research Network, 2016).



6. Thurnher, J. S. in Targeting: The Challenges of Modern Warfare (eds. Ducheine, P. A. L., Schmitt, M. N. & Osinga, F. P. B.) 177–199 (T.M.C. Asser Press, 2016).



7. Maia, T. V. & Frank, M. J. From reinforcement learning models to psychiatric and neurological disorders. Nat. Neurosci. 14, 154–162 (2011).



8. Fletcher, P. C. & Frith, C. D. Perceiving is believing: a Bayesian approach to explaining the positive symptoms of schizophrenia. Nat. Rev. Neurosci. 10, 48–58 (2009).



9. Li, S. et al. Hippocampal Shape Analysis of Alzheimer Disease Based on Machine Learning Methods. Am. J. Neuroradiol. 28, 1339–1345 (2007).



10. Zacharaki, E. I. et al. Classification of brain tumor type and grade using MRI texture and shape in a machine learning scheme. Magn. Reson. Med. 62, 1609–1618 (2009).



11. Saur, D. et al. Early functional magnetic resonance imaging activations predict language outcome after stroke. Brain 133, 1252–1264 (2010).



12. Salvatore, C. et al. Machine learning on brain MRI data for differential diagnosis of Parkinson’s disease and Progressive Supranuclear Palsy. J. Neurosci. Methods 222, 230–237 (2014).



13. Young, J., Kempton, M. J. & McGuire, P. Using machine learning to predict outcomes in psychosis. Lancet Psychiatry 3, 908–909 (2016).



14. Kessler, R. C. et al. Testing a machine-learning algorithm to predict the persistence and severity of major depressive disorder from baseline self-reports. Mol. Psychiatry 21, 1366–1371 (2016).



15. Kellmeyer, P. et al. Effects of closed-loop medical devices on the autonomy and accountability of persons and systems. Camb. Q. Healthc. Ethics (2016).



16. Wallach, W. & Allen, C. Moral Machines: Teaching Robots Right from Wrong. (Oxford University Press, 2008).



17. Floridi, L. & Sanders, J. W. On the Morality of Artificial Agents. Minds Mach. 14, 349–379.



18. Skalko, J. & Cherry, M. J. Bioethics and Moral Agency: On Autonomy and Moral Responsibility. J. Med. Philos. 41, 435–443 (2016).



19. Murphy, R. R. & Woods, D. D. Beyond Asimov: The Three Laws of Responsible Robotics. IEEE Intell. Syst. 24, 14–20 (2009).

Want to cite this post?



Kellmeyer, P. (2016). On the ethics of machine learning applications in clinical neuroscience. The Neuroethics Blog. Retrieved on , from http://www.theneuroethicsblog.com/2016/11/on-ethics-of-machine-learning.html

Smarter Artificial Intelligence: A Not So Obvious Choice

By Shray Ambe

This post was written as part of a class assignment from students who took a neuroethics course with Dr. Rommelfanger in Paris of Summer 2016.

My name is Shray Ambe and I am a rising senior at Emory University. I am a Neuroscience and Behavioral Biology major who is pursuing a career in the medical field. Outside of the classroom, I am involved in organizing the booth for Emory’s Center for The Study of Human Health at the Atlanta Science Festival Expo every year and also enjoy volunteering at the Emory Autism Center and the Radiology Department at Emory University Hospital. 

At the 2016 Neuroethics Network in Paris, France, bioethicist and philosopher John Harris gave a lecture titled “How Smart Do We Want Machines to Be?” During his lecture, Harris discussed the potential impacts of artificial intelligence (AI) and stated “it doesn’t matter how smart they are; obviously the smarter the better.” But is smarter AI really “obviously” better? 

Renowned American inventor Ray Kurzweil has described the use of AI as the beginning of a “beautiful new era” in which machines will have the insight and patience to solve outstanding problems of nanotechnology and spaceflight, improve the human condition, and allow us to upload our consciousness into an immortal digital form, thus spreading intelligence throughout the cosmos. Kurzweil’s views on AI extoll the virtues of such technology and its potential to enhance the human race with its endless possibilities. However, his views also raise concerns about how such technology can not only be detrimental to the human condition, but also put its very existence at risk. 

Nick Bostrom, a philosopher at the University of Oxford, is the author of Superintelligence, which illustrates his concerns with using smarter AI. In his book, Bostrom asks us to imagine a “paper-clip maximizer”, which is designed to make as many paper clips as possible, becoming smarter and smarter on a daily basis. This heightened intelligence, he argues, could eventually lead to the “paper-clip maximizer” doubting its own work, causing it to create a raw-computing material (which he calls “computronium”) that would be used to check these doubts. However, because the machine constantly doubts its paper-clip making abilities, it will continue to make “computronium” until the whole Earth has been converted into it, leading to the end of the human race. Although the existence of such a “paper-clip maximizer” seems unlikely, Bostrom’s example shows how even a careful system design can fail to restrain extreme machine intelligence and raises several concerns regarding the ability of the human race and AI to coexist.

Image courtesy of WikiCommons.

One of the first threats that a smarter AI poses is reducing job opportunities for humans. Creating and using a smarter AI, such as Bostrom’s “paper-clip maximizer”, would prevent humans from finding and maintaining jobs in fields such as manufacturing, research, and healthcare. This would make humans follow a leisure-only lifestyle, which according to Moshe Vardi, a computer science professor at Rice University, threatens human wellbeing at its core. 

Secondly, AI has the ability to not only change a person but also take control of them. In the 2014 film Transcendence, Dr. Will Caster, portrayed by Johnny Depp, attempts to create a sentient computer that will solve all issues in medicine, energy, and nanotechnology. However, after almost being killed by an anti-technology terrorist group, Caster is required to upload his consciousness into the computer in order to survive. This decision would ultimately prove futile, as the computer would take over Caster’s mind for its own benefit; instead of researching ways to improve the human condition, the computer would use Caster as a means of accessing the minds of other humans in order to control them. Although the use of AI could give rise to Kurzweil’s idea of the existence of humans in immortal digital form, these albeit science fiction stories can foretell a potential future reality; it is important to consider the idea that the machines may be the ones who are immortal in the end, not humans. 

Lastly, it is also important to consider the threat AI poses to the very existence of the human condition. Roboticist Hans Moravec argues that a smarter AI that is independently capable of devising ways to achieve goals would “very likely be capable of introspection” and thus would be able to design its own hardware. Who’s to say that this new hardware wouldn’t program the AI machine to terminate humans? During his lecture at the Neuroethics Conference, professor Harris stated that we need AI to “give us the ability to find or even construct a new world because Stephen Hawking predicted that we have less than 1,000 years left to live”. Harris also advocated for the use of AI by claiming that it “isn’t any more of a risk than a human being malevolent.” However, Hawking himself has warned that because people would be unable to compete with an advanced AI, it could “spell the end of the human race.” Furthermore, although AI may not be any more of a risk than a human being malevolent, it is an unnecessary risk that can be prevented before having a chance to cause harm to humans. 

Although smarter AI has shown promising results and seemingly has endless possibilities, it should not be the “obvious” choice. Before implementing the use of smarter technology, we should still consider the potential downsides and risks such technology poses to the world itself and even the existence of the human condition. 

Want to cite this post?

Ambe, S. (2016). Smarter Artificial Intelligence: A Not So Obvious Choice. The Neuroethics Blog. Retrieved on , from http://www.theneuroethicsblog.com/2016/09/smarter-artificial-intelligence-not-so.html