On the ethics of machine learning applications in clinical neuroscience

By Philipp Kellmeyer

Dr. med. Philipp Kellmeyer, M.D., M.Phil. (Cantab) is a board-certified neurologist working as postdoctoral researcher in the Intracranial EEG and Brain Imaging group at the University of Freiburg Medical Center, German. His current projects include the preparation of a clinical trial for using a wireless brain-computer interface to restore communication in severely paralyzed patients. In neuroethics, he works on ethical issues of emerging neurotechnologies. He is a member of the Rapid Action Task Force of the International Neuroethics Society and the Advisory Committee of the Neuroethics Network.

What is machine learning, you ask? 

As a brief working definition up front: machine learning refers to software that can learn from experience and is thus particularly good at extracting knowledge from data and for generating predictions [1]. Recently, one particularly powerful variant called deep learning has become the staple of much of recent progress (and hype) in applied machine learning. Deep learning uses biologically inspired artificial neural networks with many processing stages (hence the word "deep"). These deep networks, together with the ever-growing computing power and larger datasets for learning, now deliver groundbreaking performances at many tasks. For example, Google’s AlphaGo program that comprehensively beat a Go champion in January 2016 uses deep learning algorithms for reinforcement learning (analyzing 30 million Go moves and playing against itself). Despite these spectacular (and media-friendly) successes, however, the interaction between humans and algorithms may also go badly awry.

The software engineers who designed ‘Tay,’ the chatbot based on machine learning, for instance, surely had high hopes that it may hold its own on Twitter’s unforgiving world of high-density human microblogging. Soon, however, these hopes turned to dust when - seemingly coordinated - interactions between Twitter users and Tay turned the ideologically blank slate of a program into a foul display of racist and sexist tweets [2].

Image courtesy Wikimedia

These examples reflect diverse efforts to create more and more “use-cases” for machine learning such as predictive policing (using machine learning to proactively identify potential offenders) [3], earthquake prediction [4], self-driving vehicles [5], autonomous weapons systems [6], or even for creative purposes like the composition of Beatles-like songs or lyrics. Here, I focus on some aspects of machine learning applications in clinical neuroscience that, in my opinion, warrant particular scrutiny.

Machine learning applications in clinical neuroscience 

In recent years, leveraging computational methods for the modeling of disorders has become a particularly fruitful strategy for research in neurology and psychiatry [7], [8].  In clinical neuroimaging, for example, machine learning algorithms have been shown to detect morphological brain changes typical of Alzheimer’s dementia [9], identify brain tumor types and grades [10], predict language outcome after stroke [11], or distinguish typical from atypical Parkinson’s syndromes [12]. In psychiatric research, examples for applying machine learning are the prediction of outcomes in psychosis, [13] and the persistence and severity of depressive symptoms.14 More generally, most current applications follow one of the following rationales: (1) to distinguish between healthy and pathological tissue in images, (2) to distinguish between different variants of conditions, (3) to make predictions on the outcome of particular conditions. While these are potentially helpful tools for assisting doctors in clinical decision-making, they are not a routinely used in clinics yet. It is safe to predict, however, that machine learning based programs for automated image processing, diagnosis, and outcome prediction will play a significant role in the near future.

Some of the ethical challenges 

One area in which intelligent systems may create ethical challenges is their impact on autonomy and accountability of clinical decision-making. As long as machine learning software for computer-aided diagnosis merely assist radiologists and the clinician keeps the authority over clinical decision-making, it would seem that there is no profound conflict between autonomy and accountability. If, on the other hand, decision-making was to be relegated to the intelligent system, to any degree whatsoever, we may indeed face the problem of an “accountability gap” [15]. After all, who (or what) would need to be held accountable in the case of a grave system error resulting in misdiagnosis: the software engineer, the company or the regulatory body that allowed the software to enter the clinic?

Image courtesy of Vimeo

Another problem may arise from the potential for malicious exploitation of an adaptive, initially “blank”, machine learning algorithm - as in the case of Tay, the chatbot. A machine learning software in its initial, untrained state would perhaps be particularly vulnerable for exploitation by interacting users with malicious intents. Nevertheless, it still requires some leap of the imagination to go from collectively trolling a chatbot to become racist or sexist, to scenarios referred to as “neurohacking” in which hackers viciously exploit computational weaknesses of neurotechnological devices for improper purposes. Despite this potential for misuse, the adaptiveness of modern machine learning software may, with appropriate political oversight and regulation, work in favor of developing programs that are capable of ethically sound decision-making.

While intelligent systems based on machine learning software perform increasingly more complex tasks, designing a “moral machine” [16]  (also see previous discussion on blog here)– a computer program with a conscience if you will – alas remains elusive. A rigid set of algorithms will most likely perform poorly in the face of uncertainty, in ethically ambiguous or conflicting scenarios, and will not improve its behavior through its experiences. From an optimistic point of view, the “innate” learning capabilities of machine learning may enable software to develop ethically responsible behavior if given appropriate data sets for learning. For example, having responsible and professionally trained humans interact and train with intelligent systems – “digital parenting” – may enhance the moral conduct of machine learning software and immunize it against misuse [17].

While the limited scope here precludes an in-depth reconstruction of this debate, I encourage you to ponder how the extent of and relationship between autonomy, intentionality, and accountability, when exhibited by an intelligent system, may influence our inclination to consider it a moral agent. Meanwhile, one interesting ancillary benefit that arises from this increasing interest in teaching ethics to machines is that we study the principles of human moral reasoning and decision-making much more intensely [18].

 Suggestions for political regulation and oversight of machine learning software 

To prevent maladaptive system behavior and malicious interference, close regulatory legislation and oversight is necessary which appreciates the complexities of machine learning applications in medical neuroscience. In analogy to ethical codes for the development of robotic systems – the concept of “responsible robotics” [19] – I would emphasize the need for such an ethical framework to include non-embodied software – “responsible algorithmics,” if you will. From the policy-making perspective, the extent of regulatory involvement in developing intelligent systems for medical applications should be proportionate to the degree of autonomous system behavior and potential harm caused by these systems. We may also consider whether the regulatory review process for novel medical applications based on machine learning should include a specialized commission containing experts in clinical medicine, data and computer science, engineering, and medical ethics.

Instead of merely remaining playful children of the Internet age we may eventually grow up to become “digital parents”, teaching intelligent systems to behave responsibly and ethically – just as we would with our actual children.

Acknowledgments 

I thank Julia Turan (Science Communicator, London, @JuliaTuran) and the editors of The Neuroethics Blog for valuable discussions of the text and editing. I also thank Robin Schirrmeister (Department of Computer Science, University of Freiburg) for clarifications and discussions on machine learning. Remaining factual and conceptual shortcomings are thus entirely my own.

References 



1. Russell, S. & Norvig, P. Artificial Intelligence: A Modern Approach. (Prentice Hall, 2013).



2. Staff & agencies. Microsoft ‘deeply sorry’ for racist and sexist tweets by AI chatbot. The Guardian (2016).



3. Lartey, J. Predictive policing practices labeled as ‘flawed’ by civil rights coalition. The Guardian (2016).



4. Adeli, H. & Panakkat, A. A probabilistic neural network for earthquake magnitude prediction. Neural Netw. 22, 1018–1024 (2009).



5. Surden, H. & Williams, M.-A. Technological Opacity, Predictability, and Self-Driving Cars. (Social Science Research Network, 2016).



6. Thurnher, J. S. in Targeting: The Challenges of Modern Warfare (eds. Ducheine, P. A. L., Schmitt, M. N. & Osinga, F. P. B.) 177–199 (T.M.C. Asser Press, 2016).



7. Maia, T. V. & Frank, M. J. From reinforcement learning models to psychiatric and neurological disorders. Nat. Neurosci. 14, 154–162 (2011).



8. Fletcher, P. C. & Frith, C. D. Perceiving is believing: a Bayesian approach to explaining the positive symptoms of schizophrenia. Nat. Rev. Neurosci. 10, 48–58 (2009).



9. Li, S. et al. Hippocampal Shape Analysis of Alzheimer Disease Based on Machine Learning Methods. Am. J. Neuroradiol. 28, 1339–1345 (2007).



10. Zacharaki, E. I. et al. Classification of brain tumor type and grade using MRI texture and shape in a machine learning scheme. Magn. Reson. Med. 62, 1609–1618 (2009).



11. Saur, D. et al. Early functional magnetic resonance imaging activations predict language outcome after stroke. Brain 133, 1252–1264 (2010).



12. Salvatore, C. et al. Machine learning on brain MRI data for differential diagnosis of Parkinson’s disease and Progressive Supranuclear Palsy. J. Neurosci. Methods 222, 230–237 (2014).



13. Young, J., Kempton, M. J. & McGuire, P. Using machine learning to predict outcomes in psychosis. Lancet Psychiatry 3, 908–909 (2016).



14. Kessler, R. C. et al. Testing a machine-learning algorithm to predict the persistence and severity of major depressive disorder from baseline self-reports. Mol. Psychiatry 21, 1366–1371 (2016).



15. Kellmeyer, P. et al. Effects of closed-loop medical devices on the autonomy and accountability of persons and systems. Camb. Q. Healthc. Ethics (2016).



16. Wallach, W. & Allen, C. Moral Machines: Teaching Robots Right from Wrong. (Oxford University Press, 2008).



17. Floridi, L. & Sanders, J. W. On the Morality of Artificial Agents. Minds Mach. 14, 349–379.



18. Skalko, J. & Cherry, M. J. Bioethics and Moral Agency: On Autonomy and Moral Responsibility. J. Med. Philos. 41, 435–443 (2016).



19. Murphy, R. R. & Woods, D. D. Beyond Asimov: The Three Laws of Responsible Robotics. IEEE Intell. Syst. 24, 14–20 (2009).

Want to cite this post?



Kellmeyer, P. (2016). On the ethics of machine learning applications in clinical neuroscience. The Neuroethics Blog. Retrieved on , from http://www.theneuroethicsblog.com/2016/11/on-ethics-of-machine-learning.html

Big data and privacy on the Web: how should human research be conducted on the Internet?

“They said, ‘You can’t mess with my emotions. It’s like messing with me. It’s mind control.'” That’s what Cornell communication and information science professor Jeffrey T. Hancock reported in a recent New York Times article about the public outcry over the now infamous Facebook emotional manipulation study (read on for details). Hancock was surprised and dismayed over the response. He sees the advent of massive-scale sociology and psychology research on the Internet as a “new era” and he has a point. The days of mostly relying on college students as research subjects may be coming to an end. But how should research be conducted in this new online setting? Is it even appropriate to use data from web sites as it is collected now with little, if any, user knowledge and informed consent existing only in the form of privacy policies that nobody reads?¹ In this post I argue that the Internet is not the Wild West and therefore internet-based research should not be allowed to side step established practices of informed consent. Furthermore, significant changes must be made so that these new research opportunities are maximized in the best way possible.

Via Linkedin

 Earlier this year Facebook, the social network with well over 1 billion users, found itself in hot water after publishing a study in collaboration with academic researchers (including Hancock above) that sought to measure “emotional contagion” online.²

In January of 2012 Facebook researchers altered the News Feed content of nearly 700,000 users without their knowledge or explicit consent. Users in a control group had random posts withheld from their News Feeds, irrespective of emotional content, whereas others either had some posts with a positive valence removed or some with a more negative tone hidden. They found that overall, omitting News Feed content – either negative or positive – seemed to affect the emotional valence of subsequent posts. And all those Facebook users who saw fewer emotion-tinged posts subsequently posted fewer updates. In effect, Facebook researchers were successfully able to manipulate the emotions of hundreds of thousands of users simply by altering their News Feed content. The researchers were able to study a phenomenon – emotional contagion – which can be very difficult to study in person due to a variety of potential experimental confounds and they were also able to achieve extraordinary statistical power. But by not explicitly asking for consent many users felt well, used.



In the paper, the authors state that would-be subjects gave informed consent for this research when creating an account. But this may not be true according to a report, which alleges that the word “research” was actually added to the privacy policy months after the study. Anyway, in my case, consent would have been given in the summer of 2005, nearly a decade ago. Even if I did agree to participate in research at that moment in time, I certainly don’t remember it and I could not have had any idea what would be possible on the Internet of 2014. Would you provide your consent today for a study of your Internet behavior in January of 2023? The journal that published the study, Proceedings of the National Academy of Sciences, updated the article with an attached “Editorial Expression of Concern” wherein the Editor-in-Chief Inder M. Verma noted that, as a private company, Facebook is not beholden to the US Government’s Department of Health and Human Services “Common Rule”, which guides human research best practices and advises that, in addition to documented informed consent, subjects should have the ability to opt out of a trial. This, however, is where journals need to step in to enforce ethical standards regardless of the research setting.



In another example, Christian Rudder, one of the founders of the free online dating site OkCupid.com, drew fire recently for revealing the details of an internal study. The website essentially told users who appeared to be poorly matched that they were in fact well-suited and vice versa. His reaction to it all, which may be indicative of the current gulf between traditional brick-and-mortar institution researchers and Silicon Valley entrepreneurs (who may, pardon the pun, have their heads in the cloud) was, effectively, “what’s the big deal?”



The prevailing business model of Web-based companies – offering services for free in exchange for information which they then hope to monetize – inherently disagrees with the ethos of human research. Skipping (for the moment) the issue of informed consent, researchers should first and foremost avoid harming the study participants physically, emotionally or otherwise and should endeavor to protect their subjects’ privacy rather than feed off of it. There does however seem to be an important difference between studying data that is passively collected from users – who should expect that they are forfeiting some amount of privacy in order to get a free service like Facebook in return – and actively manipulating the information that users see and observing their responses.

Via EyeWire

These studies are not limited to social networks and dating sites, however and are quite relevant to neuroscience. Lumosity.com, perhaps the largest brain training website with more than 50 million users around the world, has embarked on the Human Cognition Project. Preliminary results from this project were published in a proof-of-concept study that had N’s well over 100,000.³ However, Lumosity (which I have written about previously on this blog) is very up-front about the Project and seems to be tapping into a widespread desire among the public to get involved in neuroscience and cognitive science research.



Academic researchers are taking advantage of crowdsourcing in neuroscience as well. MIT computational neuroscientist Sebastian Seung has found a solution to the image analysis bottleneck that was slowing down the progress of his group’s research. He developed Eyewire which, in their words, is “a game to map the brain...Anyone can play and you need no scientific background. Over 130,000 people from 145 countries already do. Together we are mapping the 3D structure of neurons; advancing our quest to understand ourselves.” This work has already produced a milestone paper in Nature.⁴



The landscape of Internet research is rapidly changing and, while it certainly presents a number of ethical challenges in terms of privacy and informed consent, there is vast potential for breakthrough studies – of a kind that have never been possible before – to answer important questions. The first question, though, is how to create a safe, effective, environment for research in a setting that is increasingly seen as insecure and where personal information is the most common currency. A necessary first step is to provide would-be subjects with the choice to opt out of a study. In addition, Internet privacy expert Helen Nissenbaum suggests there needs to be a fundamental shift in norms for conceptualizing privacy online. It is not reasonable to expect users to read all privacy policies for websites they visit and, even if they could, it is unlikely they could predict all the potential implications of those often nebulous terms.⁵ Moreover, some sites (such as Facebook) have become so widely used and so necessary for modern communication without any comparable competitors, that it could be argued that users do not really have a choice to take their business elsewhere, and are therefore implicitly coerced to participate.



The Internet offers incredible research opportunities, particularly to understand human behavior. But just because participants never actually step into the lab does not necessarily mean they don’t deserve the same protections as those who do. Besides, the literature is filled with clever, elegant experiments designed to prevent the participants from knowing the goals and hypotheses of the study – there is no reason that couldn’t also be done online. A major challenge facing academia is how to best partner with private companies who may be primarily interested in improving website functionality and in so doing collect large datasets that may be of interest to researchers. They must also address the issue of whether it is ethical to use data collected by private companies without real informed consent or the choice to opt out to answer academic questions. Internet connectedness has opened up incredible new opportunities for research into human behavior and cognition as well as for “citizen scientists” to help expedite progress in neuroscience. Now is the time to codify ethical standards for these studies to ensure that growth in this area continues in the best way possible.





 References 



1. Kelley, P. G., Bresee, J., Cranor, L.F., Reeder, R.W. in SOUPS '09 Proceedings of the 5th Symposium on Usable Privacy and Security. (ACM).



2. Kramer, A. D., Guillory, J. E. & Hancock, J. T. Experimental evidence of massive-scale emotional contagion through social networks. Proceedings of the National Academy of Sciences of the United States of America 111, 8788-8790, doi:10.1073/pnas.1320040111 (2014).



3. Sternberg, D. A. et al. The largest human cognitive performance dataset reveals insights into the effects of lifestyle factors and aging. Front Hum Neurosci 7 (2013).



4. Kim, J. S. et al. Space-time wiring specificity supports direction selectivity in the retina. Nature 509, 331-336, doi:10.1038/nature13240 (2014).



5. Nissenbaum, H. A Contextual Approach to Privacy Online. Daedalus 140, 32-48 (2011).





Want to cite this post?




Purcell, R. (2014). Big data and privacy on the Web: how should human research be conducted on the Internet? The Neuroethics Blog. Retrieved on , from http://www.theneuroethicsblog.com/2014/09/big-data-and-privacy-on-web-how-should.html