Research

Publications

LUCID: Exposing Algorithmic Bias through Inverse Design (with C. Mazijn, A. Algaba, J. Danckaert, and V. Ginis)
Proceedings of the AAAI Conference on Artificial Intelligence, forthcoming
AI systems can create, propagate, support, and automate bias in decision-making processes. To mitigate biased decisions, we both need to understand the origin of the bias and define what it means for an algorithm to make fair decisions. Most group fairness notions assess a model's equality of outcome by computing statistical metrics on the outputs. We argue that these output metrics encounter intrinsic obstacles and present a complementary approach that aligns with the increasing focus on equality of treatment.

Algorithmic Profiling and Epistemic Injustice (with S. Milano)
Philosophical Studies, forthcoming
It is a well-established fact that algorithms can be instruments of injustice. It is less frequently discussed, however, how current modes of AI deployment often make the very discovery of injustice difficult, if not impossible. In this article, we focus on the effects of algorithmic profiling on epistemic agency. In particular, we show how algorithmic profiling can give rise to epistemic injustice through the depletion of epistemic resources that are needed to interpret and evaluate certain experiences.

Human autonomy in the age of artificial intelligence
Nature Machine Intelligence 4: 99–101, 2022
Current AI policy recommendations differ on what the risks to human autonomy are. To systematically address risks to autonomy, we need to confront the complexity of the concept itself and adapt governance solutions accordingly.

Is there a trade-off between human autonomy and the ‘autonomy’ of AI systems?
Philosophy and Theory of Artificial Intelligence 2021, ed. V. Müller, Springer Cham, 2022
Autonomy is often considered a core value of Western society that is deeply entrenched in moral, legal, and political practices. The development and deployment of artificial intelligence (AI) systems to perform a wide variety of tasks has raised new questions about how AI may affect human autonomy. Numerous guidelines on the responsible development of AI now emphasise the need for human autonomy to be protected. In some cases, this need is linked to the emergence of increasingly ‘autonomous’ AI systems that can perform tasks without human control or supervision. Do such ‘autonomous’ systems pose a risk to our own human autonomy? In this article, I address the question of a trade-off between human autonomy and system ‘autonomy’.

Simulation Intelligence: Towards a New Generation of Scientific Methods (with A. Lavin et al.)
arXiv preprint arXiv:2112.03235
We present the "Nine Motifs of Simulation Intelligence", a roadmap for the development and integration of the essential algorithms necessary for a merger of scientific computing, scientific simulation, and artificial intelligence. We call this merger simulation intelligence (SI), for short. We argue the motifs of simulation intelligence are interconnected and interdependent, much like the components within the layers of an operating system. Using this metaphor, we explore the nature of each layer of the simulation intelligence operating system stack (SI-stack) and the motifs therein: (1) Multi-physics and multi-scale modeling; (2) Surrogate modeling and emulation; (3) Simulation-based inference; (4) Causal modeling and inference; (5) Agent-based modeling; (6) Probabilistic programming; (7) Differentiable programming; (8) Open-ended optimization; (9) Machine programming.

Institutionalising Ethics in AI through Broader Impact Requirements (with C. Ashurst, M. Anderljung, H. Webb, J. Leike, A. Dafoe) Nature Machine Intelligence 3:104–110, 2021
In response to growing concerns about the ethical, social, and economic impacts of artificial intelligence, the organisers of one of the world’s leading conferences on machine learning and computational neuroscience, NeurIPS, have announced that researchers submitting articles for peer review must now include a statement on the broader impacts of their research. In this article, we consider the advantages and challenges such requirements offer for the responsible development and deployment of AI technologies. We identify a set of tentative recommendations to conference organisers and the wider ML community.

Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims, (with M. Brundage et al.)
Report, 2020
With the recent wave of progress in artificial intelligence (AI) has come a growing awareness of the large-scale impacts of AI systems, and recognition that existing regulations and norms in industry and academia are insufficient to ensure responsible AI development. In order for AI developers to earn trust from system users, customers, civil society, governments, and other stakeholders that they are building AI responsibly, they will need to make verifiable claims to which they can be held accountable. Those outside of a given organization also need effective means of scrutinizing such claims. This report suggests various steps that different stakeholders can take to improve the verifiability of claims made about AI systems and their associated development processes, with a focus on providing evidence about the safety, security, fairness, and privacy protection of AI systems. We analyze ten mechanisms for this purpose--spanning institutions, software, and hardware--and make recommendations aimed at implementing, exploring, or improving those mechanisms.

A Guide to Writing the NeurIPS Impact Statement (with C. Ashurst, M. Anderljung, J. Leike, Y. Gal, T. Shevlane, and A. Dafoe), Medium, 2020
From improved disease screening to authoritarian surveillance, ML advances will have positive and negative social impacts. Policymakers are struggling to understand these advances, to build policies that amplify the benefits and mitigate the risks. ML researchers need to be part of this conversation: to help anticipate novel ML applications, assess the social implications, and promote initiatives to steer research and society in beneficial directions. Innovating in this respect, NeurIPS has introduced a requirement that all paper submissions include a statement of the “potential broader impact of their work, including its ethical aspects and future societal consequences.” This is an opportunity for authors to think about and better explain the motivation and context for their research to other scientists. To help ML researchers think through the impacts of their work, we — a team of AI governance, AI ethics, and machine learning (ML) researchers — have compiled some suggestions and a guide for how to write such a broader impact statement.

Beyond near and far: A new taxonomy for the AI Policy space (with J. Whittlestone)
Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society 138-143, 2020
One way of carving up the broad "AI ethics and society" research space that has emerged in recent years is to distinguish between "near-term" and "long-term" research. While such ways of breaking down the research space can be useful, we put forward several concerns about the near/long-term distinction gaining too much prominence in how research questions and priorities are framed. We highlight some ambiguities and inconsistencies in how the distinction is used, and argue that while there are differing priorities within this broad research community, these differences are not well-captured by the near/long-term distinction. We unpack the near/long-term distinction into four different dimensions, and propose some ways that researchers can communicate more clearly about their work and priorities using these dimensions. We suggest that moving towards a more nuanced conversation about research priorities can help establish new opportunities for collaboration, aid the development of more consistent and coherent research agendas, and enable identification of previously neglected research areas.

On the Equivalence of von Neumann and Thermodynamic Entropy
Philosophy of Science 87:2, 262-280, 2020
In 1932, John von Neumann argued for the equivalence of the thermodynamic entropy and −Trρlnρ, since known as the von Neumann entropy. Meir Hemmo and Orly R. Shenker recently challenged this argument by pointing out an alleged discrepancy between the two entropies in the single-particle case, concluding that they must be distinct. In this article, their argument is shown to be problematic as it (a) allows for a violation of the second law of thermodynamics and (b) is based on an incorrect calculation of the von Neumann entropy.

On the thermodynamical cost of some interpretations of quantum theory (with C. Timpson)
Studies in History and Philosophy of Science Part B: Studies in History and Philosophy of Modern Physics 63: 114-122, 2018.
Recently, Cabello et al. (2016) claim to have proven the existence of an empirically verifiable difference between two broad classes of quantum interpretations. On the basis of three seemingly uncontentious assumptions, (i) the possibility of randomly selected measurements, (ii) the finiteness of a quantum system's memory, and (iii) the validity of Landauer's principle, and further, by applying computational mechanics to quantum processes, the authors arrive at the conclusion that some quantum interpretations are associated with an excess heat cost and are thereby untenable—or at least—that they can be distinguished empirically from their competitors by measuring the heat produced. Here, we provide an explicit counterexample to this claim and demonstrate that their surprising result can be traced back to a lack of distinction between system and external agent. By drawing the distinction carefully, we show that the resulting heat cost is fully accounted for in the external agent, thereby restoring the tenability of the quantum interpretations in question.

The Road to Quantum Thermodynamics. C. Prunkl, forthcoming in Quantum Foundations of Statistical Mechanics, eds. C. Timpson, D. Bedingham, OUP 2019

Impulsivity,self-control and hypnotic suggestibility. (with V.U. Ludwig, C. Stelzel, H. Krutiak, R. Steimke, L.M. Paschke, N. Kathmann and H. Walter)
Consciousness and Cognition, 22(2):647-653, 2013

Working Papers

AI, Control and Human Autonomy, C. Prunkl, manuscript upon request, 2020
Autonomy is often considered one of the core values of Western society and as such it is deeply entrenched in moral, legal, and political practices. The development and deployment of artificial intelligence (AI) systems to perform a wide variety of tasks, including driving cars, operating trains, assessing risk, or making recommendations, has raised new questions about how AI may affect human autonomy. Yet, current discussions on the topic are both rare and often held on a case-by-case basis, preventing a systematic assessment of the impacts of AI on human autonomy. In this article, I provide a framework for human autonomy in the context of AI that allows us to distinguish between different types of impacts on human autonomy, each of which require distinct policy and design responses.

Black Hole Entropy is Thermodynamic Entropy (with C. Timpson), revise and resubmit, BJPS
The comparison of geometrical properties of black holes with classical thermodynamic variables reveals surprising parallels between the laws of black hole mechanics and the laws of thermodynamics. Since Hawking's discovery that black holes when coupled to quantum matter fields emit radiation at a temperature proportional to their surface gravity, the idea that black holes are genuine thermodynamic objects with a well-defined thermodynamic entropy has become more and more popular. Surprisingly, arguments that justify this assumption are both sparse and rarely convincing. Most of them rely on an information-theoretic interpretation of entropy, which in itself is a highly debated topic in the philosophy of physics. We discuss some of the pertinent arguments that aim at establishing the identity of black hole surface area (times a constant) and thermodynamic entropy and show why these arguments are not satisfactory. We then present a simple model of a Black Hole Carnot cycle to establish that black hole entropy is genuine thermodynamic entropy which does not require an information-theoretic interpretation.

Responsible Research Practice in Machine Learning, C. Prunkl, 2020

Popular Articles

Endlich Unendlich - auf der Suche nach dem ewigen Leben. C. Prunkl, SHIFT, 4:14-19, 2016

Das Schummeln der Lämmer - Von kleinen Lügen und großen Konsequenzen, C. Prunkl, SHIFT, 1:42-46, 2013

Policy engagement

United Nations Development Programme China

Centre for Data Ethics and Innovation, Dep. for Digital, Culture, Media & Sport, UK

Defence Science and Technology Laboratory, Ministry of Defence, UK

EU Delegation in Russia

Senate and Ministry of Economics, Mexico

Other Engagements

Board Member
Oxford AI Society, 2019

Advisor on future governance of AI, UK 2070 Commission, Expert consultation, 2019

Expert Panelist, MentorA.I. Impact Weekend, Oxford Foundry, 8.-10.2.2019