PIBBSS: Notes on Motivations and Scope

Note: The following is organised as a FAQ. It assumes familiarity with our project plan; without having taken a look at concrete examples, most of the discussion here could sound like premature abstraction and ungrounded theorization.


1. What is this ‘AI’ thing we care about?

As our civilization makes progress in our abilities to engineer and instantiate sophisticated forms of complex and intelligent behaviours in artificial systems, such systems will have an increasingly dominating effect over the future course of our civilization and, potentially, the universe. Alongside its positive potential, the possibility that such progress could lead to irreversible harmful effects and ruin, like extinction or large amounts of suffering, demands further care in how we approach such artificial systems. This motivates us to prioritise careful and value-sensitive progress in AI research.

2. What kind of research progress does PIBBSS aim to foster?

Insofar as AI research deals with mechanisms that can search and instantiate complex behaviours, the research is shaped by the concepts, metaphors and frameworks that can be used for reasoning about them, which are accessible to these researchers. For example, one can observe contemporary AI research to be rich with concepts inherited from mid-20th century behaviourism (Hebb, Rosenblatt, etc.) and operations research (eg. optimal control theory). 

Equivalently, research can be constrained and bottlenecked by the absence of relevant concepts and frameworks. This is particularly true when dealing with usually neglected problems and research directions. This raises the following question: from where can we supply new useful concepts from?

One way could be to employ mathematical and philosophical first principles thinking to clarify and concretize new concepts. Another approach could be to iteratively improve our existing concepts by iterative experimentation with existing AI systems, and using empirical knowledge to clarify new distinctions and phenomena.

Yet another approach is to look at other disciplines with a rich history of studying complex behaviours in natural and social systems, such as evolutionary biology or social anthropology. We argue that these disciplines can act as rich reservoirs of conceptual tools and insights that provide valuable support in several important research directions. And therefore attempt in this fellowship plan, aim at enabling translation and transfer between these source domains and AI research by identifying valuable, otherwise neglected, and reasonably well-scoped frontiers.

3. Which source [1] domains are we excited about?

We are interested in facilitating transfer of insights from disciplines which have rich literature dealing with complex behaviours and phenomena in some relevant natural or social system. We are sensitive to neglectedness considerations and thus less interested on source domains that are already well-represented or easily accessible within AI, e.g. microeconomics.

This leads us to a tentative list that currently includes: evolutionary biology, ecology, systems biology; neuroscience, cognitive psychology, philosophy of mind; thermodynamics, statistical mechanics, complex systems studies; economic, legal, and political theory; sociology, social anthropology, linguistics; computational social science; and media studies. We are, however, open to other similar topics and disciplines that we may have overlooked at the moment, but which share the properties that we care about (ie. insightfulness in existing literature and associated community, and relevance to AI alignment).

We are also cognizant of, and wish to learn from successes and failures of, past efforts in cross-domain unification of frameworks from similar ranges of disciplines such as mid-20th century Cybernetics, ‘its slightly more general cousin’ Systems Sciences, and their more recent avatar Complex Systems Studies. In particular, these historical and contemporary endeavours have provided us with valuable transdisciplinary vocabulary that plays a critical bootstrapping role in the kind of translation work we are interested in (for example, by people having relatively similar intuitions around ‘robustness’ across disciplines). 

However, unlike these past endeavours, the kind of research we are interested in has significantly more directedness towards progress in a specific target domain (ie. AI alignment), as opposed to being about general-purpose abstraction. Therefore, we see the eventual yardstick for the concepts and frameworks that come out of the type of research not in their cross-domain generality, but whether they helped answer important questions in AI alignment.

4. Which target problems and research directions are we excited about?

Within the general space of AI research, we are mainly interested in a wide range of technical and semi-technical problems that have coalesced around the label of ‘AI alignment’. We interpret AI alignment broadly. We take it to be inclusive of, but not limited to, (semi-/)technical problems in: safety, goal/intent alignment, corrigibility, cooperation and coordination, inner alignment, etc.

In our broad interpretation, we also include directions that are sometimes not included within technical AI alignment research, particularly: a) general investigation of properties of advanced and powerful AI systems, and b) subareas of AI Governance which reason about governability, cooperativeness, or other functional aspects of AI systems and their interaction with other actors.

Beyond our central focus on (this broad notion of) alignment, we also aim to be sensitive towards important adjacent topics which, however, may not be considered part of AI research, such as relevant topics on: a) digital and emulated minds, b) advanced institutional design and collective intelligence, c) industrial and scientific automation and progress (PASTA). We consider these topics important in that they concern civilisational risks and flourishing, and intend to remain open to these topics if presented with any relevant project proposal and opportunities.

We also intend to remain open-minded towards important directions within contemporarist ML ethics (e.g. fairness, misinformation, and privacy), insofar as the project passes an adequate bar of impact potential and neglectedness.

On a case-to-case basis, we may consider applied AI research directions, conditioned on the particular project passing a high bar of potential social impact, the interdisciplinary interaction being judged crucial to progress on the project, the project promising to generate some generally valuable epistêmê or technê relevant to AI alignment, and any dual-use (and/or capabilities-enhancing) concerns being addressed [2].

Finally, we would also like to be welcoming of any underexplored research directions from control theory, information theory, or theory of agency which have not already been explored within work on AI alignment, insofar as we can establish a genuine case in favour of its exploration and the interdisciplinary interaction is judged to be crucial to its maturation within AI alignment.


[1] We use the terms ‘source’ and ‘target’ domain to capture the directedness of knowledge transfer that PIBBSS aims to facilitate (namely transfer towards/targeted at AI alignment).

[2] Last factor added retrospectively to this document in December 2022.