Quantifying AI-infused Science

Introduction

The recent public release of Artificial Intelligence (AI) that can master natural language has sparked a debate about the capabilities of AI and whether it can produce scientific content. Despite the ongoing debate, AI-generated written content has already entered the world of science:

• Papers co-written by AI are published.
• Companies release models to assist scientists with producing scientific content.
• A large fraction of scientists use AI-based tools to augment their writing.

We will thus inevitably have AI-infused science in our future.

Project Goal

The goal of scAIence is to quantify whether, how, and with which implications generative AI is changing how scientists write, communicate, perceive, and diffuse science. The project aims to rigorously explore the opportunities, dangers, and implications of scientists augmenting their science with AI.

Key Hypothesis

The key hypothesis of scAIence is that current AI lacks the ability to combine knowledge entities (e.g. references) in the same manner as humans and is unable to replicate the social information (e.g. homophily or recency) present in human scientific output.

Importance of Testing

Testing the implications of this hypothesis is crucial since the scientific enterprise relies heavily on social information present in scientific output data for a wide range of purposes, including:

1. Metrics definition
2. Information retrieval
3. Predictions

Methodology

The scAIence project will deploy a novel computational social science approach, based on a wide array of quantitative disciplines, leveraging large-scale databases of human-generated information and controlled experiments.

Innovations

scAIence will break new ground by:

1. Introducing the quantitative methods required to understand AI-infused science.
2. Redefining metrics and models to account for AI-generated content in science.
3. Delivering quantitative scientific insights into how AI is changing the diffusion of science.

Conclusion

Taken together, scAIence will lay the scientific foundation for the quantitative study of AI-infused science.