I was invited to contribute to a seminar/discussion on AI, and using language models in research. These are the notes I prepared my short presentation. I am not presenting anything new, or original - I will merely be sharing what I consider being the main take home messages from information I have been collecting since April 2023. I also realised that these don’t seem to be widely known among my immediate peers.

Who am I

I feel it is important to say a few words to put my talk into perspective:

• I am a Computational Biologist, heading the CBIO lab at the de Duve Institute, UCLouvain. We occasionally use/develop of DL in the CBIO lab as part of our research. I am not an expert in AI.
• I have never used ChatGPT or any similar tools, and I’ll tell you why! I however actively follow discussions on AI and its impact on society.
• I will focus on ChatGTP and similar LMs that are released by large and very powerful commercial entities for wide public consumption. I am not focusing on application of DL, LM, or AI in general in research.

Reference

If there’s one reference to remember, it is this seminal paper from March 2021, published in the Proceedings 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT ‘21)

On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? by Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, Shmargaret Shmitchell.

• Emily Bender is a professor of computational linguistics at Washington University. Angelina McMillan-Major is/was a PhD student in her lab.
• Timnit Gebru is one of the most well-known and respected Black female scientists working in AI. She was a co-lead of Google’s ethical AI research team. In December 2020, her employment with Google ended after Google management asked her to either withdraw the paper before publication, or remove the names of all the Google employees from the paper.
• Margaret Mitchell was later also fired from Google.

More about the Stochastic parrot paper here.

When can I use ChatGPT?

Following Yves Deville and Christine Jacqmot’s recommendations (ChatGPT : Menace ou opportunité pour l’enseignement supérieur, March 2023):

Given that

LLMs have absolutely no notion of “true” or “false”, nor any understanding of what it is asked.

Use it if

1. You don’t can about the validity of the results.
2. You are an expert in the field.

Note: lots has been said about ChatGPT’s “occasional” hallucinations (beware of the anthropomorphising word here). They always hallucinate. It just happens so that sometimes, what is made up, is not wrong. I will come back to some of these points in the later stochastic parrot section.

At what cost?

Many have tested ChatGPT. Some (New AI tools much hyped but not much used, study says) possibly make regular use of the free and/or the paid version. It might be used for important or minor/mundane tasks. But at what costs?

Human cost

Human cost is real and current. It is not a potential science-fiction picture of AI vs humanity. Such a picture diminishes the current human cost of AI, as is force-fed by big tech.

Here are some investigations about worker exploitation at OpenAI and Google with respect to the curation of AI-generated content:

• TIME: OpenAI Used Kenyan Workers on Less Than $2 Per Hour.
• The Guardian: ‘It’s destroyed me completely’: Kenyan moderators decry toll of training of AI models.
• Business Insider: Kenyan Workers Paid $2/hr Labeled Horrific Content for OpenAI.

and

• Fortune: I’m paid $14 an hour to rate AI-generated Google search results. Subcontractors like me do key work but don’t get fair wages or benefits.

Note that this isn’t specific to ChatGPT. Similar workers exploitation has been documented for Meta/Facebook reviewers from the Global South.

Already marginalised communities suffer the highest human cost.

Environmental cost

Here are some relevant articles and illustrative quotes;

• Nature Machine Intelligence: The carbon impact of artificial intelligence.
• technologyreview.com: We’re getting a better idea of AI’s true carbon footprint.
• Nature Climate Change: Aligning artificial intelligence with climate change mitigation.
• nature.com: Generative AI’s environmental costs are soaring - and mostly secret.
• nature.com: How to shrink AI’s ballooning carbon footprint.

• The GuardianThe ugly truth behind ChatGPT: AI is guzzling resources at planet-eating rates.

  Despite its name, the infrastructure used by the “cloud” accounts for more global greenhouse emissions than commercial flights. In 2018, for instance, the 5bn YouTube hits for the viral song Despacito used the same amount of energy it would take to heat 40,000 US homes annually.

  Furthermore, while minerals such as lithium and cobalt are most commonly associated with batteries in the motor sector, they are also crucial for the batteries used in datacentres. The extraction process often involves significant water usage and can lead to pollution, undermining water security. The extraction of these minerals are also often linked to human rights violations and poor labour standards. Trying to achieve one climate goal of limiting our dependence on fossil fuels can compromise another goal, of ensuring everyone has a safe and accessible water supply.

• The Guardian: As the AI industry booms, what toll will it take on the environment? (citing - Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Model)

  [Luccioni et al.] tallied the amount of energy used to train […] Bloom, on a supercomputer; the energy used to manufacture the supercomputer’s hardware and maintain its infrastructure; and the electricity used to run the program once it launched. They found that it generated about 50 metric tons of carbon dioxide emissions, the equivalent of an individual taking about 60 flights between London and New York.

  By contrast, limited publicly available data suggests about 500 metric tonnes of CO2 were produced just in the training of ChatGPT’s GPT3 model 3 – the equivalent of over a million miles driven by average gasoline-powered cars, the researchers noted.

  Even more unclear is the amount of water consumed in the creation and use of various AI models. Data centers use water in evaporative cooling systems to keep equipment from overheating. One non-peer-reviewed study, led by researchers at UC Riverside, estimates that training GPT3 in Microsoft’s state-of-the-art US data centers could potentially have consumed 700,000 liters of freshwater.

• theconversation.com: AI has a large and growing carbon footprint, but there are potential solution on the horizon.

  Since the AI boom started in the early 2010s, the energy requirements of AI systems known as large language models (LLMs) – the type of technology that’s behind ChatGPT – have gone up by a factor of 300,000. With the increasing ubiquity and complexity of AI models, this trend is going to continue, potentially making AI a significant contributor of CO₂ emissions. In fact, our current estimates could be lower than AI’s actual carbon footprint due to a lack of standard and accurate techniques for measuring AI-related emissions.

• tomshardware.com: AI may eventually consume a quarter of America’s power by 2030, warns Arm CEO.

• bloomberg.com: Microsoft’s AI Investment Imperils Climate Goal As Emissions Jump 30%.

  How ironic!!

  “The company’s goal to be carbon negative by 2030 is harder to reach, but President Brad Smith says the good AI can do for the world will outweigh its environmental impact.”

Note that this is also relevant for other cloud services, such as video on demande (detail for Netflix here).

Already marginalised communities (will) suffer the highest environmental cost.

Intellectual property

Where does all that training data come from?

• What about the credit and licensing of text, voice and images of those that produced that data used for training.

Stochastic parrot

I’ll borrow here directly from the paper, to highlight specific issues with the vast amounts of data needed to train these large models, and the (absence of) meaning output by the models.

Unfathomable training data

• Size doesn’t guarantee diversity: from initial participation, to data filtering, the data reflect the hegemonic viewpoint.
• Data is static data, but social views change.
• Biais is encoding and amplified in the training data, in particular stereotypical associations and negative sentiment towards specific groups.
• Large data and the lack of curation, documentation and accountability lead to a major documentation debt, that can’t be addressed after the fact.

Systematic biais against already marginalised communities.

Stochastic parrot

Coherence is in the eye of the beholder

• There is no meaning, no model of the world, no intend to communicate in ChatGPT’s output.
• Perceived “fluency” and “confidence” give the illusion of (implicit) meaning and expertise.
• We tend to mistake the coherence of LLM outputs for meaningful text or expertise.

Contrary to how it may seem when we observe its output, an LM is a system for haphazardly stitching together sequences of linguistic forms it has observed in its vast training data, according to probabilistic information about how they combine, but without any reference to meaning: a stochastic parrot.

It is important to note that, in addition to highlight the risks, the authors do propose paths forward for LM research and development.

AI contamination

AI-generated text is already ubiquitous on-line, and it becomes more and more difficult to identify AI-generated text. How long until AI-generated (meaningless) text (including as answers in Q&A sites), will be (or are) re-used for training.

Outlets are terminating journalist contract to replace them by AI, and independent writers are ‘competing’ against AI.

We have all faced AI chat-bots in so-called help-desks. But AI chatbots are intruding into online communities where people are trying to connect with other humans.

Both of these responses were lies. That child does not exist and neither do the camera or air conditioner. The answers came from an artificial intelligence chatbot.

According to a Meta help page, Meta AI will respond to a post in a group if someone explicitly tags it or if someone “asks a question in a post and no one responds within an hour.”

There are prime examples of enshittification (from Wikipedia):

Enshittification is the pattern of decreasing quality observed in online services and products such as Amazon, Facebook, Google Search, Twitter, Bandcamp, Reddit, Uber, and Unity. The term was used by writer Cory Doctorow in November 2022, and the American Dialect Society selected it as its 2023 Word of the Year. Doctorow has also used the term platform decay to describe the same concept.

ChatGPT in research

• Reproducibility? AlphaFold3 — why did Nature publish it without its code?

  When AlphaFold2 was published, the full underlying code was made accessible to all researchers. But AlphaFold3 comes with ‘pseudocode’ — a detailed description of what the code can do and how it works.

  […] for AlphaFold2, the DeepMind team worked with the European Molecular Biology Laboratory’s European Bioinformatics Institute […] Now, DeepMind has partnered with Isomorphic Labs, a London-based drug-development company owned by Google’s parent, Alphabet. In addition to the non-availability of the full code, there are other restrictions on the use of the tool — for example, in drug development. There are also daily limits on the numbers of predictions that individual researchers can perform.

• Science journals ban listing of ChatGPT as co-author on papers

• Paper writing (paper mills) and reviews (ChatGPT is polluting/influencing peer review).

Who benefits from ChatGTP/AI?

AI, as a hyped-up surveillance business model, force-fed by big tech:

• In search engines (Google’s “AI Overviews”). Not the users.
• Use your social media photos, posts, info, … to train AI. Not the users.
• Facial recognition. Not the citizens.
• Microsoft Windows Recall. Not the employees.

Already marginalised communities likely to benefit the least. Privileged communities to benefit the most.

What about regulations?

In the light of what has been said so far, I think it is reasonable to wonder whether regulations shouldn’t be put in place, to address current and future impact and scope of the technologies put in place, their concrete risks and harms, and their implications in terms of systematic (private) data collection and use. Every major big tech company is investing vast amounts of money in AI technologies, data centres, and data collection. And they are demanding returns on these investments.

These same companies are actively lobbying to assure support in their vested interested. This becomes clear when reviewing their implications in various working groups and how AI is framed and communicated to the public and various stakeholders.

Here are two examples, one very recent from the Guardian, and one that directly relates to the influence of Silicon Valley on academia:

• OpenAI forms safety council as it trains latest artificial intelligence model: The safety committee is filled with company insiders, including Sam Altman, the OpenAI CEO, and its chairman, Bret Taylor, and four OpenAI technical and policy experts. It also includes the board members Adam D’Angelo, who is the CEO of Quora, and Nicole Seligman, a former Sony general counsel.

• How Big Tech Manipulates Academia to Avoid Regulation: The discourse of “ethical AI” was aligned strategically with a Silicon Valley effort seeking to avoid legally enforceable restrictions of controversial technologies.

Conclusions

Despite some notable failures with ‘AI for public consumption’ , one can’t ignore that there there are also success stories, and possibly still untapped opportunities. But …

AI can be kind of useful, but I’m not sure that a “kind of useful” tool justifies the harm.

AI isn’t useless. But is it worth it?, Molly White

Update (2024-10-10)

There are many more relevant articles that could be added and referenced here, too many for me to keep up with. But the following Mastodon post by sneedy maccreedy and article it links to seem particularly relevant:

hinton getting the nobel is a good time to re-read @emilymbender ‘s excellent piece on so-called ‘AI safety’ and different take on hinton than you’re likely to see in the next few days

The article is Talking about a ‘schism’ is ahistorical by Emily M. Bender (also cited above), documenting the phony shism rhetoric of AI safety fantasy on one hand and very real AI ethics on the other.