QFM032: Irresponsible Ai Reading List - August 2024

The advent of photography in the 1880s democratized the art, but it also spurred a movement concerned with the right to privacy. The Kodak camera, introduced in 1888, made photography widespread and accessible, leading to issues like image rights and unauthorized use. The article explores historical cases, including one where a woman's portrait was used without consent in a medicine ad, and the societal response to these privacy invasions. Legislation slowly developed to address these issues, highlighting the ongoing struggle between technology and privacy.

The article examines the existence of gender biases in GPT language models, revealing that these models often reinforce traditional gender stereotypes and exhibit unequal treatment of different genders. The findings underscore the need for improved measures to mitigate bias in AI systems to ensure fair and equitable outcomes.

The article explores the limitations of ChatGPT in summarising texts. Through various examples, the author explains how ChatGPT often fails to accurately capture the core ideas of long documents, instead producing shortened versions that miss key points and sometimes fabricate information. The article concludes that ChatGPT doesn't truly understand the content it is summarising, leading to unreliable summaries. The author suggests that while ChatGPT can shorten texts, it lacks the deeper understanding needed for genuine summarisation.

Experts argue that the benchmark tests widely used to evaluate AI performance are outdated and lack validity. These tests, often sourced from amateur websites and designed for simpler models, fail to measure the nuances of newer AI systems' capabilities. Additionally, marketing of AI tools using these benchmarks can mislead users about their true functionality, raising concerns especially in high-stakes domains like healthcare and law.

AI models are crucial in analyzing medical images like X-rays for diagnostics. However, these models often show biases, particularly towards certain demographic groups, such as women and people of color. MIT researchers have demonstrated that the most accurate AI models for demographic predictions also show the largest fairness gaps, highlighting how these models use demographic shortcuts, leading to inaccuracies. While debiasing techniques can improve fairness on the same data sets they were trained on, these improvements don't always transfer when applied to new data sets from different hospitals.

This paper investigates the potential impacts of large language models (LLMs) like GPTs on the U.S. labor market. The authors estimate that around 80% of the U.S. workforce could have at least 10% of their work tasks affected by LLMs, while around 19% may see at least 50% of their tasks impacted. The study also suggests that higher-income jobs may face greater exposure to LLM capabilities, with significant implications for economic, social, and policy aspects.

Developers' claims that GitHub Copilot was unlawfully copying their code have largely been dismissed, with only two allegations remaining in their lawsuit against GitHub, Microsoft, and OpenAI. The lawsuit, filed in November 2022, contended that Copilot was trained on open source software and violated the original creators' intellectual property rights. Most of the claims have been thrown out, including an important one under the DMCA, leaving just a couple of allegations still standing, both related to license violations and breach of contract.

In his article 'Put Up Or Shut Up', Edward Zitron criticizes the tech industry's current trend of hyping up AI with empty promises and meaningless marketing. He particularly focuses on Lattice's recent announcement about employing AI 'digital workers', labeling the idea as nonsensical and disconnected from practical use, and criticizes OpenAI's questionable claims about progressing towards Artificial General Intelligence (AGI). Zitron calls for more scrutiny and skepticism when dealing with flashy tech advancements that lack substantial evidence or practical application.

Researchers from the University of Washington and the Allen Institute for AI have discovered a vulnerability in the safety alignment of large language models (LLMs) like GPT, Llama, and Claude. Known as 'ChatBug,' this vulnerability exploits the chat templates used for instruction tuning. Attacks such as format mismatch and message overflow can trick LLMs into producing harmful responses. The research highlights the difficulty in balancing safety and performance in AI systems and calls for improved safety measures in instruction tuning.