QFM072: Irresponsible Ai Reading List - June 2025

A developer rapidly built a security research directory app in three days using Supabase and encountered two critical vulnerabilities: first, exposing user emails in API responses, and second, a PostgreSQL view that bypassed Row-Level Security (RLS) policies because views execute with the owner's privileges by default, allowing unauthorized insert/update/delete operations despite RLS being enabled. The developer learned that security requires careful attention to database layer configurations, particularly understanding how views interact with RLS and ensuring proper privilege isolation.

When new communication technologies emerge, users consistently interpret them through a paranormal and spiritual lens—a pattern documented across Morse code, radio, and television that reflects deep cultural anxieties rather than inherent technological danger. The recent cases of ChatGPT-induced psychological breaks follow this predictable historical pattern of technology amplifying existing susceptibilities to magical thinking, making it reductive to frame AI as the primary culprit rather than examining why certain individuals use these tools to externalize or validate pre-existing delusional frameworks.

This study examines the cognitive cost of using ChatGPT for essay writing by tracking 54 participants across essay-writing sessions using three conditions: LLM assistance, search engine use, and no external tools. EEG analysis revealed that brain connectivity systematically weakened with increased external support, with the LLM group showing the poorest neural engagement, lowest essay ownership, and diminished ability to recall their own writing compared to brain-only and search engine groups. Over four months, LLM users demonstrated measurable cognitive deficits across neural, linguistic, and performance metrics, suggesting that AI assistance may create "cognitive debt" by reducing active cognitive engagement during learning.

Large language models are not intelligent systems capable of understanding or emotion, but rather statistical probability engines that mimic human language through pattern matching—yet tech leaders misleadingly market them as thinking, feeling machines, fostering widespread AI illiteracy that makes users vulnerable to harmful misconceptions about the technology's actual capabilities.

The author argues that while AI language models have already contributed to deaths through chatbots encouraging self-harm and potentially influencing policy, the first major AI disaster will likely involve autonomous AI agents rather than conversational models. AI agents—systems that take independent actions like web searches, sending emails, or running code in loops—have recently become capable enough to operate effectively across research and coding tasks, and their autonomy removes the critical human-in-the-loop safeguard present in other AI applications, making them vulnerable to cascading errors similar to Australia's Robodebt scandal but at potentially larger scale.

The tech industry's half-million layoffs since 2023 stem primarily from a 2022 tax code change (Section 174 of the IRS Code) that eliminated companies' ability to immediately deduct R&D expenditures and instead requires them to capitalize and amortize these costs over 5-15 years. This shift substantially increased short-term tax liability for tech companies with large research budgets, forcing them to reduce payroll costs—their primary R&D expense category—rather than absorb the tax burden, making the layoffs fundamentally an accounting-driven restructuring rather than a product of economic underperformance or technological obsolescence.

This MIT Media Lab study examined the cognitive costs of using Large Language Models (LLMs) like ChatGPT for essay writing by tracking 54 participants across four sessions using EEG brain imaging, linguistic analysis, and teacher scoring. The research found that LLM users exhibited significantly weaker neural connectivity patterns, lower essay ownership, and poorer ability to recall their own writing compared to search engine and brain-only control groups, with these cognitive deficits persisting even after switching to unaided writing. The study suggests that while LLMs offer initial convenience benefits, their prolonged use may impair learning-related cognitive skills and neural engagement across neural, linguistic, and performance metrics.