QFM017: Machine Intelligence Reading List - May 2024

The article demonstrates how to achieve state-of-the-art exact binary vector search for Retrieval Augmented Generation (RAG) using only 100 lines of Julia code. It highlights the benefits of binary vector spaces, such as reduced server costs and improved in-memory retrieval feasibility, and provides benchmarking results showing the efficiency and speed of this approach compared to traditional methods.

The article discusses the long-standing debate over the potential and implications of Artificial General Intelligence (AGI), referencing historical predictions, current developments in AI, and differing expert opinions on its feasibility and risks. It highlights how AGI, if achieved, could fundamentally change automation and poses significant ethical and existential considerations.

The article discusses a study finding that refusal behavior in large language models (LLMs) is controlled by a single direction in the model's activation space. By modifying this direction, researchers can either bypass or induce the refusal of harmful or harmless instructions, demonstrating the fragility of safety fine-tuning in open-source chat models. This method of controlling refusal behaviors validates interpretability results and highlights potential vulnerabilities in LLMs' safety mechanisms.

Artificial intelligence (AI) and machine learning (ML) are often misunderstood, leading to misconceptions about their capabilities and limitations. This article debunks five common myths, clarifying that AI and ML are not the same, are not magic solutions, do not require advanced degrees to understand, will not replace all jobs, and do not need perfect data to be effective.

The video provides a detailed explanation of diffusion models, covering their fundamental concepts, mathematical derivations, and comparisons with other methods, aiming to make the topic accessible and understandable.

Apple plans to launch a new AI-powered version of Safari with iOS 18 and macOS 15, featuring advanced content blocking, an "Intelligent Search" tool for summarising web pages, and a "Web Eraser" for removing unwanted page elements, all accessible through an updated user interface.

The article explores the potential impact of large language models (LLMs) like GPT-4 on software development, suggesting that soon all computer users may be able to create and modify small software tools. This shift could lead to significant changes in software production and distribution, enabling users to perform tasks such as creating one-off scripts and GUIs, developing custom in-house software, and extending existing applications without traditional programming skills.

Google DeepMind and Isomorphic Labs have introduced AlphaFold 3, an advanced AI model that predicts the structure and interactions of a wide range of biomolecules with unprecedented accuracy, aiming to transform biological research and drug discovery. The model is available for free to scientists via the AlphaFold Server, enhancing the accessibility and speed of molecular predictions critical for understanding diseases and developing new treatments.

The Anthropic Prompt Engineering Interactive Tutorial provides a step-by-step guide for individuals with a basic understanding of AI to master the art of crafting effective prompts for Claude language models. The course covers essential techniques from basic prompt structuring to advanced methods like avoiding hallucinations and using role prompting, ensuring participants can build and troubleshoot prompts for various applications.

John Carmack, a renowned tech innovator, is independently pursuing artificial general intelligence (AGI) through his startup Keen Technologies, aiming to achieve human-level AI by 2030. Carmack believes his unique approach and extensive background in gaming, aerospace, and virtual reality position him well to contribute significantly to this challenging goal.

This article provides an in-depth explanation of diffusion models, covering their mathematical foundations and operational principles in generating data. It details how these models function by iteratively refining noise into coherent data through a reverse diffusion process.

Daniel Dennett's essay "Where Am I?" explores the philosophical and practical implications of a brain transplant, where his brain is removed and placed in a vat while his body operates remotely via radio links. This story delves into questions of identity, consciousness, and the nature of the self, illustrating the complexities and paradoxes that arise when considering where "self" truly resides.

Dr. Rosemary Lall, a family physician in Ontario, credits new AI note-taking software with saving her job by significantly reducing the time spent on administrative tasks, thus restoring her work-life balance and joy in her profession. The AI Scribe technology, which transcribes and organizes patient notes in real time, is being piloted with promising early results but raises concerns about data privacy and patient confidentiality.

OpenAI has released its first AI-made music video, "The Hardest Part," directed by Paul Trillo and produced using their new text-to-video generator, Sora, for the artist Washed Out. This launch aims to attract collaborations with major studios and media executives while sparking debate over the impact of AI on the creative industry.

The article discusses how generative AI enables the automation of tasks at a massive scale, fundamentally changing their nature and raising new ethical and practical concerns. It illustrates this through examples like surveillance, face recognition, and deepfakes, emphasizing that these issues are often rooted in societal and cultural perceptions rather than the technology itself.

John Mount argues that while retrieval augmented generation (RAG) attempts to address the limitations of large language models (LLMs) by supplementing them with external references, the true value of LLMs lies in their flexible natural language query interface rather than their generative capabilities, which often produce unreliable results. He illustrates this with a personal anecdote where a traditional search engine, despite its flaws, provided a more satisfactory outcome than a purely generative AI.

The article discusses how the rapid adoption of generative AI technologies in 2023, driven by tools like ChatGPT, has accelerated advancements typically seen over a decade into a single year. It outlines three major trends for 2024: the evolution of model orchestration, the transition from prototyping to production with a focus on data utilization, and the emphasis on return on investment (ROI) for AI investments, predicting continued innovation and practical applications across various industries.

The "Deep Reinforcement Learning: Zero to Hero" course provides a practical introduction to foundational deep reinforcement learning algorithms, guiding participants to write and understand key algorithms such as DQN, SAC, and PPO, while training AI to play Atari games and perform tasks like landing on the Moon.

The article explains how Large Language Models (LLMs) like GPT work without using advanced mathematics, describing their core functionality of predicting the next token in a text sequence based on the input provided. It covers the concepts of tokenization, probability prediction, and the use of neural networks with a focus on the Transformer architecture and its attention mechanism.

OpenAI is reportedly planning to launch a search engine in May, potentially to coincide with Google's annual developer conference, as suggested by a subdomain and SSL certificate for search.chatgpt.com. Speculations are based on information shared by Jimmy Apples and comments by OpenAI's CEO Sam Altman, who hinted at a new approach to web search rather than merely replicating Google.

Ilya Sutskever provided John Carmack with a curated list of around 30 research papers, suggesting that mastering this selection would cover 90% of the essential knowledge in the field of artificial intelligence today.

The author argues that OpenAI is unlikely to surpass Apple, Google, and Meta in software or hardware and must be the first to achieve Artificial General Intelligence (AGI) to remain competitive. However, due to significant brain drain and a focus on consumer products that waste valuable computing resources, the author doubts OpenAI's ability to reach AGI soon.

GPT-4o integrates text, audio, and visual inputs into a single model, enhancing its ability to process and generate multimodal outputs more cohesively. Currently, the API supports text and image inputs, with audio capabilities expected soon.

The article argues that machine learning in Elixir is not only ready for production but is highly efficient due to its integration with the BEAM and OTP, providing robust, fault-tolerant systems. The Nx library's architecture and capabilities, such as multi-stage compilation and hardware agnosticism, further enhance Elixir's suitability for machine learning tasks.