Artificial Intelligence Research and Industry Intelligence

arXiv cs.AI covers all areas of AI except Vision, Robotics, Machine Learning, Multiagent Systems, and Computation and Language (Natural Language Processing), which have separate subject areas. In particular, includes Expert Systems, Theorem Proving (although this may overlap with Logic in Computer Science), Knowledge Representation, Planning, and Uncertainty in AI. Roughly includes material in ACM Subject Classes I.2.0, I.2.1, I.2.3, I.2.4, I.2.8, and I.2.11.

The 2025 AI Industry Engineering Reading List: Q3 Update

The Latent.Space Reading List for 2025, published 12/27/2024, is certainly not a bad start and definitely gives an idea of what we want to develop as we go forward.

From Scaling to Agency — The New AI Engineering Paradigm

The field of Artificial Intelligence engineering has undergone a fundamental paradigm shift between late 2024 and the third quarter of 2025. The previous era, defined by the relentless pursuit of scale as the primary driver of capability, has given way to a more nuanced and complex landscape. While the principles of scaling laws remain foundational, the frontier of AI is no longer solely defined by parameter counts. Instead, the current state of the art is characterized by the rise of three interconnected pillars: agentic AI, native multimodality, and advanced reasoning.

This updated reading list reflects this new reality. It moves beyond the foundational models and techniques that characterized the 2024 curriculum to address the challenges and opportunities of building, deploying, and managing systems that are increasingly autonomous, perceptive, and capable of complex problem-solving.

The first pillar, Agentic AI, marks the transition of artificial intelligence from a reactive tool that responds to prompts into a proactive, goal-driven collaborator. This shift is not theoretical; it is a commercial and scientific reality, with enterprises rapidly adopting agentic systems to automate complex, cross-functional workflows and research labs deploying AI "co-scientists" to accelerate discovery.1 The engineering challenges have consequently evolved from prompt engineering to agentic orchestration, state management, and the mitigation of novel systemic risks.

The second pillar is the emergence of Natively Multimodal Systems. The architectural convergence of text, vision, audio, and video processing into single, unified models has rendered previous modality-specific categories obsolete.4 Frontier models are now designed from the ground up to perceive, reason about, and generate content across a seamless spectrum of data types. For the AI engineer, this means the era of the siloed "NLP" or "Computer Vision" specialist is waning; proficiency across the full multimodal stack is now a baseline requirement.

Finally, the third pillar is a dedicated focus on Advanced Reasoning. Responding to enterprise demands for tangible ROI and the need to solve increasingly complex problems, frontier models are now explicitly designed and evaluated on their ability to perform multi-step logical inference.6 This moves beyond the pattern matching and knowledge retrieval capabilities of previous generations, demanding architectures and training methodologies that foster genuine problem-solving abilities.

This report is structured to provide a comprehensive roadmap for the modern AI engineer navigating this new paradigm. It is organized into ten re-evaluated categories, each featuring five seminal papers or technical reports that have defined the field in 2025. From the architectural principles of new frontier models to the specialized techniques for ensuring their safety and the hardware they run on, this list serves as a definitive guide to the state of the art.

Table 1: The Q3 2025 AI Engineering Reading List at a Glance

Part I: The New Model Frontier

1. Frontier Models & Architectures

The architectural landscape of frontier models in 2025 has consolidated around a set of core principles that represent a significant departure from the monolithic, dense transformer models of previous years. The defining innovations are the widespread adoption of Mixture-of-Experts (MoE) for efficient scaling and the native integration of multimodality through "early fusion" techniques. These are no longer experimental concepts but have become the standard for state-of-the-art systems, fundamentally altering the trade-offs between model size, computational cost, and capability. Understanding these new architectures is the starting point for any contemporary AI engineer.

Core Papers

1. "GPT-5 System Card" (OpenAI, Aug 2025)
   This document is foundational for understanding the state of the art in production AI systems. It details the architecture of GPT-5, revealing a strategic shift towards a heterogeneous system of models rather than a single, monolithic network. The system comprises a fast, high-throughput model for routine queries, a deeper, more computationally intensive model for complex reasoning, and a real-time router that dynamically allocates requests based on complexity and user intent.9 This architecture is a direct engineering solution to the challenge of providing both low-latency responses and high-quality reasoning within a single product. The system card also substantiates GPT-5's state-of-the-art performance on difficult coding benchmarks like SWE-bench and highlights its advanced agentic capabilities, which allow it to autonomously use tools to accomplish tasks.9
2. "Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities" (Google, Jul 2025)
   This technical report from Google is crucial for understanding the continued evolution of sparse Mixture-of-Experts (MoE) architectures.11 Gemini 2.5 is presented as Google's first "fully hybrid reasoning model," a significant engineering advancement that gives developers granular control over the trade-off between performance and cost. The model introduces the concept of a "thinking budget," allowing users to turn a model's deep reasoning capabilities on or off and set explicit limits on computational expenditure.11 This feature addresses a key pain point for enterprise applications, where balancing quality, cost, and latency is paramount. The report also details the model's native multimodal support and its ability to process up to three hours of video content, showcasing the power of its sparse MoE design.12
3. "Llama 4 Technical Analysis" (Meta, Apr 2025)
   While not a traditional academic paper, the collection of technical blogs and deep-dive analyses on Meta's Llama 4 family of models is essential reading for understanding the frontier of open-weight AI. These documents deconstruct the model's key architectural innovations. First is its sparse MoE design, which allows it to achieve massive parameter counts while maintaining computational efficiency.13 Second is its native multimodality, achieved via an "early fusion" strategy that processes text and visual tokens jointly from the first layer. Third is the novel Interleaved Rotary Position Embedding (iRoPE), an architectural pattern that alternates between layers with and without positional encodings to effectively manage an industry-leading context window of up to 10 million tokens in its "Scout" variant.13
4. "A 2025 Guide to Mixture-of-Experts for Lean LLMs" (Consolidated Survey)
   This selection represents a synthesis of recent work that has solidified MoE as a core competency for AI engineers. Such a guide explains the fundamental anatomy of an MoE layer, including the expert subnetworks and the routing (or gating) mechanism that directs tokens to a small subset of experts for processing.15 It details crucial training techniques, such as the use of auxiliary load-balancing losses to prevent a few experts from dominating the computation. Critically for engineers, it also covers production deployment strategies, such as expert parallelism, where different experts are sharded across different GPUs, and techniques like DeepSpeed-Inference that enable efficient serving of these massive-but-sparse models.15
5. "Claude Opus 4.1 Announcement" (Anthropic, Aug 2025)
   The release of Claude Opus 4.1 solidified Anthropic's position as a leader in building highly capable models with a safety-first engineering ethos. The announcement and accompanying technical documentation position the model as a top performer on complex agentic and coding tasks, rivaling other frontier systems.17 For an AI engineer, this model is significant not just for its capabilities but for the principles underlying its development. Anthropic's research on "Constitutional AI" and its public commitment to safety frameworks represent a crucial perspective on how to build and deploy powerful AI systems responsibly, making their technical releases essential reading for understanding the intersection of capability and safety.

Analysis of the New Architectural Paradigm

The architectural trends of 2025 reveal a clear departure from the pursuit of ever-larger dense models. The frontier is now defined by complexity, heterogeneity, and efficiency. One of the most significant shifts is the end of the monolithic model as the sole architectural pattern. The design of GPT-5, with its explicit router directing traffic between a fast, high-throughput model and a more powerful reasoning model, is a case in point.9 This is mirrored by Google's "hybrid reasoning" approach in Gemini 2.5, which allows for dynamic allocation of computational "thinking" resources.11 This evolution is a direct response to the diverse needs of enterprise applications, where a single, one-size-fits-all model is inefficient. A simple summarization task does not require the same computational budget as a multi-step agentic workflow that plans and executes a series of actions. Consequently, the engineering challenge has pivoted from simply training one massive model to designing, orchestrating, and optimizing a system of specialized models that work in concert. This introduces new complexities in API design, inference routing, and cost management.

Parallel to this move towards heterogeneity, Mixture-of-Experts has become table stakes for achieving performance at the frontier. Both Google's Gemini 2.5 and Meta's Llama 4 are explicitly built on sparse MoE architectures.11 The primary advantage of this approach is the decoupling of a model's total capacity (its total number of parameters) from its computational cost per token.15 By activating only a small subset of "expert" subnetworks for any given input, MoE allows for the creation of models with trillions of parameters that can be trained and served with a fraction of the computation required by an equivalent dense model. The success of open-weight MoE models like Mixtral paved the way for this widespread adoption.16 For the AI engineer, this means that a deep understanding of the principles of MoE—including gating networks, load-balancing mechanisms, and expert parallelism for distributed inference—is no longer a niche specialty but a fundamental requirement for working with state-of-the-art models.

2. Scaling Laws & Model Efficiency

The classic scaling laws that defined the previous era of AI development provided a simple, powerful heuristic: more compute, more data, and more parameters predictably yield better models. While this principle remains a guiding force, the understanding of scaling in 2025 has become far more sophisticated. The conversation has expanded to encompass the entire model lifecycle, with new "laws" emerging for post-training optimization and inference-time compute allocation.19 Simultaneously, a powerful counter-trend has gained momentum: the rise of highly efficient Small Language Models (SLMs) that achieve performance comparable to much larger predecessors by leveraging extremely high-quality, curated data. This challenges the "bigger is always better" mantra and presents engineers with a more complex and nuanced optimization landscape.

Core Papers

1. "How Scaling Laws Drive Smarter, More Powerful AI" (NVIDIA, Feb 2025)
   This conceptual paper from NVIDIA is pivotal because it articulates and popularizes an expanded framework for scaling laws, moving beyond the singular focus on pre-training. It formally distinguishes between three distinct phases of scaling.19
   Pre-training scaling is the classic law where performance improves with data, model size, and compute. Post-training scaling describes performance gains achieved through subsequent optimization steps like domain-specific fine-tuning, quantization, pruning, and knowledge distillation. Test-time scaling, also referred to as "long thinking," involves applying additional compute during inference to improve the quality of a single output. This framework is essential for modern AI engineers, as it provides a holistic view of performance optimization across the entire model lifecycle.
2. "Training Compute-Optimal Large Language Models" (DeepMind, 2022)
   Known as the "Chinchilla" paper, this is a retrospective but indispensable inclusion. Its core finding—that for optimal performance, model size and training dataset size must be scaled in proportion—is more relevant than ever in an environment of escalating compute and data curation costs.20 It established that many earlier large models were significantly undertrained, having been scaled up in parameter count without a corresponding increase in data. The Chinchilla scaling laws serve as the theoretical baseline against which the efficiency and design of all modern models are measured, making it required reading for anyone training or selecting a model.
3. "Small Language Models are the Future of Agentic AI" (arXiv, Jun 2025)
   This influential position paper presents a compelling counter-narrative to the race for ever-larger frontier models. It argues that for the vast majority of specialized and repetitive tasks common in agentic systems, SLMs are not only sufficient but are operationally more suitable and economically necessary.21 The paper posits that the flexibility, low inference cost, and ease of fine-tuning make SLMs the ideal choice for building modular and scalable agentic applications. This is a critical perspective for engineers focused on building practical, cost-effective products, suggesting that a fleet of specialized SLMs may be superior to a single, monolithic LLM.
4. "Phi-3 Technical Report" (Microsoft, 2024) / "Gemma 3 270M Announcement" (Google, Aug 2025)
   This selection represents the technical documentation for a state-of-the-art SLM. The report for a model like Microsoft's Phi-3 or Google's compact Gemma 3 270M is essential as it details the training methodology that enables such high performance from a small parameter count.22 These models are trained on smaller, but extremely high-quality and carefully curated, "textbook-like" data. This approach demonstrates a key principle of modern efficiency: data quality can be a direct substitute for model scale, a lesson of immense practical importance for engineering teams with finite resources.
5. "Inference Scaling Laws and Compute-Optimal Inference" (Wu et al., 2024)
   This research paper provides the formal theoretical underpinnings for the concept of test-time scaling. It systematically studies the trade-offs between a model's size (pre-trained capacity) and the amount of computation invested during inference, such as generating multiple candidate responses and selecting the best one.24 This work is crucial for understanding and optimizing production inference systems, as it provides a mathematical basis for techniques like speculative decoding, self-consistency, and other methods that improve output quality by using more compute at inference time.

Analysis of the Evolving Optimization Landscape

The field of AI engineering in 2025 is defined by a more complex, multi-dimensional optimization space than ever before. The simple question of "which is the biggest model I can afford?" has been replaced by a sophisticated analysis across a new Pareto frontier. The original scaling laws presented a one-dimensional path: bigger models trained on more data yielded better performance.19 The Chinchilla paper refined this into a two-dimensional trade-off, demonstrating the need to balance model size with data volume for compute-optimal training.20 The rise of SLMs like Phi-3, trained on highly curated datasets, introduced a third dimension: data

quality can be traded for model size.22 Finally, the formalization of post-training and test-time scaling laws adds further dimensions to this optimization problem. An engineering team can now achieve superior performance not just by selecting a larger base model, but by investing their compute budget in targeted fine-tuning or more intensive inference-time reasoning.19 An AI engineer in 2025 must therefore navigate this complex, multi-dimensional space, where a smaller, more efficient SLM, combined with domain-specific finetuning and advanced test-time reasoning, might outperform a larger, more generic model at a fraction of the total cost.

This shift has elevated the role of data curation from a preliminary step to a dominant driver of cost and value. As models become more efficient at learning from data, the primary bottleneck and key competitive differentiator is no longer just access to raw compute, but access to high-quality, diverse, and clean training datasets. The success of models like Phi-3 is explicitly attributed to the quality of their training data, not its sheer volume.22 The 2025 AI Index Report notes that while training compute for notable models doubles approximately every five months, the size of their datasets doubles only every eight months, indicating a growing premium on high-value data.25 This trend is compounded by intensifying legal and ethical challenges surrounding data usage, such as the copyright lawsuits faced by major labs.26 As a result, the role of the "Data Engineer for AI" has become as critical as that of the ML engineer. The processes of sourcing, cleaning, synthesizing, and curating data are no longer preparatory tasks but core, ongoing activities that directly determine the quality, safety, and competitive advantage of AI systems.

Part II: Core Engineering Tooling & Techniques

3. Advanced Retrieval & Augmentation

Retrieval-Augmented Generation (RAG) has firmly established itself as an indispensable component of the modern AI stack. In 2025, the conversation has moved beyond the novelty of the technique to the engineering challenges of making it robust, dynamic, and capable of handling the multimodal nature of modern AI. The simple "retrieve-then-generate" pipeline of early RAG systems is being replaced by sophisticated, multi-step workflows that can manage conversational context, reconcile conflicting information from multiple sources, and operate over a diverse range of data types including images, audio, and video. This evolution reflects the maturation of RAG from a clever trick to a core engineering discipline.

Core Papers

1. "MTRAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systems" (IBM, ACL 2025)
   This paper is critical because it addresses a primary failure mode of first-generation RAG systems: maintaining context and relevance in multi-turn conversations. Simple RAG often fails when a user's follow-up question depends on the history of the dialogue. MTRAG provides the first human-generated benchmark specifically designed to evaluate this capability across multiple domains.27 The findings show that even state-of-the-art systems struggle with these challenges, underscoring the need for more advanced retrieval and context management strategies and providing a clear target for engineering improvement.
2. "RAG-Critic: Leveraging Automated Critic-Guided Agentic Workflow for Retrieval Augmented Generation" (ACL 2025)
   This work exemplifies the powerful fusion of RAG with the agentic AI paradigm. Instead of a static, one-shot retrieval step, this approach introduces an LLM-based "critic" that evaluates the retrieved information and guides a dynamic, iterative process of information seeking.28 The system can refine queries, seek additional sources, and synthesize information over multiple steps. This transforms RAG from a simple pipeline into an intelligent, goal-driven workflow, representing a significant leap in sophistication and robustness.
3. "REAL-MM-RAG: A Real-World Multi-Modal Retrieval Benchmark" (ACL 2025)
   As frontier models become natively multimodal, RAG systems must evolve to support them. This paper introduces a crucial benchmark for this new frontier: multimodal RAG.28 It evaluates a system's ability to retrieve and reason over a combination of text, images, and other data formats. This defines the next major challenge for RAG engineering, which must now move beyond text-centric vector databases and develop new methods for embedding, indexing, and ranking complex, multi-format data sources to ground language in a rich, multimodal world.
4. "TreeRAG: Unleashing the Power of Hierarchical Storage for Enhanced Knowledge Retrieval in Long Documents" (ACL 2025)
   This paper presents a practical engineering solution to a common and persistent pain point: performing effective RAG over long, structured documents. A flat, chunk-based retrieval approach often fails to capture the hierarchical nature of documents like technical manuals, legal contracts, or research papers. TreeRAG proposes a hierarchical storage and retrieval strategy that respects the document's structure, allowing for more precise and contextually aware information retrieval.29 This is a vital technique for building enterprise-grade RAG applications that work with complex, real-world documents.
5. "Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Models" (Google, ACL 2025)
   This paper from Google tackles the "garbage-in, garbage-out" problem inherent in RAG. When a system retrieves noisy, irrelevant, or contradictory information, the LLM's output quality degrades significantly. Astute RAG focuses on techniques for identifying and handling these knowledge conflicts.30 By developing mechanisms to assess the reliability of retrieved sources and reconcile conflicting facts, this work provides a pathway to building more robust and trustworthy production-grade RAG systems, a crucial step for applications in high-stakes domains.

Analysis of the Evolving RAG Paradigm

The evolution of RAG in 2025 demonstrates a clear trajectory from a simple pipeline to a dynamic, agentic process. The initial "retrieve-then-generate" paradigm, while effective for single-shot Q&A, has been shown to be brittle in more complex scenarios. The limitations exposed by conversational benchmarks like MTRAG, where the information need evolves with the dialogue, necessitate a more adaptive approach.27 The solution, as framed by papers like RAG-Critic, is to re-imagine retrieval as an "agentic workflow".28 In this new model, the AI system is an active information seeker, equipped with tools for querying, evaluating, and synthesizing information over multiple steps to achieve a goal. This mirrors the broader industry trend towards agentic AI, where information retrieval becomes one of many tools in an agent's arsenal.1 For engineers, this means that building a state-of-the-art RAG system is now less about configuring a vector database and more about designing a robust agentic loop. This requires a new set of skills in workflow orchestration, state management, and multi-step reasoning, fundamentally increasing the complexity and power of the RAG stack.

Furthermore, as the underlying models become natively multimodal, the RAG layer must follow suit, creating a new frontier that exposes the limitations of today's text-centric infrastructure. Frontier models like GPT-5 and Gemini 2.5 can process interleaved sequences of text, images, and video.9 Consequently, users expect to be able to ask questions that require retrieving information from these diverse modalities. New benchmarks like REAL-MM-RAG are being created specifically to test this capability.28 This shift effectively breaks the existing RAG infrastructure, which is heavily optimized for text. The processes of vectorizing images, chunking video streams, and performing cross-modal relevance ranking are all non-trivial, open research problems. The next wave of RAG engineering will therefore be defined by the challenge of building truly multimodal data pipelines and retrieval systems. This will require significant investment in new embedding models, novel indexing strategies, and more sophisticated data structures, representing a major area of innovation for infrastructure and application teams alike.

4. Finetuning & Preference Optimization

In an ecosystem where access to powerful base models is increasingly commoditized, the art and science of finetuning has become a primary driver of product differentiation and competitive advantage. By 2025, Parameter-Efficient Fine-Tuning (PEFT) methods, particularly LoRA and its variants, have become standard industry practice for adapting models to specific domains.32 The frontier of research and engineering has consequently moved to more sophisticated methods of aligning model behavior with complex human preferences. The complexity of Reinforcement Learning from Human Feedback (RLHF) has given way to simpler yet powerful techniques like Direct Preference Optimization (DPO), which now form the foundation for advanced methods that aim to align models with a pluralistic and often conflicting set of human values.

Core Papers

1. "Direct Preference Optimization: Your Language Model is Secretly a Reward Model" (Stanford, 2023)
   This paper, while published in 2023, is included retrospectively as it is the foundational text for the current era of preference optimization. DPO represented a paradigm shift by demonstrating that the complex, multi-stage process of RLHF could be replaced by a simple classification loss on preference data. By directly optimizing the language model to satisfy human preferences, it eliminated the need to train a separate reward model and the instabilities of reinforcement learning. By Q3 2025, DPO is the baseline technique upon which nearly all modern alignment methods are built, making it an essential concept for any engineer involved in model tuning.
2. "SDPO: Segment-Level Direct Preference Optimization for Social Agents" (arXiv, Jan 2025)
   This paper represents the natural evolution and refinement of DPO. It addresses a key limitation of the original method, which optimizes an entire generated response against another. SDPO introduces a more granular approach, optimizing specific segments of a response based on preference data.33 This allows for much finer-grained control over model behavior, which is particularly crucial for nuanced applications like social chatbots, where specific phrases or tones can dramatically alter the user's perception. It showcases the move towards more precise and targeted alignment techniques.
3. "On The Impact of Preference Alignment On Trustworthiness" (ICLR 2025)
   This is a critical paper that serves as a vital reality check for the field of AI alignment. Through a systematic study, it demonstrates that naively optimizing for general human preferences via RLHF or DPO does not uniformly improve all aspects of model trustworthiness. The research found that while such alignment significantly improved machine ethics, it also dramatically increased stereotypical bias and reduced truthfulness.34 This counterintuitive result highlights the complex, non-monotonic relationships between different human values and is essential reading for anyone involved in responsible AI development. It proves that alignment is not a simple optimization problem but a complex balancing act.
4. "MAP: Multi-Human-Value Alignment Palette" (ICLR 2025)
   Responding directly to the challenge identified in the previous paper, MAP offers a sophisticated solution to the problem of multi-objective alignment. It reframes the goal from maximizing a single, monolithic preference score to optimizing a model's behavior within a multi-dimensional "palette" of human values.34 This framework allows practitioners to define target levels and constraints for competing objectives—for example, balancing harmlessness with helpfulness, or factual accuracy with a humorous tone. MAP represents the state-of-the-art in thinking about how to align AI with the complex, pluralistic, and often contradictory nature of human values.
5. "LoRA Done RITE: Robust Invariant Transformation Equilibration for LoRA Optimization" (ICLR 2025)
   While LoRA is an established PEFT technique, mastering its application for optimal results is a key engineering challenge in 2025. This paper delves into the practical science of making LoRA more effective. It introduces advanced techniques for improving the robustness and efficiency of LoRA-based finetuning, ensuring that the learned adaptations generalize well and are less sensitive to hyperparameters.35 For an engineer in the trenches, this type of paper is invaluable, as it provides the deep, practical knowledge required to move from simply using a library to achieving state-of-the-art results in production.

Analysis of Modern Finetuning and Alignment

The practice of aligning AI models in 2025 has matured significantly, moving beyond the monolithic goal of making a model "better" to a multi-objective balancing act. The simplistic notion of a single axis of "helpfulness and harmlessness" is now obsolete. The stark findings from ICLR 2025—that optimizing for general human preferences can inadvertently degrade crucial dimensions like truthfulness and fairness—have forced the field to confront the complexity of real-world values.34 A response can be factually correct but harmful, or helpful but biased. This realization has shifted the engineering problem from simple optimization to constrained optimization. Frameworks like MAP, which allow for the explicit definition of a "palette" of values and their trade-offs, are the direct result of this shift.34 This requires AI engineers to adopt a more interdisciplinary mindset, working alongside product managers, ethicists, and social scientists to define the explicit, multi-dimensional value sets that are appropriate for their specific applications. This, in turn, demands more sophisticated approaches to preference data collection, labeling, and the training process itself.

Concurrently, the abstraction layer for performing this complex finetuning has solidified, making it more accessible than ever. The prohibitive cost of full-finetuning frontier models was first addressed by PEFT methods like LoRA, which made adaptation feasible with limited resources.32 However, the subsequent alignment step, RLHF, remained a complex and resource-intensive process requiring a separate reward model and a reinforcement learning pipeline. The introduction of DPO dramatically simplified this process by reframing it as a more stable and straightforward classification task. The combination of LoRA for parameter-efficient adaptation and DPO (or its more advanced variants) for preference optimization has now become the standard, powerful, and relatively simple stack for model customization. This standardization is a democratizing force, empowering a much broader range of teams to build highly specialized and aligned models, thereby accelerating the proliferation of AI into countless niche domains.

5. Evaluation, Benchmarking & Observability

As the capabilities of AI models have surged, the tools and methodologies used to measure them have been forced to evolve at a breakneck pace. Traditional academic benchmarks like MMLU, once considered challenging, have become saturated by frontier models, pushing the research community to devise new, more difficult tests that probe advanced reasoning, coding, and agentic capabilities. Concurrently, the industry is grappling with a distinct but related challenge: the engineering of production-grade observability. The task of reliably monitoring, evaluating, and debugging complex, non-deterministic AI systems in live environments has emerged as a critical discipline, distinct from offline benchmarking, and is essential for ensuring the safety, reliability, and business value of deployed AI.

Core Papers

1. "AI Index Report 2025" (Stanford HAI)
   This annual report serves as an essential state-of-the-union for the AI field, providing critical context on evaluation trends. The 2025 edition highlights the rapid performance gains on a new generation of difficult benchmarks, including GPQA (graduate-level Q&A) and SWE-bench (real-world software engineering tasks), demonstrating how quickly the frontier of measurable capability is advancing.36 Crucially, the report also sounds an alarm, noting that standardized evaluations for Responsible AI (RAI) remain rare among major developers, even as the number of documented AI-related incidents rises sharply.25 This frames the dual challenge for the field: capabilities are out-pacing our ability to reliably measure them, especially on safety dimensions.
2. "Berkeley Function Calling Leaderboard (BFCL)" (ICML 2025)
   Function and tool calling is the fundamental mechanism that enables agentic AI. This paper introduces what has become the de-facto industry standard for evaluating this critical capability.37 The BFCL is comprehensive, testing a model's ability to handle not only simple, single-turn function calls but also complex scenarios involving serial and parallel calls, multi-turn interactions where state must be maintained, and, importantly, the ability to correctly abstain when a query cannot or should not be fulfilled by a tool. For any engineer building an AI agent, performance on this benchmark is a key indicator of a model's utility.
3. "AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents" (ICLR 2025)
   As AI systems become more autonomous, the potential for them to cause harm increases. This paper addresses the urgent need for safety evaluations that go beyond passive text generation. AgentHarm introduces a benchmark specifically designed to measure the potential for harmful behavior in goal-seeking LLM agents.38 It presents agents with scenarios where they could achieve their objectives through unsafe, unethical, or harmful means, testing the robustness of their safety alignment in active, decision-making contexts. This represents a critical shift in safety evaluation from content moderation to behavioral assessment.
4. "Libra-Leaderboard: A Balanced Leaderboard for LLMs" (NAACL 2025)
   This work offers a trenchant critique of existing LLM leaderboards, arguing that their near-exclusive focus on capability metrics creates perverse incentives for developers to prioritize performance over safety.39 To counteract this, Libra-Leaderboard proposes a novel evaluation framework that explicitly balances performance and safety. It uses a "distance-to-optimal-score" method to rank models, rewarding holistic excellence rather than one-dimensional capability. This represents a more mature and responsible approach to public model evaluation, reflecting the growing consensus that safety and capability must be developed in tandem.
5. "The Practice of Production AI Observability: A Survey" (Consolidated Survey)
   This selection represents a foundational paper or survey detailing the practical engineering challenges of monitoring deployed AI systems. While academic benchmarks are vital, production observability is a distinct discipline. This paper would cover the emerging best practices for monitoring key performance indicators in live AI applications. This includes not only system metrics like latency, token usage, and cost, but also quality metrics like hallucination rates, user feedback scores, and drift detection. It would also detail the importance of robust offline evaluation pipelines, which over 50% of companies rely on as a primary method for monitoring their AI systems.32

Analysis of the Duality in Evaluation

The field of AI evaluation in 2025 is characterized by a fundamental shift in focus from what a model knows to what it can do. Early benchmarks, such as MMLU, were primarily tests of a model's crystallized knowledge across a wide range of academic and professional subjects. As the AI Index Report documents, these benchmarks are now being rapidly mastered by frontier models, necessitating the creation of new and more challenging evaluations.36 This new wave of benchmarks, including SWE-bench for coding, BFCL for function calling, and AgentHarm for safety in autonomous systems, are process-oriented.36 They do not test static knowledge but rather a model's ability to execute a sequence of actions—using tools, writing code, making decisions—to achieve a complex goal. This evolution in benchmarking directly mirrors the economic driver of the field: the enterprise shift towards using AI to automate entire workflows, not just to answer isolated questions.1 As a result, the very definition of a "capable" model has been transformed. AI engineers must now design evaluation pipelines that are more complex, interactive, and process-driven to accurately assess a model's fitness for these new, agentic tasks.

This shift has also exposed a significant chasm between the sophistication of academic benchmarking and the practical realities of production observability. While the research community develops intricate, static, offline benchmarks, the industry is still grappling with the foundational challenge of gaining reliable, real-time insights into the behavior of dynamic, non-deterministic AI systems deployed in the wild. The Amplify Partners report reveals that a majority of teams still rely heavily on offline evaluation and "standard observability" tools, which are often ill-suited for the unique failure modes of AI.32 The pressing need to measure business ROI and ensure responsible AI practices in production is paramount.7 This gap represents a major area of opportunity and necessity in AI engineering: the development of a new "AI Observability" stack. This stack must go far beyond traditional software metrics like latency and error rates. It needs to incorporate robust tracking for model-specific issues such as hallucinations, bias drift, prompt injection attacks, and alignment degradation over time. Building these systems is a complex, unsolved engineering problem that will be a primary focus for the industry in the coming years.

Part III: The Agentic & Multimodal Shift

6. Agentic AI & Autonomous Systems

The concept of agentic AI has explosively transitioned from a research curiosity to a core enterprise strategy in 2025, representing the most significant paradigm shift of the year. The industry is moving rapidly to deploy systems where the AI acts as a proactive, goal-driven collaborator rather than a passive tool. A survey from PwC revealed that 79% of companies are already adopting AI agents, with budgets surging to support these initiatives.1 The frontier of this work is not in single-purpose bots, but in sophisticated multi-agent systems, where specialized agents collaborate to automate complex, end-to-end business and scientific workflows. Understanding the architectural patterns, capabilities, and risks of these systems is now a non-negotiable skill for AI engineers.

Core Papers

1. "Seizing the agentic AI advantage" (McKinsey, Jun 2025)
   This report provides the essential business and strategic context for the agentic AI revolution. Written for a C-suite audience, it articulates why agentic AI is poised to unlock scalable impact where previous generative AI applications have struggled.2 It introduces the concept of the "agentic AI mesh," a new architectural paradigm for orchestrating fleets of both custom-built and off-the-shelf agents. For engineers, this paper is vital for understanding the business drivers behind agentic systems and for learning the language needed to design and justify architectures that can manage the new classes of risk and technical debt that autonomous systems introduce.
2. "Agentic AI for Scientific Discovery: A Survey of Progress, Challenges, and Future Directions" (ICLR 2025)
   This comprehensive survey paper provides a scholarly overview of one of the most impactful application domains for agentic AI: scientific research.41 It categorizes existing systems and tools that are transforming how scientists perform literature reviews, generate hypotheses, design experiments, and analyze results. The paper highlights progress across diverse fields such as chemistry, biology, and materials science, offering a detailed map of the current state of the field. It is an indispensable resource for engineers looking to apply agentic principles to complex, knowledge-intensive domains.
3. "AI co-scientist: plan and advance your research with AI assistance" (Google, Feb 2025)
   This landmark paper from Google provides a concrete, powerful implementation of the concepts discussed in the survey above.3 It details the "AI co-scientist," a multi-agent system built on the Gemini 2.0 model. The system is designed to function as a collaborative tool for scientists, capable of generating novel and experimentally verifiable research hypotheses. The paper describes the system's architecture, which includes a "Supervisor" agent that decomposes a research goal and assigns tasks to a team of specialized worker agents. This work is a powerful demonstration of the tangible potential of multi-agent systems to accelerate scientific discovery.
4. "Magma: A Foundation Model for Multimodal AI Agents" (CVPR 2025)
   This paper is crucial because it forges the link between agency and multimodality, the two defining trends of 2025. Magma is a foundation model trained not just to understand language and vision, but to act within visual environments.44 The researchers introduce novel data labeling techniques—"Set-of-Mark" (SoM) for actionable objects and "Trace-of-Mark" (ToM) for motion trajectories—to ground the model's actions in visual data. Magma achieves state-of-the-art results on tasks like UI navigation and robotic manipulation, demonstrating how native multimodal understanding is a prerequisite for building capable embodied agents.
5. "Agent S: An Open Agentic Framework that Uses Computers Like a Human" (ICLR 2025 Workshop)
   This paper represents the practical, open-source engineering work required to build general-purpose digital agents. Agent S is an open framework designed to enable AI agents to interact with standard computer graphical user interfaces (GUIs) just as a human would—by looking at the screen and using a mouse and keyboard.38 This line of research is critical for unlocking the potential of AI to automate the vast number of digital tasks that constitute modern knowledge work. It provides a blueprint for the type of foundational frameworks that will underpin many future agentic applications.

Analysis of the Agentic Paradigm Shift

The dominant architectural pattern that has emerged for building sophisticated agentic AI is the multi-agent system. While a single, powerful agent can perform isolated tasks, the complexity of real-world, end-to-end business and scientific processes necessitates a collaborative approach. The most effective and scalable solutions are not monolithic agents but are instead systems of specialized agents orchestrated by a higher-level framework. McKinsey's proposal of an "agentic AI mesh" is a strategic vision for this, providing a governance layer to integrate numerous custom and off-the-shelf agents across an enterprise.2 This vision is mirrored in the technical implementation of Google's "AI co-scientist," which employs a Supervisor agent to manage a team of specialized workers for tasks like literature review and experimental design.3 With industry surveys showing that 99% of developers are exploring agents, the engineering focus is rapidly shifting from building individual agents to designing robust, scalable, and observable multi-agent architectures.46 This introduces a new set of complex engineering challenges, including inter-agent communication protocols, credit assignment in collaborative tasks, and managing the unpredictable emergent behaviors of complex adaptive systems.

This shift towards autonomy also introduces a new and more severe class of systemic and security risks. A passive generative model can produce harmful content, but an autonomous agent can execute harmful actions—deleting files, sending unauthorized communications, or manipulating physical systems. The risks of "uncontrolled autonomy" and an "expanding surface of attack" are identified by McKinsey as fundamental new challenges posed by agentic AI.2 Research from safety-focused labs like Anthropic is now explicitly modeling these "agentic misalignment" scenarios, exploring how an LLM could be co-opted to act as a malicious insider threat.18 This elevates the importance of safety from a content moderation problem to a systems security problem. Deploying autonomous agents safely requires a fundamentally different engineering approach, demanding robust sandboxing, fine-grained permissioning, and real-time monitoring and intervention capabilities. The "move fast and break things" ethos of traditional software development is dangerously incompatible with this new paradigm.

7. Natively Multimodal Systems

The historical division of AI into distinct fields like Natural Language Processing, Computer Vision, and Speech Recognition is an artifact of past model limitations. In 2025, this separation has become obsolete at the research frontier. The state of the art is now defined by natively multimodal systems—single, unified models that are trained from the ground up to process, understand, and generate information across a seamless spectrum of data types. This architectural convergence is a profound shift, meaning that every AI engineer must now, to some extent, be a multimodal engineer. This single, unified category replaces the separate Vision, Voice, and Image/Video Diffusion sections from the previous year's reading list, reflecting the integrated nature of modern AI.

Core Papers

1. "GPT-4o Technical Report" (OpenAI, May 2024)
   This paper is included retrospectively as it marked the definitive arrival of true native multimodality in a widely accessible, production-grade model. GPT-4o was the first major model to demonstrate the ability to process text, audio, and vision end-to-end within a single neural network, enabling fluid, real-time conversational interactions that were previously impossible.5 Its architecture, which tokenizes and processes all modalities in a unified sequence, set the stage for the wave of natively multimodal models that followed in 2025 and remains a crucial reference point.
2. "SAM 2: Segment Anything in Images and Videos" (ICLR 2025)
   The original Segment Anything Model (SAM) was a revolutionary step in computer vision, providing a foundational model for zero-shot segmentation. SAM 2 extends this powerful capability to the temporal domain of video—a significantly more complex challenge.35 The ability to consistently identify and track objects and segments through time is a fundamental building block for deep video understanding. As a foundational vision capability, SAM 2 underpins many of the more advanced multimodal systems that reason about dynamic scenes, making it essential reading.
3. "Genie 3: a general purpose world model that can generate a diversity of interactive environments" (Google, Aug 2025)
   Genie 3 represents the pinnacle of generative multimodality in 2025. It moves beyond the generation of passive video clips to the creation of dynamic, interactive, and navigable 2D worlds from a variety of inputs, including text and images.23 This is a "world model"—an AI system that learns an internal simulation of the world's dynamics. Its ability to generate playable environments is a key technological step towards training more capable embodied AI agents in rich, simulated worlds, marking a new frontier for generative AI.
4. "Introducing the V-JEPA 2 world model and new benchmarks for physical reasoning" (Meta, Jun 2025)
   Meta's V-JEPA 2 offers a different architectural philosophy for building world models. Unlike the generative approach of Genie, V-JEPA 2 is a non-generative, self-supervised model trained on video to learn a predictive model of how the world works.48 It learns to predict what will happen next in a scene at an abstract representation level, rather than generating pixels. This approach is designed to learn more efficient and generalizable representations of physical dynamics, which are crucial for building agents that can plan and act effectively in the real world.
5. "Matryoshka Multimodal Models" (ICLR 2025)
   This paper presents a clever and practical architectural innovation for building efficient multimodal models. Inspired by Matryoshka dolls, the model learns a nested set of visual representations at different levels of granularity or resolution.47 This allows for a flexible trade-off between performance and computational cost at inference time; a simple query might use a coarse representation, while a more complex query can access finer-grained details at a higher computational cost. This is a key engineering technique for deploying powerful but resource-intensive multimodal models in a cost-effective manner.

Analysis of the Multimodal Convergence

The most profound impact of the multimodal shift is the convergence of the AI engineering stack. The old paradigm of building applications by "stitching together" separate, siloed models for text, vision, and audio is being rapidly replaced by the use of single, unified systems. Models like GPT-4o, Gemini 2.5, and Llama 4 are natively multimodal, operating on a single, interleaved sequence of tokens that can represent text, image patches, or audio snippets.5 This "early fusion" architecture enables a much deeper and more nuanced level of cross-modal understanding than was possible with late-fusion approaches.13 This architectural shift has cascading implications for the entire engineering infrastructure. Vector databases must now be able to store and query multimodal embeddings. Data annotation platforms must support complex labeling tasks that span multiple data types. APIs must be redesigned to gracefully handle mixed-media inputs. The era of the narrowly specialized "NLP Engineer" or "Computer Vision Engineer" is giving way to the "AI Engineer," who must be proficient across all modalities and capable of designing systems that are multimodal from the ground up.

Within this new multimodal paradigm, the frontier of generative AI is decisively shifting towards "world models." The focus of 2023 and 2024 was on generating static or passive content: images with DALL-E and Midjourney, and then non-interactive videos with models like Sora. In 2025, leading research labs like Google with Genie 3 and Meta with V-JEPA 2 have explicitly reoriented their efforts toward building models that learn an internal, predictive simulation of the world.43 These systems are trained on vast quantities of video data to learn the underlying principles of physics, object permanence, and agent interactions. The ultimate goal is not merely to create a visually plausible video, but to generate a dynamic, interactive, and physically consistent

simulation. This represents a foundational technology for the next generation of robotics and embodied AI, as these learned world models will serve as the rich, scalable "simulators" in which future autonomous agents are trained and tested before being deployed in the physical world.

Part IV: Specialized Applications & The Full Stack

8. Code & Scientific Generation

The application of AI to specialized, high-value domains like software engineering and scientific discovery has matured significantly in 2025. In coding, AI has evolved from a simple autocomplete tool into a genuine collaborator, with frontier models demonstrating state-of-the-art performance on complex, real-world software engineering tasks.10 Even more profoundly, the advanced reasoning and generative capabilities of these models are being harnessed to accelerate scientific breakthroughs. The paradigm is shifting from using AI to predict existing properties to using it to generate novel, functional designs in fields like biology, chemistry, and materials science.

Core Papers

1. "Introducing GPT-5 for Developers" (OpenAI, 2025)
   This announcement and its accompanying technical details are essential for understanding the state of the art in AI for code. It documents GPT-5's superior performance on challenging coding benchmarks like SWE-bench (74.9%) and Aider (88%), establishing it as the leading model for software development.10 Crucially, it frames the model not as a code generator but as a "coding collaborator," highlighting its ability to follow detailed instructions, fix bugs, and reason about complex codebases. Its capacity to reliably chain dozens of tool calls makes it particularly well-suited for agentic coding workflows, as validated by its adoption in leading AI developer tools.10
2. "Accelerating life sciences research with Retro Biosciences" (OpenAI, Aug 2025)
   This blog post represents a landmark case study in the application of AI to scientific generation. It details a collaboration where a specialized GPT model was used to design novel variants of the Yamanaka factors—proteins critical for cellular rejuvenation.49 The AI-designed proteins demonstrated a greater than 50-fold higher expression of key markers in vitro, a result validated across multiple cell types. This work is a powerful demonstration of AI moving beyond prediction (e.g., predicting the structure of existing proteins) to
   generation—designing new, functional biological entities with enhanced properties.
3. "MOOSE-Chem: Large Language Models for Rediscovering Unseen Chemistry Scientific Hypotheses" (ICLR 2025)
   This paper showcases the potential of LLMs to act as autonomous research assistants in the domain of chemistry. The MOOSE-Chem system uses an agentic framework to parse scientific literature, identify gaps in knowledge, and formulate novel, testable hypotheses.38 It demonstrates how LLMs can not only retrieve and summarize existing information but also synthesize it in a way that can guide future research directions, effectively rediscovering scientific insights from the vast corpus of published work.
4. "A Physics-Informed Machine Learning Framework for Safe and Optimal Control of Autonomous Systems" (ICML 2025)
   This paper is representative of a crucial trend: the integration of deep learning with classical scientific and engineering principles. Instead of treating a physical system as a black box to be learned from data alone, this framework explicitly incorporates the laws of physics as constraints or biases within the machine learning model.50 This "physics-informed" approach leads to models that are not only more accurate and data-efficient but also safer and more reliable, as their behavior is grounded in well-understood physical laws. This is a critical technique for applying AI to high-stakes engineering domains like robotics and autonomous systems.
5. "AlphaFold 3 Paper" (Google DeepMind, 2024)
   Included retrospectively, the paper detailing the latest version of AlphaFold is foundational for the entire field of AI in life sciences. AlphaFold 2 revolutionized biology by solving the protein folding problem. AlphaFold 3 expands this capability to predict the structure and interactions of nearly all of life's molecules, including DNA, RNA, and ligands.43 This tool provides the structural "parts list" of biology with unprecedented accuracy and scale. It is the foundational predictive model upon which the new wave of generative biology, as seen in the Retro Biosciences work, is being built. Understanding its capabilities is essential context for any engineer working in this space.

Analysis of AI in Specialized Domains

A clear paradigm shift is underway in the application of AI to scientific discovery, moving from prediction to generation. The groundbreaking success of AlphaFold was in predicting the structure of existing proteins, a monumental analytical achievement. The next frontier, as demonstrated by the OpenAI and Retro Biosciences collaboration, is designing novel proteins with new or enhanced functions.49 This generative approach is being applied across scientific domains, with AI models designing new molecules for drug discovery and proposing new materials with desired properties.51 This transforms AI from a powerful analytical tool that helps scientists interpret data into a creative partner that actively participates in the process of discovery. This shift is creating an entirely new discipline of "AI for Science" engineering, which requires a hybrid skillset blending deep learning expertise with deep domain knowledge in biology, chemistry, or physics to effectively design, train, and validate these powerful generative systems.

A parallel evolution is occurring in the domain of software engineering. The "AI coding assistant," which began as a sophisticated autocomplete, is rapidly maturing into an "AI software engineering agent." Early coding tools focused on generating lines or blocks of code in response to a specific prompt. State-of-the-art models like GPT-5, however, are designed to handle entire software engineering workflows.10 They can reason about large, complex codebases, identify and fix bugs, and execute multi-step plans that involve chaining together dozens of tool calls. This aligns with the broader agentic AI trend, where the AI is given a high-level goal (e.g., "refactor this module to improve performance") and is responsible for formulating and executing the necessary steps. The future of AI in software development is therefore not just about writing code faster, but about automating larger and more abstract parts of the development lifecycle. This will require human engineers to become adept at a new skill set: "AI-driven software development," which involves designing, managing, and collaborating with a team of autonomous AI agents.

9. AI Safety & Alignment

As AI systems have become more powerful, autonomous, and deeply integrated into society, the disciplines of AI safety and alignment have transitioned from niche academic pursuits to a critical, mainstream component of the engineering lifecycle. The conversation in 2025 has necessarily evolved to address the new challenges posed by the current generation of AI. The focus has shifted from the risks of passive content generation to the unique and more severe risks posed by agentic AI. Concurrently, the simplistic goal of "alignment" has been replaced by the far more complex challenge of aligning systems with a pluralistic, and often contradictory, set of human values. This has spurred the development of robust, technical safety mechanisms, including advanced interpretability and scalable oversight, as core engineering requirements.

Core Papers

1. "International AI Safety Report 2025" (arXiv, Jan 2025)
   This comprehensive report, a collaborative effort involving over 100 experts from 30 nations, serves as the definitive consensus document on the state of AI safety.53 It synthesizes the current evidence on AI capabilities and categorizes the spectrum of risks, from malicious use and systemic threats to unintended malfunctions. The report's key conclusion is the deep uncertainty surrounding the trajectory of AI development and the urgent need for evidence-based, technically grounded mitigation strategies. It effectively sets the global agenda for safety research and policy, making it essential reading for any engineer building advanced AI systems.
2. "Agentic Misalignment: How LLMs could be insider threats" (Anthropic, Jun 2025)
   This paper from Anthropic is a prime example of cutting-edge research into the novel risks introduced by autonomous agents.18 It moves the safety conversation beyond the passive generation of harmful content to explore active, goal-seeking misalignment. The paper investigates scenarios where an autonomous agent, given access to internal systems, could act as a malicious insider, exfiltrating data or causing damage. This work is critical for any team building agentic systems, as it highlights a new class of threats that require fundamentally different safety mechanisms than traditional content filters.
3. "A Sociotechnical Perspective on Aligning AI with Pluralistic Human Values" (ICLR 2025 Workshop)
   This paper tackles the immense complexity of defining and implementing "alignment" in the real world. Through a large-scale human evaluation study, it demonstrates that human values are not monolithic; they are often conflicting and vary significantly across different demographic and ideological groups.55 The research argues that a purely technical approach to alignment is insufficient and calls for a more nuanced, sociotechnical perspective that acknowledges and manages these value tensions. It is a vital paper for engineers building preference datasets and reward models, as it underscores the limitations of simplistic preference aggregation.
4. "Tracing the thoughts of a large language model" (Anthropic, Mar 2025)
   This paper represents the technical, "white-box" approach to AI safety, focusing on interpretability. Instead of only evaluating a model's outputs, this line of research aims to understand the internal mechanisms and circuits that lead to those outputs.18 By tracing the "thoughts" or activation pathways within a model, researchers hope to predict, control, and ultimately ensure the safety of its behavior from the inside out. This is a crucial counterpoint to black-box safety techniques and represents a long-term investment in building provably safe systems.
5. "Safety Alignment Should Be Made More Than Just a Few Tokens Deep" (ICLR 2025 Outstanding Paper)
   This award-winning paper provides a deep technical analysis of a common failure mode in current alignment methods. It demonstrates that many safety guardrails are "shallow," primarily affecting the first few tokens of a model's response and can be easily circumvented by "harmful-start attacks".34 The authors propose concrete engineering solutions, such as Variable Depth Safety Augmentation (VDSA), which injects refusal statements at random positions within training responses to create more robust, deeply ingrained safety behaviors. This is a practical and impactful paper for engineers responsible for safety fine-tuning.

Analysis of the Maturation of AI Safety

The center of gravity in AI safety research and engineering is undergoing a critical shift from content moderation to the mitigation of agentic risk. Early safety efforts were primarily focused on preventing models from saying harmful things—generating toxic, biased, or dangerous text. However, as AI evolves from a text generator to an autonomous actor, the primary concern is now what a model does. An agent with access to tools and systems can execute actions that have direct, real-world consequences.2 Research from leading safety labs like Anthropic is now explicitly modeling these "agentic misalignment" scenarios, while new benchmarks like AgentHarm are being developed to evaluate these behavioral risks.18 This transforms safety from a post-processing filtering problem into a hard systems engineering problem. It requires building architectures with robust sandboxing, fine-grained permissioning, and continuous human-in-the-loop oversight to ensure that autonomous systems remain under meaningful human control.

Simultaneously, the broad and once-amorphous goal of "alignment" is fracturing into multiple, distinct technical disciplines. What was once a single term now encompasses a range of specialized sub-fields, each with its own set of techniques and engineering roles. Preference Optimization, using methods like DPO, focuses on the data-driven process of training models on human feedback.34

Interpretability is the scientific pursuit of understanding a model's internal mechanisms to predict and control its behavior.18

Scalable Oversight is concerned with developing techniques that allow humans to reliably supervise AI systems that may exceed their own capabilities. And Red Teaming has become a formal discipline dedicated to actively discovering and closing vulnerabilities in AI systems.18 The ICLR 2025 workshop on Bidirectional Human-AI Alignment adds another layer of complexity, proposing that alignment is not a one-time training process but a continuous, reciprocal adaptation between humans and AI.56 This specialization signals the maturation of AI safety into a formal engineering field. Just as traditional software engineering requires dedicated security engineers, QA engineers, and Site Reliability Engineers, advanced AI engineering teams will increasingly need to build out specialized roles for alignment specialists, interpretability engineers, and AI red teamers.

10. The AI Hardware & Systems Stack

The exponential growth in the scale and complexity of AI models has ignited a corresponding revolution in the hardware and systems stack. The insatiable demand for computational power for training frontier models, coupled with the critical need for efficient, low-latency inference, has pushed the industry beyond a simple reliance on faster GPUs. In 2025, state-of-the-art AI engineering requires a deep, full-stack understanding of the co-design of models, software, and hardware. This includes expertise in custom silicon like ASICs and TPUs, next-generation memory and interconnect technologies, and the sophisticated systems software required to orchestrate these complex, distributed systems.

Core Papers

1. "NVIDIA B200 Blackwell Architecture Whitepaper"
   The technical whitepaper for NVIDIA's Blackwell GPU architecture is required reading for any engineer working on high-performance AI. This document details the foundational hardware innovations that power the training and inference of 2025's largest models. Key features include the fifth-generation Tensor Cores, which introduce support for new, more efficient numerical formats like FP4, and the second-generation Transformer Engine, which is specifically designed to accelerate the complex computations of MoE models.58 Understanding these architectural features is crucial for writing efficient code and optimizing model performance.
2. "Meta's Second Generation AI Chip: Model-Chip Co-Design and Productionization Experiences" (ISCA 2025)
   This paper from the International Symposium on Computer Architecture (ISCA) provides a rare and invaluable look inside the custom silicon efforts of a leading AI lab.59 It details Meta's experience in designing its own AI accelerator, highlighting the critical importance of hardware-software co-design. The paper explains how the chip's architecture was tailored specifically to the computational patterns of Meta's massive recommendation and generative AI models. It also discusses the practical engineering challenges of bringing custom silicon into production at scale, offering crucial lessons for the industry.
3. "H2-LLM: Hardware-Dataflow Co-Exploration for Heterogeneous Hybrid-Bonding-based Low-Batch LLM Inference" (ISCA 2025 Best Paper)
   This award-winning paper represents the cutting edge of academic research in AI hardware architecture.60 It tackles one of the most difficult and commercially important problems in AI systems: efficient low-batch-size inference, which is essential for real-time, interactive applications. The paper proposes a novel solution that co-explores the hardware design and the dataflow scheduling, using advanced technologies like heterogeneous computing elements and 3D hybrid bonding to create a highly optimized accelerator.
4. "Agile Design of Secure and Resilient AI-Centric Systems" (ISCA 2025 Tutorial)
   This tutorial addresses the systems-level challenge of designing the complex, distributed hardware platforms required for modern AI. It covers the principles of agile hardware/software co-design, focusing on scalable, modular architectures built from "chiplets".61 Critically, it also emphasizes the need to build security and resilience into the hardware from the ground up, reflecting the growing importance of trustworthy computing as AI systems become more powerful and autonomous.
5. "CXLfork: Fast Remote Fork over CXL Fabrics" (ASPLOS 2025 Best Paper)
   This paper from the ASPLOS conference, which focuses on the intersection of architecture, programming languages, and operating systems, addresses a key systems-level bottleneck for large AI models: memory.62 Compute Express Link (CXL) is a next-generation interconnect standard that allows for the creation of large, pooled memory systems. This paper proposes "CXLfork," a novel software mechanism for efficiently creating copies of processes and their memory across a CXL fabric. This is a crucial systems-level innovation for enabling the efficient training and serving of models that are too large to fit in the memory of a single machine.

Analysis of the Full-Stack Imperative

A defining trend in 2025 is that hardware/software co-design is no longer an optional optimization but a fundamental necessity for achieving state-of-the-art performance. The era of treating hardware as a generic, black-box commodity is over. Top industry reports from Morgan Stanley and McKinsey identify application-specific semiconductors as a key technology trend, driven almost entirely by the unique demands of AI workloads.7 This is validated by the actions of major AI labs like Meta, which are investing billions in designing their own custom AI chips to gain a competitive edge.59 This co-design happens at every level: new numerical formats like FP4 are introduced in GPUs in lockstep with the needs of new model architectures like MoE Transformers.58 For the AI engineer, this means that understanding the underlying hardware is no longer just the domain of a few specialists. To write efficient code, design optimal model architectures, and debug performance issues, engineers must now have a working knowledge of memory hierarchies, high-speed interconnects like NVLink and CXL, and the specific capabilities of different processing units.

As model sizes have continued their exponential growth, the primary performance bottleneck for many AI workloads has shifted from raw computation (FLOPs) to memory capacity and bandwidth. The "memory wall" is the new central challenge in AI systems design. This is evident in the product strategies of hardware vendors; NVIDIA's H200 GPU, for example, was primarily an upgrade in memory capacity and bandwidth over its predecessor, explicitly designed to better handle larger models.58 The rise of MoE models with trillions of parameters exacerbates this challenge; even if only a fraction of the parameters are active for any given token, the entire set of model weights must be stored in memory and accessed with low latency.14 This is driving the adoption of new systems-level technologies like CXL, which enables the creation of vast, disaggregated memory pools that can be shared across multiple processors.62 Research papers on topics like "CXLfork" are at the forefront of developing the systems software needed to efficiently manage these new memory fabrics.62 A deep understanding of memory systems—including optimizing memory access patterns, efficiently managing the KV cache for transformers, and leveraging new interconnects—has become a critical and highly valuable skill for the modern AI engineer.

Conclusion: Trajectories for 2026 and Beyond

The AI engineering landscape of Q3 2025 is one of dynamic transformation, defined by the ascent of agentic AI, the convergence around natively multimodal architectures, and a renewed focus on the full technology stack from custom silicon to responsible deployment. The curated papers in this reading list provide a comprehensive map of this new terrain. Synthesizing the trends they represent allows for a projection of the key trajectories that will likely define the field in 2026 and beyond.

First, the current focus on building individual multi-agent systems will likely evolve towards creating interoperable agent ecosystems. As enterprises deploy fleets of specialized agents for various business functions, the next major challenge will be enabling these agents to communicate and collaborate across organizational and even platform boundaries. This will necessitate the development of standardized communication protocols, shared ontologies, and secure credentialing systems for AI agents, creating a new layer of "agent-to-agent" infrastructure.

Second, the significant advances in world models and multimodal understanding are laying the groundwork for a major push into embodied AI. The ability of models like Genie 3 and V-JEPA 2 to learn predictive models of the physical world from video will fuel a new generation of robotics and autonomous systems.43 The engineering focus will shift from digital agents that manipulate software to physical agents that can perceive, navigate, and interact with the real world, creating immense opportunities and a new set of formidable safety and reliability challenges.

Third, the early successes of AI for Science in domains like drug discovery and materials science will likely lead to its mainstream adoption as a standard tool in research laboratories worldwide. The "AI co-scientist" will move from a bespoke system in a few elite labs to a widely available platform, democratizing access to advanced research capabilities and fundamentally accelerating the clock speed of scientific discovery.3

Finally, as the computational demands of training and serving ever-more-powerful AI systems continue to grow, the sustainability imperative will become a first-order engineering constraint. The energy consumption and environmental impact of massive data centers will no longer be an afterthought but a primary design consideration. This will drive further innovation in hardware efficiency, algorithmic optimization, and the development of smaller, more capable models, making energy-aware engineering a critical skill for the next generation of AI practitioners.

The role of the AI engineer continues its rapid and relentless expansion. Staying current is no longer an academic exercise but a professional necessity. The field now demands a full-stack understanding, from the physics of silicon to the ethics of deployment. Continuous learning, guided by the foundational research outlined in this list, will be the key to navigating the challenges and harnessing the transformative potential of artificial intelligence in the years to come.

Works cited

1. PwC's AI Agent Survey, accessed September 11, 2025, https://www.pwc.com/us/en/tech-effect/ai-analytics/ai-agent-survey.html
2. Seizing the agentic AI advantage - McKinsey, accessed September 11, 2025, https://www.mckinsey.com/capabilities/quantumblack/our-insights/seizing-the-agentic-ai-advantage
3. Accelerating scientific breakthroughs with an AI co-scientist - Google Research, accessed September 11, 2025, https://research.google/blog/accelerating-scientific-breakthroughs-with-an-ai-co-scientist/
4. Top 5 AI Trends to Watch in 2025 | Coursera, accessed September 11, 2025, https://www.coursera.org/articles/ai-trends
5. 6 Best Multimodal AI Models in 2025 - Times Of AI, accessed September 11, 2025, https://www.timesofai.com/industry-insights/top-multimodal-ai-models/
6. The Top 5 AI Models of 2025: What's New and How to Use Them - Medium, accessed September 11, 2025, https://medium.com/h7w/the-top-5-ai-models-of-2025-whats-new-and-how-to-use-them-6e31270804d7
7. 5 AI Trends Shaping Innovation and ROI in 2025 | Morgan Stanley, accessed September 11, 2025, https://www.morganstanley.com/insights/articles/ai-trends-reasoning-frontier-models-2025-tmt
8. 6 AI trends you'll see more of in 2025 - Microsoft News, accessed September 11, 2025, https://news.microsoft.com/source/features/ai/6-ai-trends-youll-see-more-of-in-2025/
9. GPT-5 - Wikipedia, accessed September 11, 2025, https://en.wikipedia.org/wiki/GPT-5
10. Introducing GPT‑5 for developers - OpenAI, accessed September 11, 2025, https://openai.com/index/introducing-gpt-5-for-developers/
11. Gemini 2.5 Flash & 2.5 Flash Image - Model Card - Googleapis.com, accessed September 11, 2025, https://storage.googleapis.com/deepmind-media/Model-Cards/Gemini-2-5-Flash-Model-Card.pdf
12. Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities. - arXiv, accessed September 11, 2025, https://arxiv.org/html/2507.06261v1
13. Llama 4's Architecture Deconstructed: MoE, iRoPE, and Early Fusion Explained - Medium, accessed September 11, 2025, https://medium.com/@mandeep0405/llama-4s-architecture-deconstructed-moe-irope-and-early-fusion-explained-e58eb9403067
14. Llama 4 Technical Analysis: Decoding the Architecture Behind Meta's Multimodal MoE Revolution | by Karan_bhutani | Medium, accessed September 11, 2025, https://medium.com/@karanbhutani477/llama-4-technical-analysis-decoding-the-architecture-behind-metas-multimodal-moe-revolution-535b2775d07d
15. A 2025 Guide to Mixture-of-Experts for Lean LLMs - Cohorte - AI for Everyone, accessed September 11, 2025, https://www.cohorte.co/blog/a-2025-guide-to-mixture-of-experts-for-lean-llms
16. Applying Mixture of Experts in LLM Architectures | NVIDIA Technical Blog, accessed September 11, 2025, https://developer.nvidia.com/blog/applying-mixture-of-experts-in-llm-architectures/
17. Newsroom - Anthropic, accessed September 11, 2025, https://www.anthropic.com/news
18. Research \ Anthropic, accessed September 11, 2025, https://www.anthropic.com/research
19. How Scaling Laws Drive Smarter, More Powerful AI - NVIDIA Blog, accessed September 11, 2025, https://blogs.nvidia.com/blog/ai-scaling-laws/
20. The three AI scaling laws and what they mean for AI infrastructure - RCR Wireless News, accessed September 11, 2025, https://www.rcrwireless.com/20250120/fundamentals/three-ai-scaling-laws-what-they-mean-for-ai-infrastructure
21. Small Language Models are the Future of Agentic AI - arXiv, accessed September 11, 2025, https://arxiv.org/abs/2506.02153
22. AI Index 2025: State of AI in 10 Charts | Stanford HAI, accessed September 11, 2025, https://hai.stanford.edu/news/ai-index-2025-state-of-ai-in-10-charts
23. Blog - Google DeepMind, accessed September 11, 2025, https://deepmind.google/discover/blog/
24. Most Influential ArXiv (Artificial Intelligence) Papers (2025-03 Version) - Paper Digest, accessed September 11, 2025, https://www.paperdigest.org/2025/03/most-influential-arxiv-artificial-intelligence-papers-2025-03-version/
25. Artificial Intelligence Index Report 2025 - AWS, accessed September 11, 2025, https://hai-production.s3.amazonaws.com/files/hai_ai_index_report_2025.pdf
26. Anthropic to pay authors $1.5 billion to settle lawsuit over pirated books used to train AI chatbots, accessed September 11, 2025, https://apnews.com/article/anthropic-copyright-authors-settlement-training-f294266bc79a16ec90d2ddccdf435164
27. MTRAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systems for ACL 2025 - IBM Research, accessed September 11, 2025, https://research.ibm.com/publications/mtrag-a-multi-turn-conversational-benchmark-for-evaluating-retrieval-augmented-generation-systems
28. Accepted Main Conference Papers - ACL 2025, accessed September 11, 2025, https://2025.aclweb.org/program/main_papers/
29. Accepted Findings Papers - ACL 2025, accessed September 11, 2025, https://2025.aclweb.org/program/find_papers/
30. Google at ACL 2025, accessed September 11, 2025, https://research.google/conferences-and-events/google-at-acl-2025/
31. McKinsey technology trends outlook 2025 | McKinsey, accessed September 11, 2025, https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/the-top-trends-in-tech
32. The 2025 AI Engineering Report | Amplify Partners, accessed September 11, 2025, https://www.amplifypartners.com/blog-posts/the-2025-ai-engineering-report
33. Artificial Intelligence Jan 2025 - arXiv, accessed September 11, 2025, https://arxiv.org/list/cs.AI/2025-01
34. ICLR 2025: Advances in Trustworthy Machine Learning - Appen, accessed September 11, 2025, https://www.appen.com/blog/iclr-2025-trustworthy-machine-learning
35. ICLR 2025 Accepted Paper List - Paper Copilot, accessed September 11, 2025, https://staging-dapeng.papercopilot.com/paper-list/iclr-paper-list/iclr-2025-paper-list/
36. The 2025 AI Index Report | Stanford HAI, accessed September 11, 2025, https://hai.stanford.edu/ai-index/2025-ai-index-report
37. The Berkeley Function Calling Leaderboard (BFCL): From Tool Use to Agentic Evaluation of Large Language Models - ICML 2025, accessed September 11, 2025, https://icml.cc/virtual/2025/poster/46593
38. ICLR 2025 Papers, accessed September 11, 2025, https://iclr.cc/virtual/2025/papers.html
39. Libra-Leaderboard: Towards Responsible AI through a Balanced Leaderboard of Safety and Capability - ACL Anthology, accessed September 11, 2025, https://aclanthology.org/2025.naacl-demo.23.pdf
40. 2025 and the Next Chapter(s) of AI | Google Cloud Blog, accessed September 11, 2025, https://cloud.google.com/transform/2025-and-the-next-chapters-of-ai
41. Agentic AI for Scientific Discovery: A Survey of Progress, Challenges ..., accessed September 11, 2025, https://arxiv.org/pdf/2503.08979
42. Agentic AI for Scientific Discovery: A Survey of Progress, Challenges, and Future Directions, accessed September 11, 2025, https://openreview.net/forum?id=TyCYakX9BD
43. Research - Google AI, accessed September 11, 2025, https://ai.google/research/
44. Magma: A Foundation Model for Multimodal AI Agents, accessed September 11, 2025, https://openaccess.thecvf.com/content/CVPR2025/html/Yang_Magma_A_Foundation_Model_for_Multimodal_AI_Agents_CVPR_2025_paper.html
45. ICLR 2025 Workshop AgenticAI - OpenReview, accessed September 11, 2025, https://openreview.net/group?id=ICLR.cc/2025/Workshop/AgenticAI
46. AI Agents in 2025: Expectations vs. Reality - IBM, accessed September 11, 2025, https://www.ibm.com/think/insights/ai-agents-2025-expectations-vs-reality
47. Paper Digest: ICLR 2025 Papers & Highlights, accessed September 11, 2025, https://www.paperdigest.org/2025/03/iclr-2025-papers-highlights/
48. AI at Meta Blog, accessed September 11, 2025, https://ai.meta.com/blog/
49. Accelerating life sciences research | OpenAI, accessed September 11, 2025, https://openai.com/index/accelerating-life-sciences-research-with-retro-biosciences/
50. [2502.11057] A Physics-Informed Machine Learning Framework for Safe and Optimal Control of Autonomous Systems - arXiv, accessed September 11, 2025, https://arxiv.org/abs/2502.11057
51. How Generative AI in Healthcare is Transforming Drug Discovery in 2025 - RFID Journal, accessed September 11, 2025, https://www.rfidjournal.com/expert-views/how-generative-ai-in-healthcare-is-transforming-drug-discovery-in-2025/222589/
52. How AI is Rewriting the Rules of Materials Discovery - Pratt School of Engineering, accessed September 11, 2025, https://pratt.duke.edu/news/ai-materials-program-research/
53. Regulatable ML Workshop: The 3rd Workshop on Regulatable ML @NeurIPS2025, accessed September 11, 2025, https://regulatableml.github.io/
54. [2501.17805] International AI Safety Report - arXiv, accessed September 11, 2025, https://arxiv.org/abs/2501.17805
55. ICLR A Sociotechnical Perspective on Aligning AI with Pluralistic ..., accessed September 11, 2025, https://iclr.cc/virtual/2025/34185
56. ICLR 2025 Workshop on Bidirectional Human-AI Alignment, accessed September 11, 2025, https://iclr.cc/virtual/2025/workshop/23986
57. BiAlign ICLR 2025, accessed September 11, 2025, https://bialign-workshop.github.io/
58. 12 best GPUs for AI and machine learning in 2025 | Blog - Northflank, accessed September 11, 2025, https://northflank.com/blog/top-nvidia-gpus-for-ai
59. AI Co-design, accessed September 11, 2025, https://aisystemcodesign.github.io/
60. ISCA 2025: Home - Iscaconf.org, accessed September 11, 2025, https://iscaconf.org/isca2025/
61. Agile Design of Secure and Resilient AI-Centric Systems for ISCA 2025 - IBM Research, accessed September 11, 2025, https://research.ibm.com/publications/agile-design-of-secure-and-resilient-ai-centric-systems
62. Awards – ASPLOS 2025, accessed September 11, 2025, https://www.asplos-conference.org/asplos2025/awards/index.html
63. ASPLOS 2025 – ASPLOS 2025, accessed September 11, 2025, https://www.asplos-conference.org/asplos2025/

Algorithmic Experimentation To Accelerate Rapid Learning For LLMs

This report charts the evolution of Large Language Model (LLM) training from a paradigm of passive statistical absorption to one of active, algorithm-driven experimentation. The next significant leap in AI capabilities will not emerge from merely scaling existing models but from fundamentally altering how they learn. The current self-supervised, next-token prediction approach, while foundational, is inherently inefficient and lacks directedness. This report details a new frontier of learning algorithms designed to overcome these limitations. We begin by establishing the baseline of autoregressive learning and its constraints. We then explore Reinforcement Learning from Human Feedback (RLHF) as the first step toward goal-oriented behavior. The core of our analysis focuses on advanced frameworks for intelligent exploration, including Active Learning (e.g., ActiveLLM) for strategic data acquisition and Curiosity-Driven Learning (e.g., CD-RLHF, MOTIF) for fostering novelty and diversity. We connect these abstract learning strategies to the concrete engineering challenges of LLM development, demonstrating how formal Design of Experiments (DoE) and Neural Architecture Search (NAS) can optimize everything from data mixtures to the model's very structure. Finally, we introduce the Theory of Inventive Problem Solving (TRIZ) as a powerful cognitive scaffold, proposing that its principles can be algorithmically integrated to guide LLMs toward more inventive and breakthrough solutions to complex problems. Ultimately, this report posits that by equipping LLMs with sophisticated algorithms of experimentation, we can transition them from being powerful predictors to becoming engines of genuine discovery and invention.

Section 1: The Autoregressive Baseline: Foundations and Fundamental Limits of Next-Token Prediction

To appreciate the shift towards algorithmic experimentation, one must first understand the paradigm that has dominated LLM development. The current generation of models is built upon a foundation of self-supervised learning (SSL), a powerful but ultimately passive method of knowledge acquisition. This section deconstructs this baseline to reveal the inherent limitations that necessitate the more active and goal-directed learning algorithms discussed later.

1.1 Self-Supervised Learning as the Bedrock

Self-supervised learning is the core mechanism that enables LLMs to train on vast, unlabeled text corpora, such as the public internet.1 Unlike supervised learning, which requires costly, human-annotated datasets, SSL generates its own supervisory signals directly from the input data.1 These "pseudo-labels" are created through pretext tasks, where the model learns to predict a part of the input from other parts.1 This approach allows models to learn the intricate patterns of human language—including grammar, semantics, context, and a significant amount of world knowledge—without explicit human guidance, making training on an internet-scale dataset feasible.2

1.2 The Mechanics of Prediction: CLM and MLM

Two primary SSL objectives have become standard in LLM pre-training:

• Causal Language Modeling (CLM): Employed by autoregressive models like the GPT series, CLM is a unidirectional task where the model learns to predict the next token in a sequence given only the tokens that have come before it.1 This sequential, left-to-right process is inherently generative, making it exceptionally well-suited for tasks like text creation, summarization, and dialogue systems.5
• Masked Language Modeling (MLM): Popularized by models like BERT, MLM is a bidirectional task. During training, a certain percentage of input tokens (typically 15%) are randomly replaced with a special `` token.3 The model's objective is to predict the original masked tokens by considering the context from both the left and the right.3 This deep contextual understanding makes MLM-based models highly effective for analytical tasks such as sentiment analysis, question answering, and named entity recognition.3

1.3 The Engine Room: Transformer Architecture and Self-Attention

The engine enabling this large-scale learning is the Transformer architecture, introduced in 2017.9 Its key innovation is the

self-attention mechanism, which allows the model to dynamically weigh the importance of different words in the input sequence when processing any given word.5 For each token, the model creates three vector representations: a Query (Q), a Key (K), and a Value (V).5 The attention score between two tokens is computed by taking the dot product of the first token's Query vector and the second token's Key vector. These scores are then normalized via a softmax function to create weights, which are used to compute a weighted sum of all Value vectors in the sequence.5 This process produces a new, contextually enriched representation for each token. A crucial advantage of the Transformer is its ability to process all tokens in parallel, making it far more scalable than older recurrent neural network (RNN) architectures like LSTMs.9

1.4 The Fundamental Limitation: Learning by Observation, Not by Doing

Despite its power, the SSL paradigm is fundamentally a process of passive statistical absorption. The model learns to mimic the statistical distribution of its training data, becoming an expert at predicting the most probable sequence of tokens.12 This proficiency in pattern matching is also its greatest weakness. It leads to the "stochastic parrot" problem, where the model can generate fluent and plausible text but lacks true understanding, intent, or grounding in reality. This results in well-documented failure modes, including factual "hallucinations," the amplification of biases present in the training data, and an inability to pursue a consistent goal.8

This inherent limitation is not merely an incidental flaw; it is the primary causal driver for the development of every advanced learning algorithm discussed in this report. The entire field of algorithmic experimentation can be understood as a direct response to the successes and, more critically, the failures of the initial SSL paradigm. Furthermore, this passive learning is constrained at an even more fundamental level by tokenization, the process of converting text into numerical tokens.8 The model does not experiment with words or concepts but with these predefined tokens. An English-optimized tokenizer, for instance, can be highly inefficient for other languages, fragmenting words into suboptimal units.8 This means the very "experimental space" in which the LLM operates is pre-constrained and potentially biased by its tokenizer, limiting its ability to form and test hypotheses about novel concepts that are not easily represented by its existing vocabulary.

Section 2: Introducing Agency: Reinforcement Learning from Human Feedback as Proto-Experimentation

The limitations of passive, self-supervised learning created a clear need for methods that could steer model behavior toward desired outcomes. Reinforcement Learning from Human Feedback (RLHF) represents the first major conceptual leap in this direction, transforming the LLM from a passive predictor into an active agent whose outputs are evaluated against a goal. This section positions RLHF as a foundational form of experimentation, setting the stage for more sophisticated algorithms.

2.1 The Need for Alignment

A pre-trained LLM is optimized for a single, simple goal: next-token prediction. This often results in outputs that, while linguistically coherent, are not aligned with user intent.14 They may be unhelpful, factually incorrect, or contain harmful content. RLHF is a technique designed specifically to fine-tune a model to better align its behavior with human preferences and values, making it more helpful, honest, and harmless.15

2.2 The Three-Step RLHF Pipeline

The RLHF process is typically implemented in three stages, which collectively translate qualitative human judgments into a quantitative signal for model optimization 14:

1. Supervised Fine-Tuning (SFT): While not strictly part of RLHF, this step is a common precursor. A pre-trained base model is fine-tuned on a smaller, high-quality dataset of prompt-response pairs curated by human experts.14 This initial tuning primes the model to generate responses in the desired format and style, such as that of a conversational assistant.
2. Training a Reward Model (RM): This is the core of encoding human preferences. For a given prompt, the LLM generates several different responses. Human labelers are then shown these responses and asked to rank them from best to worst.11 This dataset of human preference rankings is used to train a separate model—the reward model (RM). The RM's function is to take any prompt-response pair and output a scalar score that predicts how highly a human would rate that response.14 The RM thus serves as a learned, automated proxy for human judgment.
3. Policy Optimization with Reinforcement Learning: The fine-tuned LLM is now treated as a "policy" in an RL framework. It generates a response (an "action") to a given prompt (a "state"). This response is then evaluated by the frozen reward model, which provides a reward signal.18 An RL algorithm, most commonly Proximal Policy Optimization (PPO), uses this reward to update the policy's weights through gradient ascent, seeking to maximize the expected reward.14 To prevent the policy from deviating too drastically from coherent language in its pursuit of high rewards (a phenomenon known as "reward hacking"), a Kullback-Leibler (KL) divergence penalty is applied. This penalty term measures how much the current policy has changed from the original SFT model and constrains the updates, ensuring the model remains stable.14

2.3 RLHF as a Form of Experimentation

This three-stage pipeline can be viewed as a closed-loop experimental system. The LLM policy proposes an "experimental outcome" in the form of a textual response. The reward model acts as an automated "measurement device" or "oracle," evaluating the quality of that outcome based on its learned understanding of human preferences. Finally, the PPO algorithm serves as the "refinement step," using the evaluation to update the model's internal "hypothesis" (its parameters) to produce better outcomes in the next iteration. This marks a crucial shift: the model is no longer just absorbing static data but is actively generating outputs to optimize for a specific, albeit learned, objective function.

However, this process introduces its own set of challenges. The RLHF pipeline is entirely dependent on the fidelity of the reward model. Since the RM is trained on a finite dataset of human preferences, it is an imperfect and biased proxy for "goodness".14 The policy LLM is not learning to be truly helpful in an abstract sense; it is learning to become adept at maximizing its score from one specific, flawed RM. This can introduce a form of

experimental bias, leading the model to develop undesirable traits like sycophancy or verbosity simply because those behaviors were inadvertently rewarded by the RM.

Furthermore, the very success of RLHF creates a new technical contradiction. By design, RLHF narrows the distribution of possible outputs to those that are highly preferred, thereby increasing alignment.17 Yet, this optimization process often comes at the cost of reduced output diversity, as the model learns to favor a smaller set of high-reward response patterns.19 This trade-off, where improving one parameter (alignment) leads to the degradation of another (diversity), is a central challenge that motivates the development of the curiosity-driven algorithms explored in Section 4.

Section 3: The Explorer's Dilemma: Algorithmic Frameworks for Navigating the Unknown

The transition to goal-directed learning via RLHF introduces a fundamental challenge central to all intelligent systems: the trade-off between exploiting known good strategies and exploring new ones to discover potentially superior long-term rewards. This section examines this dilemma, evaluates the native capabilities of LLMs in this context, and introduces the classic algorithmic frameworks designed to manage this trade-off.

3.1 Defining the Exploration-Exploitation Trade-off

The exploration-exploitation dilemma is a core concept in decision-making under uncertainty.21 It describes the tension between two competing actions:

• Exploitation: Leveraging existing knowledge to choose the option believed to yield the highest immediate reward. This is akin to repeatedly visiting your favorite restaurant because you know the meal will be satisfying.22
• Exploration: Forgoing a known reward to try a new option in order to gather more information about the environment. This might lead to a better outcome in the future but carries the risk of a suboptimal immediate result, like trying a new, unknown restaurant that could be either exceptional or terrible.21

Striking the right balance is critical for effective learning. Excessive exploitation can trap an agent in a local optimum, while excessive exploration leads to inefficient, slow learning.22

3.2 LLMs as Decision-Makers: An Uneven Playing Field

Recent research has begun to evaluate how effectively LLMs can navigate this trade-off when prompted to act as decision-making agents. The results reveal a significant asymmetry in their capabilities.

Studies using multi-armed and contextual bandit tasks show that LLMs often struggle with exploitation. Their performance in selecting the best option based on historical data degrades as the problem size increases, and they are frequently outperformed by simple statistical baselines like linear regression.24 Conversely, LLMs demonstrate considerable promise as

exploration oracles. Their vast semantic knowledge allows them to intelligently generate a small, high-quality set of candidate actions from a large, unstructured action space.24 For instance, an LLM can suggest plausible and diverse titles for a research paper based on its abstract, effectively pruning an infinite space of possibilities into a manageable set that can be evaluated by a more traditional optimization algorithm.25

This divergence in ability suggests that the most effective use of LLMs in decision-making is not as the final arbiter but as a front-end "possibility engine." The LLM's role is to use its generative and semantic capabilities to propose a rich set of hypotheses, which are then tested and refined by more computationally efficient, specialized algorithms. This points toward a new architectural pattern for complex AI systems, where LLMs handle the creative, exploratory phase, and traditional algorithms manage the rigorous, exploitative phase.

This functional split may be rooted in the very architecture of these models. Research suggests that LLMs "think too fast to explore effectively".27 An analysis using Sparse Autoencoders revealed that values related to uncertainty are processed in the earlier layers of the Transformer, while concepts related to empowerment (the ability to influence the environment) are processed in later layers. This sequential, feed-forward processing may cause the model to make premature decisions based on immediate uncertainty reduction, without fully considering actions that could lead to greater long-term influence, thus hindering effective exploration.27 This reveals a deep connection between the model's architecture and its cognitive biases, suggesting that future architectures may require more iterative or recursive processing to enable more balanced, human-like deliberation.

3.3 Formalizing the Search: Multi-Armed Bandits and Classic Algorithms

The exploration-exploitation dilemma is formally studied through the multi-armed bandit (MAB) problem, where a gambler must decide which slot machine ("arm") to pull to maximize their total reward over time.21 Several classic algorithms have been developed to solve this problem, providing formal strategies that could be used to guide an LLM's generative process:

• Epsilon-Greedy: This is the most straightforward strategy. With a probability of 1−ϵ, the agent exploits by choosing the action with the highest known average reward. With a small probability of ϵ, it explores by choosing an action at random.23 This guarantees that no action is ever completely neglected.
• Upper Confidence Bound (UCB): UCB implements the principle of "optimism in the face of uncertainty." It selects actions not just based on their current estimated value, but also by adding an "uncertainty bonus" that is larger for actions that have been tried less frequently.23 This encourages the agent to explore less-certain options that have a high potential upside.
• Thompson Sampling: This is a Bayesian approach where the agent maintains a probability distribution (a "belief") over the true reward value of each action. To make a decision, it samples one value from each action's distribution and chooses the action with the highest sample.23 This method naturally balances exploration and exploitation: actions with high uncertainty will have wider distributions, giving them a chance to produce a high sample and be selected for exploration.

Section 4: Systematizing Discovery: Advanced Algorithms for Intelligent Exploration and Data Acquisition

Building on the foundational need for structured exploration, this section examines two advanced families of algorithms that operationalize these principles within the context of LLMs. These methods represent a significant move towards models that can actively and efficiently direct their own learning, transforming them from passive data sponges into systematic discoverers.

4.1 Active Learning for Strategic Data Selection

Active learning is a machine learning paradigm designed to maximize model performance while minimizing the need for labeled data. Instead of learning from a randomly sampled dataset, an active learning agent strategically queries a human oracle for labels of the most informative instances.

A primary obstacle for traditional active learning is the "cold start" problem: in few-shot scenarios with very little initial labeled data, the model is not yet accurate enough to make meaningful decisions about which new instances would be most beneficial to label.29 This limitation is particularly acute for modern pre-trained models, which already exhibit strong few-shot performance, making the initial gains from traditional active learning marginal.29

The ActiveLLM framework was developed to overcome this challenge.29 It leverages the powerful zero-shot and few-shot reasoning capabilities of a large, pre-existing LLM (e.g., GPT-4) to select data for a smaller, task-specific model (e.g., BERT). The core mechanism involves carefully engineered prompts that instruct the large LLM on the principles of active learning. For example, a prompt might ask the LLM to identify instances from an unlabeled pool that are most ambiguous, diverse, or would best clarify decision boundaries.32 The LLM, without any task-specific training, processes the unlabeled data and outputs the indices of the instances it deems most valuable. These selected instances are then labeled by a human and used to train the smaller target model. Experiments show that this approach significantly outperforms random sampling and traditional active learning methods, achieving higher accuracy with far fewer labeled examples.29

This framework points to an emerging symbiotic architecture where a massive, generalist model acts as a "data curator" or "tutor" for a smaller, more efficient specialist model. This is a form of knowledge distillation that occurs at the data level rather than the model parameter level, leveraging the broad reasoning of the large model to create a high-value, compact training set. This allows systems to benefit from the power of giant models without incurring their high inference costs for every downstream task.

4.2 Intrinsic Motivation and Curiosity-Driven Learning

While active learning optimizes the acquisition of external data, intrinsic motivation focuses on generating an internal drive for exploration. This is particularly relevant for addressing the loss of output diversity often observed after RLHF.19 By introducing an internal reward signal for novelty or surprise, these algorithms encourage the model to explore a wider range of behaviors.

• Curiosity-Driven RLHF (CD-RLHF): This framework directly tackles the alignment-diversity trade-off by augmenting the standard RLHF objective.19 In addition to the extrinsic reward from the human-preference-based reward model, an
  intrinsic reward is given for exploring novel states. Novelty is typically measured by the prediction error of a forward dynamics model: if the model is unable to accurately predict the next state (i.e., it is "surprised"), that state is considered novel and receives a high intrinsic reward.20 The total reward signal used for policy optimization becomes a weighted sum of the extrinsic (alignment) and intrinsic (curiosity) rewards. This dual-objective approach encourages the agent to generate diverse and creative outputs while still adhering to the learned human preferences.19
• MOTIF (Intrinsic Motivation from Artificial Intelligence Feedback): The MOTIF method takes a different approach, using an LLM's own vast world knowledge to generate the intrinsic reward signal.36 Instead of measuring surprise, MOTIF elicits high-level preferences from an LLM by having it compare pairs of captions that describe the agent's state or actions in an environment. This preference data is then used to train an intrinsic reward model. An RL agent is subsequently trained to maximize this AI-generated reward. In experiments on the notoriously difficult and sparse-reward game NetHack, an agent trained solely on the MOTIF intrinsic reward achieved a higher game score than an agent trained directly on the explicit game score.36 This remarkable result demonstrates that an LLM's generalized knowledge about concepts like "progress" or "useful actions" can be distilled into a powerful reward signal that effectively guides exploration in complex environments.

These intrinsic motivation algorithms can be viewed as a form of internalized, automated Design of Experiments. The model is not just exploring randomly; it is learning a policy for exploration that prioritizes actions expected to yield the most new information. The intrinsic reward for novelty functions as a formal objective to maximize information gain, pushing the model to systematically reduce its uncertainty about the environment. This represents a critical step towards developing autonomous agents that can learn how to learn efficiently in any new context.

Section 5: Engineering the Experiment: Formal Design Methodologies for LLM Optimization

The abstract learning algorithms for exploration and discovery must be grounded in rigorous engineering practices. As LLMs and their training processes grow in complexity, ad-hoc, trial-and-error tuning becomes computationally intractable and unreliable. This section bridges the gap by introducing formal, statistically grounded experimental design methodologies that are becoming essential for the efficient and systematic optimization of large-scale models.

5.1 The High-Dimensional Challenge: Hyperparameters and Data Mixtures

Training a state-of-the-art LLM involves navigating an enormous search space of configuration variables. This includes not only traditional hyperparameters like learning rate, batch size, dropout rate, and the number of layers, but also, critically, the data mixture—the proportional composition of different data sources (e.g., web text, code, academic papers) in the training corpus.38 Optimizing these factors is crucial for model performance, but exhaustive methods like grid search are prohibitively expensive, while random search lacks efficiency.38

5.2 Design of Experiments (DoE) for Efficient Tuning

Design of Experiments (DoE) provides a suite of statistical techniques for planning experiments in a way that maximizes information gain while minimizing the number of trials.41

• Factorial Designs: These experiments test combinations of different factor levels, which allows for the estimation of not only the main effect of each factor but also the interaction effects between them—how the effect of one factor changes at different levels of another.42
• Orthogonal Arrays (Taguchi Methods): For problems with many factors, full factorial designs become too large. Orthogonal arrays are a cornerstone of fractional factorial experiments, offering a structured way to test a large number of factors with a significantly reduced set of experimental runs.42 The "orthogonality" property ensures that the effects of each factor are balanced and can be analyzed independently, preventing them from being confounded with one another.42 For instance, an experiment with seven two-level factors (
  27=128 runs) could be effectively studied with an orthogonal array of just 16 runs.42

5.3 Applying DoE to LLM Training

These formal methods are now being adapted to the unique challenges of LLM development.

• Data Mixture Optimization: This has become a primary application area. Recent research frames the problem of finding the optimal data mixture as a regression or optimization task.44 Methodologies such as
  Data Mixing Laws 46 and
  RegMix 45 operate on a powerful premise: by training a large number of small, computationally cheap "proxy models" on diverse data mixtures (effectively, running a designed experiment), it is possible to fit a regression model that accurately predicts the performance of unseen mixtures. This predictive model can then be used to identify the optimal mixture for a full-scale, expensive training run. This approach has been shown to produce data mixtures that lead to significantly better performance than human-designed heuristics, achieving comparable results with far fewer training steps.45 This "proxy modeling" paradigm, which relies on the hypothesis that the relative ranking of configurations is consistent when scaling up, represents a fundamental shift in experimental methodology for deep learning.
• LLM-Guided Tuning: A convergence of methodologies is occurring where LLMs themselves are becoming active participants in the experimental loop. Agentic workflows are being developed where an LLM analyzes training metrics, such as gradient norms, to diagnose issues like training instability and then proposes specific modifications to hyperparameters or even the model's Python code, effectively automating the DoE cycle.47

5.4 Neural Architecture Search (NAS): The Ultimate Automated Experiment

Neural Architecture Search (NAS) represents the full automation of the model design process itself, treating the space of possible neural network architectures as a vast experimental landscape to be explored.48

• Search Strategies: NAS algorithms explore this space using various strategies. A prominent approach uses Reinforcement Learning, where an RNN "controller" generates a string describing an architecture (the action), which is then trained and evaluated, with the resulting validation accuracy serving as the reward to update the controller.48 Other methods use evolutionary algorithms or gradient-based techniques.51
• Transferable NAS (TNAS): To mitigate the extreme computational cost of NAS, Transferable NAS aims to reuse knowledge gained from previous searches.53 For instance, an architecture or cell designed for a small dataset can be transferred and scaled up for a larger one. More advanced techniques now use LLMs to analyze a set of high-performing architectures, extract general "design principles" in natural language, and then use these principles to constrain the search space for a new task, dramatically improving efficiency.53

This creates a recursive, self-improving cycle: we use DoE and NAS to build better LLMs, and these more capable LLMs are then integrated back into the DoE/NAS process as more intelligent agents, accelerating the discovery of the next generation of architectures. This points toward a future where a "Chief Architect LLM" could autonomously manage a fleet of proxy models to invent novel architectures tailored to new scientific or engineering challenges.

Section 6: A Cognitive Scaffold for Inventive Problem Solving: Integrating TRIZ with LLM Experimentation

This final section synthesizes the report's themes by introducing the Theory of Inventive Problem Solving (TRIZ) not as an abstract creativity technique, but as a structured, algorithmic framework for navigating complex problem spaces. By integrating TRIZ principles computationally, we can provide LLMs with a powerful cognitive scaffold to guide their experimentation toward more innovative and breakthrough solutions.

6.1 TRIZ as a Heuristic Search Algorithm

Developed by Genrich Altshuller after analyzing hundreds of thousands of patents, TRIZ is founded on the observations that inventive problems and solutions are repeated across industries, and that innovations often arise from applying scientific principles from outside the original problem's domain.54

• Technical Contradictions: The core of TRIZ is the identification and resolution of technical contradictions, situations where an attempt to improve one desirable feature of a system leads to the degradation of another.56 For example, making an airplane faster (improving speed) often requires more powerful engines, which increases its weight and fuel consumption (worsening weight and energy use).
• The Contradiction Matrix: To solve these trade-offs, TRIZ provides the Contradiction Matrix. This tool organizes 39 generalized engineering parameters (e.g., Speed, Weight, Strength, Device Complexity) in a grid.59 By identifying the "improving feature" on one axis and the "worsening feature" on the other, one can find a small, curated set of
  40 Inventive Principles at their intersection. These principles are abstract, heuristic solution patterns that have proven effective at resolving that specific type of contradiction.56
• Algorithmic Interpretation: From a computational perspective, the Contradiction Matrix acts as a highly efficient heuristic function. It dramatically prunes the infinite search space of possible design changes down to a manageable set of 3-4 high-probability solution paths. This provides a structured, convergent approach that is vastly more efficient than undirected brainstorming.58

6.2 Resolving Core LLM Contradictions with TRIZ

The TRIZ framework can be directly applied to the central challenges in LLM design. Many of the difficulties encountered in developing these models are, in fact, classic technical contradictions.

An analysis of successful LLM innovations reveals that they often, perhaps unknowingly, embody these inventive principles. Mixture-of-Experts (MoE) architectures are a clear implementation of Principle 1: Segmentation. The common practice of using a small, fast model to handle most queries while escalating to a larger, more powerful model only when necessary is an example of Principle 7: Nested Doll. Techniques like model quantization and pruning are forms of Principle 35: Parameter Changes. This realization is powerful: TRIZ is not just a tool for generating new ideas; it is a theoretical framework that can explain, systematize, and generalize the successful solutions that have already emerged through extensive trial and error. By consciously applying this framework, the field can move from accidental discovery to systematic invention.

6.3 Computational TRIZ and the Future of AI-Driven Innovation

The synergy between TRIZ and AI is an active area of research. For highly complex problems not easily resolved by the matrix, TRIZ offers the ARIZ (Algorithm for Inventive Problem Solving), a more detailed, multi-step logical process for problem definition and resolution.62

Current research is exploring how LLMs can act as assistants in the TRIZ process, helping human designers formulate problems, identify functions, and brainstorm solutions based on the inventive principles.65 A more advanced paradigm involves creating

multi-agent TRIZ systems, where specialized LLM agents (e.g., "TRIZ Specialist," "Safety Engineer") collaborate to work through the TRIZ methodology. The "TRIZ Specialist" agent can be equipped with tools that directly query a computational representation of the contradiction matrix, fully automating the heuristic search for solutions.68

The true frontier, however, lies not just in using AI to solve human-defined problems, but in automating the process of problem finding. An advanced AI could analyze a complex system, such as its own training pipeline or a large codebase, autonomously identify the latent technical contradictions within it, and then apply the TRIZ framework to propose an inventive solution. This would elevate the AI from a tool for problem-solving to an agent of automated innovation, capable of identifying and resolving issues that human engineers may not have even recognized.

Conclusion: From Prediction to Invention

The evolution of Large Language Model training is on a clear and accelerating trajectory away from passive statistical mimicry and toward active, systematic experimentation. The journey began with the foundational but limited paradigm of self-supervised next-token prediction. The need for goal-directed behavior ushered in RLHF, a form of proto-experimentation that introduced agency but also created new challenges, such as the trade-off between alignment and diversity.

This report has detailed the subsequent wave of innovation, which focuses on equipping LLMs with the algorithms of experimentation necessary to navigate these complex trade-offs. Frameworks for intelligent exploration, such as Active Learning and Curiosity-Driven Learning, empower models to guide their own data acquisition and foster novelty. Formal engineering methodologies like Design of Experiments and Neural Architecture Search provide the systematic rigor required to optimize the vast, high-dimensional spaces of model hyperparameters, data mixtures, and architectures. Finally, cognitive scaffolds like TRIZ offer a powerful, structured logic for resolving the fundamental contradictions that arise in complex system design, guiding the experimental process toward truly inventive solutions.

The future of AI will not be defined by brute-force scaling alone. It will be shaped by the sophistication of the learning algorithms we develop. By integrating frameworks for strategic data selection, intrinsic motivation, efficient optimization, and systematic problem-solving, we are not merely improving LLM performance—we are fundamentally changing their nature. We are transforming them from probabilistic text generators into partners in discovery and, ultimately, into autonomous engines of innovation. The path forward involves creating hybrid algorithms that merge these strategies and designing new architectures that support more complex, iterative reasoning. The ultimate vision is an AI that learns not just from the world as it is, but can systematically and inventively experiment to create what comes next.

Works cited

1. What Is Self-Supervised Learning? - IBM, accessed September 12, 2025, https://www.ibm.com/think/topics/self-supervised-learning
2. The Role of Self-Supervised Learning in LLM Development - GoML, accessed September 12, 2025, https://www.goml.io/blog/the-role-of-self-supervised-learning-in-llm-development
3. Self-Supervised Learning in the Context of LLMs | by Saurabh Harak ..., accessed September 12, 2025, https://saurabhharak.medium.com/self-supervised-learning-in-the-context-of-llms-5ae7fb729a38
4. Self-supervised Learning Explained - Encord, accessed September 12, 2025, https://encord.com/blog/self-supervised-learning/
5. Mathematical explanation of Transformer for Next Word Prediction | by Rohit Pegallapati, accessed September 12, 2025, https://medium.com/@rohit.pegallapati/mathematical-explanation-of-transformer-for-next-word-prediction-01bd15845058
6. Next Word Prediction with Deep Learning in NLP - GeeksforGeeks, accessed September 12, 2025, https://www.geeksforgeeks.org/nlp/next-word-prediction-with-deep-learning-in-nlp/
7. Transformer Explainer: LLM Transformer Model Visually Explained, accessed September 12, 2025, https://poloclub.github.io/transformer-explainer/
8. Large language model - Wikipedia, accessed September 12, 2025, https://en.wikipedia.org/wiki/Large_language_model
9. Transformer (deep learning architecture) - Wikipedia, accessed September 12, 2025, https://en.wikipedia.org/wiki/Transformer_(deep_learning_architecture)
10. How do LLMs work? Next Word Prediction with the Transformer Architecture Explained, accessed September 12, 2025, https://www.youtube.com/watch?v=wl3mbqOtlmM
11. How did language models go from predicting the next word token to answering long, complex prompts? - Reddit, accessed September 12, 2025, https://www.reddit.com/r/learnmachinelearning/comments/17gd8mi/how_did_language_models_go_from_predicting_the/
12. What is an LLM (large language model)? - Cloudflare, accessed September 12, 2025, https://www.cloudflare.com/learning/ai/what-is-large-language-model/
13. What Are Large Language Models (LLMs)? - IBM, accessed September 12, 2025, https://www.ibm.com/think/topics/large-language-models
14. What Is Reinforcement Learning From Human Feedback (RLHF ..., accessed September 12, 2025, https://www.ibm.com/think/topics/rlhf
15. aws.amazon.com, accessed September 12, 2025, https://aws.amazon.com/what-is/reinforcement-learning-from-human-feedback/#:~:text=Reinforcement%20learning%20from%20human%20feedback%20(RLHF)%20is%20a%20machine%20learning,making%20their%20outcomes%20more%20accurate.
16. What is RLHF? - Reinforcement Learning from Human Feedback, accessed September 12, 2025, https://aws.amazon.com/what-is/reinforcement-learning-from-human-feedback/
17. Reinforcement learning from human feedback - Wikipedia, accessed September 12, 2025, https://en.wikipedia.org/wiki/Reinforcement_learning_from_human_feedback
18. What is RLHF - Reinforcement Learning from Human Feedback - AI with Armand, accessed September 12, 2025, https://newsletter.armand.so/p/rlhf-reinforcement-learning-human-feedback
19. [2501.11463] Curiosity-Driven Reinforcement Learning from Human Feedback - arXiv, accessed September 12, 2025, https://arxiv.org/abs/2501.11463
20. Curiosity-Driven Reinforcement Learning from Human Feedback - arXiv, accessed September 12, 2025, https://arxiv.org/html/2501.11463v1
21. Exploration–exploitation dilemma - Wikipedia, accessed September 12, 2025, https://en.wikipedia.org/wiki/Exploration%E2%80%93exploitation_dilemma
22. The Exploration vs. Exploitation Tradeoff: Navigating Life's Choices | by Charles Chi | AI: Assimilating Intelligence | Medium, accessed September 12, 2025, https://medium.com/ai-assimilating-intelligence/the-exploration-vs-exploitation-tradeoff-navigating-lifes-choices-52925e540c63
23. Exploitation and Exploration in Machine Learning - GeeksforGeeks, accessed September 12, 2025, https://www.geeksforgeeks.org/machine-learning/exploitation-and-exploration-in-machine-learning/
24. [2502.00225] Should You Use Your Large Language Model to Explore or Exploit? - arXiv, accessed September 12, 2025, https://arxiv.org/abs/2502.00225
25. Should You Use Your Large Language Model to Explore or Exploit? - arXiv, accessed September 12, 2025, https://arxiv.org/html/2502.00225v1
26. [Revue de papier] Should You Use Your Large Language Model to Explore or Exploit?, accessed September 12, 2025, https://www.themoonlight.io/fr/review/should-you-use-your-large-language-model-to-explore-or-exploit
27. Large Language Models Think Too Fast To Explore Effectively - arXiv, accessed September 12, 2025, https://arxiv.org/html/2501.18009v1
28. [2501.18009] Large Language Models Think Too Fast To Explore Effectively - arXiv, accessed September 12, 2025, https://arxiv.org/abs/2501.18009
29. ActiveLLM: Large Language Model-based Active Learning for Textual Few-Shot Scenarios, accessed September 12, 2025, https://arxiv.org/html/2405.10808v1
30. ActiveLLM: Large Language Model-based Active Learning for Textual Few-Shot Scenarios - TUbiblio - TU Darmstadt, accessed September 12, 2025, https://tubiblio.ulb.tu-darmstadt.de/152290/
31. ActiveLLM: Large Language Model-based Active Learning for ..., accessed September 12, 2025, https://www.researchgate.net/publication/394804356_ActiveLLM_Large_Language_Model-based_Active_Learning_for_Textual_Few-Shot_Scenarios
32. [Literature Review] ActiveLLM: Large Language Model-based Active Learning for Textual Few-Shot Scenarios - Moonlight, accessed September 12, 2025, https://www.themoonlight.io/en/review/activellm-large-language-model-based-active-learning-for-textual-few-shot-scenarios
33. [Literature Review] Curiosity-Driven Reinforcement Learning from Human Feedback, accessed September 12, 2025, https://www.themoonlight.io/en/review/curiosity-driven-reinforcement-learning-from-human-feedback
34. Curiosity-Driven Reinforcement Learning from Human Feedback - ACL Anthology, accessed September 12, 2025, https://aclanthology.org/2025.acl-long.1146.pdf
35. Curiosity-Driven Reinforcement Learning from Human Feedback - arXiv, accessed September 12, 2025, https://arxiv.org/pdf/2501.11463
36. Motif: Intrinsic Motivation from Artificial Intelligence Feedback ..., accessed September 12, 2025, https://openreview.net/forum?id=tmBKIecDE9
37. NeurIPS 2023 Workshop ALOE - OpenReview, accessed September 12, 2025, https://openreview.net/group?id=NeurIPS.cc/2023/Workshop/ALOE
38. Mastering LLM Hyperparameter Tuning for Optimal Performance - DEV Community, accessed September 12, 2025, https://dev.to/ankush_mahore/mastering-llm-hyperparameter-tuning-for-optimal-performance-1gc1
39. What Is Hyperparameter Tuning? - IBM, accessed September 12, 2025, https://www.ibm.com/think/topics/hyperparameter-tuning
40. An Empirical Study of Issues in Large Language Model Training ..., accessed September 12, 2025, https://www.microsoft.com/en-us/research/publication/an-empirical-study-of-issues-in-large-language-model-training-systems/
41. Evaluating Designs for Hyperparameter Tuning in Deep Neural Networks, accessed September 12, 2025, https://nejsds.nestat.org/journal/NEJSDS/article/27
42. Orthogonal Arrays: A Review - arXiv, accessed September 12, 2025, https://arxiv.org/pdf/2505.15032
43. Reducing Tunning Time with Taguchi Arrays | by Waner Miranda - Medium, accessed September 12, 2025, https://medium.com/@wanermiranda/reducing-tunning-time-with-taguchi-arrays-cee52b87cc9d
44. Data Mixing Optimization for Supervised Fine-Tuning of Large Language Models - arXiv, accessed September 12, 2025, https://arxiv.org/html/2508.11953v1
45. RegMix: Data Mixture as Regression for Language Model Pre ..., accessed September 12, 2025, https://openreview.net/forum?id=5BjQOUXq7i
46. Data Mixing Laws: Optimizing Data Mixtures by Predicting ..., accessed September 12, 2025, https://openreview.net/forum?id=jjCB27TMK3
47. Leveraging LLMs as an Augmentation to Traditional Hyperparameter Tuning - AWS, accessed September 12, 2025, https://aws.amazon.com/blogs/hpc/leveraging-llms-as-an-augmentation-to-traditional-hyperparameter-tuning/
48. Neural architecture search - Wikipedia, accessed September 12, 2025, https://en.wikipedia.org/wiki/Neural_architecture_search
49. [2301.08727] Neural Architecture Search: Insights from 1000 Papers - arXiv, accessed September 12, 2025, https://arxiv.org/abs/2301.08727
50. Neural Architecture Search with Reinforcement Learning ..., accessed September 12, 2025, https://openreview.net/forum?id=r1Ue8Hcxg
51. Neural Architecture Search for Generative Adversarial Networks: A Comprehensive Review and Critical Analysis - MDPI, accessed September 12, 2025, https://www.mdpi.com/2076-3417/15/7/3623
52. Neural Architecture Search via Trainless Pruning Algorithm: A Bayesian Evaluation of a Network with Multiple Indicators - MDPI, accessed September 12, 2025, https://www.mdpi.com/2079-9292/13/22/4547
53. Design Principle Transfer in Neural Architecture Search via Large Language Models, accessed September 12, 2025, https://ojs.aaai.org/index.php/AAAI/article/view/34463/36618
54. What is TRIZ? - Altshuller Institute for TRIZ Studies, accessed September 12, 2025, https://www.aitriz.org/triz
55. TRIZ - Wikipedia, accessed September 12, 2025, https://en.wikipedia.org/wiki/TRIZ
56. TRIZ Technical Contradictions Matrix - Minitab Workspace - Support, accessed September 12, 2025, https://support.minitab.com/en-us/workspace/help-and-how-to/forms/types-of-forms/product-development/triz-technical-contradictions-matrix/
57. TRIZ-GPT: An LLM-augmented method for problem-solving - arXiv, accessed September 12, 2025, https://arxiv.org/html/2408.05897v1
58. Inventive Principles Illustrated, Part 1 - Interviews with Corporate Innovation Leaders, accessed September 12, 2025, https://www.ideaconnection.com/interviews/00353-inventive-principles-illustrated-part-1.html
59. The classical TRIZ contradiction matrix (the red cells are empty cells... - ResearchGate, accessed September 12, 2025, https://www.researchgate.net/figure/The-classical-TRIZ-contradiction-matrix-the-red-cells-are-empty-cells-or-cells-that-have_fig4_256079930
60. Examining the structural attributes of TRIZ contradiction Matrix using exploratory data analysis, accessed September 12, 2025, https://test-api.ijosi.org/uploads/file/asp/202505201008242d3af2461.pdf
61. Oxford TRIZ Innovation Tools, accessed September 12, 2025, https://www.triz.co.uk/learning-centre-innovation-tools
62. Application of Algorithm for Inventive Problem Solving (ARIZ) for the ..., accessed September 12, 2025, https://www.mdpi.com/2071-1050/15/9/7271
63. (PDF) An Introduction to ARIZ -The Algorithm of Inventive Problem Solving - ResearchGate, accessed September 12, 2025, https://www.researchgate.net/publication/235742388_An_Introduction_to_ARIZ_-The_Algorithm_of_Inventive_Problem_Solving
64. Introduction to TRIZ – Innovative Problem Solving - EE IIT Bombay, accessed September 12, 2025, https://www.ee.iitb.ac.in/~apte/CV_PRA_TRIZ_INTRO.htm
65. Enhancing TRIZ through environment-based design methodology supported by a large language model - Cambridge University Press, accessed September 12, 2025, https://www.cambridge.org/core/services/aop-cambridge-core/content/view/C3305E839793A17763076FF8BF510E08/S0890060425000083a.pdf/enhancing_triz_through_environmentbased_design_methodology_supported_by_a_large_language_model.pdf
66. Expanding Creative Possibilities: Exploring the Synergy Between Large Language Models (LLMs) and Theory of Inventive Problem-Solving (TRIZ) | UTCN-ROBOTICA, accessed September 12, 2025, https://utcn-robotica.ro/expanding-creative-possibilities-exploring-the-synergy-between-large-language-models-llm-and-theory-of-inventive-problem-solving-triz/
67. Artificial intelligence and TRIZ: a synergy for innovation, accessed September 12, 2025, https://www.triz-consulting.de/about-triz/artificial-intelligence-and-triz-a-synergy-for-innovation/?lang=en
68. A Multi-Agent LLM Approach for TRIZ-Based Innovation - SciTePress, accessed September 12, 2025, https://www.scitepress.org/Papers/2025/133219/133219.pdf
69. [2506.18783] TRIZ Agents: A Multi-Agent LLM Approach for TRIZ-Based Innovation - arXiv, accessed September 12, 2025, https://arxiv.org/abs/2506.18783