An Assessment of the Intelligence Advanced Research Projects Activity (IARPA): Operations and Impact, 2007 - 2026

Abstract

Organizations involved in multidisciplinary research and development (R&D) programs, like the Intelligence Advanced Research Projects Activity (IARPA), often focus the external work of their teams on "challenge problems," complex issues requiring critical thinkers to offer novel solutions that push the boundaries of existing science and technology. However, an internal looking macro study of the organization—tracking and articulating its impact—and ensuring the mechanisms are in place to gather this data over time—are not necessarily codified in the standard operating procedures of R&D and science and technology (S&T) focused organizations. As researchers Azoulay et al. acknowledge, empirically evaluating the “ARPA” model is inherently difficult. Establishing causal connections between IARPA-funded research and follow-on developments of technology is complicated given the panoply of players involved (e.g. academia, industry, government) and the nonlinear nature of sponsored technology becoming operational over time. Developing impact tracking systems for public research funding programs is both an organizational and research challenge. To address these problems, using unclassified internal data from IARPA, in conversation with IARPA and intelligence community (IC) leadership, and in considering recent scholarship of the “ARPA” community, this thesis examines how effectively IARPA achieves its stated mission and goals by analyzing its operations and assessing the impacts of its programs. It finds that IARPA has built a strong organizational foundation for innovation as a flat, semi-autonomous, agile, risk-tolerant entity focused on the testing and evaluation (T&E) of its programs, but that its impact tracking, technology transition processes, and customer feedback mechanisms could be improved. These challenges limit IC returns on IARPA-funded research, and internally, complicate IARPA’s ability to fully understand and communicate its impact. The key takeaway from this work is that IARPA’s impact is not fully understood and realized not because of its program research quality, but limitations in impact tracking, knowledge retention, dissemination, and adoption, all of which are addressable through specifically targeted institutional reforms proposed on page 65. I direct practitioners seeking bottom line up front “BLUF” actionable takeaways from this academic work to the findings and recommendations section of the thesis (Chapter VII, Findings and Recommendations), which offers practical, implementable interventions that address these and other organizational challenges. These proposed interventions are consolidated into a summary matrix at the end of that chapter for quick reference on page 74. Because direct causal attribution is difficult to establish in multi-actor research and development R&D ecosystems, this thesis uses a mixed-method design combining interviews, open-source portfolio analysis, and case-study tracing to assess IARPA’s organizational effectiveness. The thesis defines IARPA’s effectiveness in two ways: on a micro level, through transition successes, operational adoptions, and research outputs (see Chapter V: Qualitative Analysis: IARPA Success Stories and Operational Impacts). On a macro level, it considers how to measure the IC’s ability to integrate IARPA-funded research into on-the-job support for analysts, improve the quality of finished intelligence, and to provide the IC with “overwhelming advantage,” per IARPA’s stated mission. It does so by referencing scholarship focused on other “ARPAs” (e.g. Bonvillian, Goldstein and Kearney, and the National Academies of Science, Engineering, and Medicine) whose research offers relevant methodological approaches to assessing IARPA impact. Scholarship about the intelligence community should be useful, endeavoring to close the gap between academics and practitioners. The thesis sits at this nexus. Finally, accountability—such as stewardship of funds—is even more important for intelligence organizations that, by necessity, operate outside the public preview. It is risky for IARPA to assume it will indefinitely enjoy public support in the coming decades as Presidents and Congress change hands. Federal budget pressures may wax and wane, and the U.S. intelligence community (IC) competes against domestic programs, social entitlements, and other demands. For IARPA to continue its groundbreaking work, it must be rigorous about assessing and communicating its efficacy and efficiency, and to be prepared to defend its record to relevant stakeholders with data it has carefully and, to the extent possible, comprehensively tracked and maintained over time. The culmination of the thesis research is the idea of intelligence organizational readiness: an inward examination of how the intelligence community, and specifically, IARPA, might adapt to meet the challenges facing policymakers and warfighters and optimize its performance to that end. I began this thesis by asking IARPA what would be most beneficial for them to understand about themselves, organizationally, and that conversation shaped this product. This thesis serves as an open invitation to IARPA to continue the conversations that have powerfully informed this work. A close reading of an existing scholarly assessment of IARPA by scholar practitioner William Bonvillian and review of relevant data points from ARPA Energy (ARPA-E) impact reports anchored this assessment. I identify organizational and structural challenges for IARPA through a review of unclassified lessons learned submitted by Program Managers at IARPA that show that some aspects of IARPA organizational practices are not conducive to organizational impact. To my knowledge, no reviews of IARPA’s effectiveness exist from which baseline data or trends over time might be assessed, either for internal organizational use or external scholarly evaluation. This thesis is the first comprehensive academic assessment of IARPA of its kind, drawing on rare primary source access including leadership interviews and unclassified internal documents. The views expressed in this thesis do not reflect the official policy or position of the Department of Defense, the U.S. Intelligence Community, or the U.S. Government. The thesis is not finished intelligence or finished policy. The validity, reliability, and relevance of the information contained have not been reviewed through intelligence or policy procedures and processes. This thesis uses only unclassified data.