Analytic Culture in the U.S. Intelligence Community: An Ethnographic Study - Working as an Intelligence Analyst, Central Intelligence Agency (CIA) Intelligence Papers

From the Foreword: It is a rare season when the intelligence story in the news concerns intelligence analysis, not secret operations abroad. The United States is having such a season as it debates whether intelligence failed in the run-up to both September 11 and the second Iraq war, and so Rob Johnston's wonderful book is perfectly timed to provide the back-story to those headlines. The CIA's Center for the Study of Intelligence is to be commended for having the good sense to find Johnston and the courage to support his work, even though his conclusions are not what many in the world of intelligence analysis would like to hear.

He reaches those conclusions through the careful procedures of an anthropologist—conducting literally hundreds of interviews and observing and participating in dozens of work groups in intelligence analysis—and so they cannot easily be dismissed as mere opinion, still less as the bitter mutterings of those who have lost out in the bureaucratic wars. His findings constitute not just a strong indictment of the way American intelligence performs analysis, but also, and happily, a guide for how to do better.

Johnston finds no baseline standard analytic method. Instead, the most common practice is to conduct limited brainstorming on the basis of previous analysis, thus producing a bias toward confirming earlier views. The validating of data is questionable—for instance, the Directorate of Operations' (DO) "cleaning" of spy reports doesn't permit testing of their validity—reinforcing the tendency to look for data that confirms, not refutes, prevailing hypotheses. The process is risk averse, with considerable managerial conservatism. There is much more emphasis on avoiding error than on imagining surprises. The analytic process is driven by current intelligence, especially the CIA's crown jewel analytic product, the President's Daily Brief (PDB), which might be caricatured as "CNN plus secrets." Johnston doesn't put it quite that way, but the Intelligence Community does more reporting than in-depth analysis.

None of the analytic agencies knows much about the analytic techniques of the others. In all, there tends to be much more emphasis on writing and communication skills than on analytic methods. Training is driven more by the druthers of individual analysts than by any strategic view of the agencies and what they need. Most training is on-the-job.

Johnston identifies the needs for analysis of at least three different types of consumers—cops, spies, and soldiers. The needs of those consumers produce at least three distinct types of intelligence—investigative or operational, strategic, and tactical.

The research suggests the need for serious study of analytic methods across all three, guided by professional methodologists. Analysts should have many more opportunities to do fieldwork abroad. They should also move much more often across the agency "stovepipes" they now inhabit. These movements would give them a richer sense for how other agencies do analysis.

CHAPTER ONE - Definitions * CHAPTER TWO - Findings * CHAPTER THREE - A Taxonomy of Intelligence Variables * CHAPTER FOUR - Testing the Intelligence Cycle Through Systems Modeling and Simulation * CHAPTER FIVE - Integrating Methodologists into Teams of Experts * CHAPTER SIX - The Question of Foreign Cultures: Combating Ethnocentrism in Intelligence Analysis * CHAPTER SEVEN - Instructional Technology: Effectiveness and Implications for the Intelligence Community * CHAPTER EIGHT - Organizational Culture: Anticipatory Socialization and Intelligence Analysts * CHAPTER NINE - Recommendations: The First Step: Recognizing A Fundamental Problem * CHAPTER TEN - Survey Methodology * CHAPTER ELEVEN - Q-Sort Methodology * CHAPTER TWELVE - The "File-Drawer" Problem and Calculation of Effect Size