31 Aug Using the organization identified from Lab 1, identify what items would you would want to include in your chosen organization’s collection plan. Additionally, you will want to identify

NOVEMBER 2001

This Page Is Intentionally Left Blank

I

PREFACE

T preliminary his training publication information joint and coalition provides on the subject of Open Source Intelligence (OSINT). It discusses the fundamentals of OSINT support to both the all-source intelligence process, and to the unclassified intelligence requirements of operators, logisticians, and civilian organizations participating in joint and coalition operations.

The focus is on relevant information that can be obtained legally and ethically from the public and private sector, and that is not classified in its origin or processing.

The information may become classified in relation to the commander's intent or its association with classified information when it is rightly blended into all-source intelligence reports.

T prepared his the publication under Supreme my direction has Allied been as Commander, Atlantic, in collaboration with staff from the Supreme Allied Commander, Europe (SACEUR).

This

I

publication has benefited greatly from the continued collaboration between my staff and the staff of Open Source Solutions Inc. With the publication of this document and its companions, the

Intelligence Exploitation of the Internet and the NATO OSINT Reader,

commanders and their staffs will have basic guidance for the development of OSINT.

The increasingly robust array of open sources available to all staffs enable commanders at all levels to attempt to satisfy their information requirements themselves rather than immediately directing Requests for Information (RFIs) elsewhere.

This manual outlines a systematic approach to OSINT exploitation.

T his all forces, information NATO member commands, nations, is relevant civil- task to military committees and working groups, and such other organizations that may be planning or engaged in combined joint operations.

W. F. KERNAN General, U.S. Army

This Page Is Intentionally Left Blank

II

Table of Contents

EXECUTIVE SUMMARY CHAPTER I OPEN SOURCE INTELLIGENCE AND JOINT OR COALITION OPERATION

Introduction Definitions 21st Century Information Operations

CHAPTER II PRIVATE SECTOR INFORMATION OFFERINGS

SECTION A. SOURCES

Traditional Media Sources Commercial Online Premium Sources Other Forms of Commercial Online Information Grey Literature Overt Human Experts and Observer Commercial Imagery Defining Source Access Requirements (Dangers of Pay-per-View)

SECTION B. SOFTWARE

Software Hierarchy

SECTION C. SERVICES

Collection Services Processing Services Analysis & Production Services Services Examples

CHAPTER III THE OPEN SOURCE INTELLIGENCE CYCLE

SECTION A. OSINT PLANNING AND DIRECTION

Overview Organizations and Responsibilities Requirements Definition Evaluation and Feedback

SECTION B. COLLECTION

Overview Knowing Who Knows Collection Discipline Collection Issues Nuances of Open Source Collection

Intentionally Blank

SECTION C. PROCESSING AND EXPLOITATION

Overview Analysis Web-Site Authentication and Source Analysis

SECTION D. SEARCHING ANONYMOUSLY ON THE WEB

Overview Leaving a Footprint Traffic analysis

V 1 1

1 2 3

5 5 5

5 6 8 8 9 9 11

12

12

13

13 13 13 14

15 15 15

15 15 16 17

17

17

19

19 20 21

23

23 23 24

27

27 28 28

III

Contact with others

SECTION E. PRODUCTION

Overview Reports Link Tables Distance Learning Expert Forums

SECTION F. DISSEMINATION AND EVALUATION

Overview Dissemination Methods Virtual Intelligence Community

CHAPTER IV OSINT AND THE EMERGING FUTURE INTELLIGENCE ARCHITECTURE OF NATO

SECTION A. BLENDING OSINT INTO THE ALL-SOURCE PROCESS

Overview Direction Collection Processing Dissemination

APPENDIX A: GENERAL REFERENCE LINK TABLE APPENDIX B: TRAINING LINK TABLE APPENDIX C: CATEGORIES OF MISPERCEPTION AND BIAS APPENDIX D: LIST OF ABBREVIATIONS FEEDBACK

29

29

29 29 31 31 32

33

33 34 34

36 36 36

36 38 39 41 42

43 45 46 48 49

IV

EXECUTIVE SUMMARY

COMMANDER'S OVERVIEW

Open Source Intelligence (OSINT) in Joint and Coalition Operations

O pen information OSINT, Source is that Intelligence, unclassified has been or deliberately discovered, discriminated, distilled and disseminated to a select audience in order to address a specific question. It provides a very robust foundation for other intelligence disciplines. When applied in a systematic fashion, OSINT products can reduce the demands on classified intelligence collection resources by limiting requests for information only to those questions that cannot be answered by open sources.

Open information sources are not the exclusive domain of intelligence staffs. Intelligence should never seek to limit

Rather, access to open sources.

intelligence should facilitate the use of open sources by all staff elements that require access to relevant, reliable information. Intelligence staffs should concentrate on the application of proven intelligence processes to the exploitation of open sources to improve its all-source intelligence products.

Familiarity with available open sources will place intelligence staffs in the position of guiding and advising other staff elements in their own exploitation of open sources.

V

Open Source Intelligence and Joint or Coalition Operations

OSINT is a vital component of NATO’s future vision.

Through its concentration upon unclassified open sources of information, OSINT provides the means with which to develop valid and reliable intelligence products that can be shared with non-NATO elements of international operations.

Experience in the Balkans, and the increasing importance of the Partnership for Peace and Mediterranean Dialogue members in security dialogue, illustrates the need to develop information sources that enable broader engagement with these vital partners.

Private Sector Information Offerings

The Internet is now the default C4I architecture for virtually the entire world. The principle exceptions are most militaries and intelligence organizations. The Internet facilitates commerce, provides entertainment and supports ever increasing amounts of human interaction. To exclude the information flow carried by the Internet is to exclude the greatest emerging data source available. While the Internet is a source of much knowledge, all information gleaned from it must be assessed for its source, bias and reliability.

As a source of reliable information, the Internet must be approached with great caution. As a means with which to gain access to quality commercial sources of validated information, the Internet is unbeatable.

A vision of open source exploitation must not be limited exclusively to electronic sources.

Traditional print, hardcopy images and other analog sources continue to provide a wealth of data of continuing relevance to NATO intelligence.

The Open Source Intelligence Cycle

As the range of NATO information needs varies depending upon mission requirements, it is virtually impossible to maintain a viable collection of open source materials that address all information needs instantly.

The focus should be on the collection of sources, not information.

With knowledge of relevant and reliable sources of open source information, an intelligence staff can quickly devote collection energy and analytical expertise to develop tailored OSINT products to the mission need.

OSINT and the Emerging Future Intelligence Architecture of NATO

OSINT is an essential building block for all intelligence disciplines.

Open sources have always played a role in classified intelligence production.

In the NATO context, a robust OSINT capability greatly increases the range of information sources available to intelligence staffs to address intelligence needs.

VI

Nations are capable of tasking classified intelligence sources to address intelligence gaps. Lacking organic intelligence collection assets, NATO intelligence staffs are unable to task classified collection. Rather than immediately directed a Request For Information (RFI) to a national intelligence centre, a robust OSINT capability enables intelligence staffs to address many intelligence needs with internal resources.

While unable to replace classified intelligence production, OSINT is able to compliment an all-source intelligence production process with essential support including tip-offs, context, validation and cover for information sanitation.

CHAPTER I OPEN SOURCE INTELLIGENCE AND JOINT OR

COALITION OPERATIONS

"OSINT is not a substitute for satellites, spies, or existing organic military and civilian intelligence capabilities. It is, however, a foundation—a very strong foundation—for planning and executing coalition operations across the spectrum from humanitarian assistance to total war. OSINT provides strategic historical and cultural insights; it provides operationally helpful information about infrastructure and current conditions; and it provides tactically vital commercial geospatial information that is not available from national capabilities. In coalition operations, OSINT is both the foundation for civil-military cooperation, and the framework for classified bilateral intelligence-sharing."

Introduction

OSINT is distinct from academic, business or journalistic research in that it represents the application of the proven process of national intelligence to a global diversity of sources, with the intent of producing tailored intelligence for the commander. OSINT is also unique, within a coalition operations context, in that it simultaneously provides a multi-lateral foundation for establishing a common view of the shared Area of Operations (AOO), while also providing a context within which a wide variety of bi-lateral classified intelligence sharing arrangements can be exploited. Figure 1 illustrates these relationships.

Figure 1

INTELLIGENCE METHODS OPEN SOURCE INFORMATION • Internet • Commercial Online (Fee for Access) • Gray Literature • Subject-Matter Experts • Commercial Geospatial Information • Direct Ground and/or Aerial Reconnaissance • Complex Human and/or Technical Services • Requirements Definition • Collection Management • Source Discovery & Validation • Multi-Source Fusion • Compelling Timely Presentation • Evaluation, Feedback, New Requirements • Operational Security Securely Exploit Bi-Lateral Classified Intelligence from Nations Create Common Open Source Intelligence for Multi-Lateral Sharing COALITION COMMANDER & STAFF

OSINT is valuable to NATO member nations and to individual Partner nations in that it can be used to provide a common understanding of the AOO across all elements of its military forces and its civilian and non-governmental organization (NGO) counterparts. Elements of the forces not authorized access to the full range of classified information, often including such vital components, as military police, logistics elements, engineers, and the public affairs staff, can be made more effective through the utilization of tailored OSINT. At the same time, external parties with whom coordination is critical, but who are also not authorized access to classified information, can receive tailored OSINT that is helpful to a shared understanding of the AOO and the challenges facing the coalition and all its elements. Figure 2 illustrates this idea.

Exchange with People Exchange with Embassies Exchange with Business Community Exchange with Civil Authorities OSINT “NET” Exchange with Media POLAD PSYOP PIO J-3 COMBAT ENGINEERS PROVOST MARSHAL CIVIL AFFAIRS CINC J-2

Figure 2

Definitions

There are four distinct categories of open information and intelligence.

Open Source Data (OSD). Data is the raw print, broadcast, oral debriefing or other form of information from a primary source. It can be a photograph, a tape recording, a commercial satellite image, or a personal letter from an individual.

Open Source Information (OSIF).

OSIF

is comprised of data that can be put together, generally by an editorial process that provides some filtering and validation as well as presentation management.

OSIF is generic information that is usually widely

Newspapers, books, broadcast, and general daily reports are part of the OSIF world.

disseminated.

Open Source Intelligence (OSINT).

OSINT is information that has been deliberately discovered, discriminated, distilled, and disseminated to a select

audience, generally the commander and their immediate staff, in order to address a specific question.

OSINT, in other words, applies the proven process of intelligence to the broad diversity of open sources of information, and creates intelligence.

Validated OSINT (OSINT-V).

OSINT-V

is information to which a very high degree of certainty can be attributed. produced by an all-source intelligence professional, with access to classified intelligence sources, whether working for a nation or for a coalition staff.

It can also come from an assured open source to which no question can be raised concerning its validity (images of an aircraft arriving at an airport that are broadcast over the media).

21st Century Information Operations

OSINT is an essential contextual and foundation element for classified intelligence operations.

Overt human sources can help target and validate clandestine human intelligence (HUMINT) sources. Overt broadcast information can be used to better understand covertly collected signals intelligence (SIGINT).

Commercial geospatial information, especially wide-area surveillance imagery, can be used to significantly enhance the value of the more narrowly focused covert imagery intelligence (IMINT) capabilities.

OSINT can also make contributions to the emerging discipline of Measurements and Signatures Intelligence (MASINT), to Counter intelligence (CI), and to Operations Security (OPSEC).

OSINT is the major new "force" in 21st Century Information Operations (IO). OSINT is not "new" in that Nations and organizations have always understood the value of legal travelers, direct observation, structured reading, and legal purchases of information services.

What is new about OSINT is the confluence of three distinct trends: first, the proliferation of the Internet as a tool for disseminating and sharing overt information; second, the consequent and related "information explosion" in which published knowledge is growing exponentially; and third, the collapse of many formerly denied areas.

OSINT is important to coalition commanders and their staffs for another reason: emerging threats, and the lower end

It can be

• f the spectrum of conflict, increasingly demand out-of-area operations and engagement in operations for which classified intelligence support is not readily available. Out of area operations such as humanitarian assistance and disaster relief
• perations in the countries of Africa or elsewhere along the NATO periphery, are all characterized by complex information needs related to infrastructure, demographics, health, and other matters not traditionally addressed by classified intelligence collection operations.

OSINT is vital to government operations, and especially to coalition operations, for one additional reason: the changing nature of command & control in the 21st Century. In the past, nations and even coalitions relied heavily on a top-down "chain of command" that relied on closed sources to direct generally unilateral actions with short term time frames.

Today, as non governmental organizations come to the fore and are often the predominant factors in many of the operations that the military must support, the dynamics of both command & control and information have changed.

Within NATO, operations must be planned and executed in a multi-cultural fashion, with bottom-up consensus often being the most effective means of arriving at sustainable decisions.

This is particularly true with the vital role played by non-NATO troop contributing nations.

Under these circumstances, a common view of the operating area, formed with the help of validated OSINT is often the most effective means of delivering decision-support.

The remainder of this manual will discuss private sector information offerings, the open source intelligence cycle, and the integration of OSINT into the NATO and prospective PfP coalition operations. OSINT will be a core element of the NATO Future Intelligence Architecture.

Special Note on Operational Security

The most common objection to the use of open sources of information, apart from the general lack of knowledge and funding with which to exploit open sources, relates to Operational Security (OPSEC). This topic is fully discussed within Chapter III. The Open Source Intelligence Cycle makes full provision for OPSEC at every stage, and ample methods exist to conceal the commander's intent, the source of the inquiry, and other sensitive aspects of open source exploitation.

CHAPTER II PRIVATE SECTOR INFORMATION OFFERINGS

The four pillars to an OSINT strategy are sources, software, services and analysis. The private sector can address all four to some degree.

Analysis is the key enabling skill that is essential to the successful integration of OSINT into an all-source intelligence product.

While some analysis of open sources can and should be acquired from private sources, those analytical skills

SECTION A.

necessary to integrate open source derived intelligence must be grown and nurtured within intelligence staffs.

Analysis will be discussed further in Chapter III. This chapter is intended to expose the wider audience to the range of OSINT-related products that the private sector are optimized to provide.

SOURCES

Traditional Media Sources

To many, media sources were the only open sources that were familiar prior to the onset of the Internet.

These include traditional foreign print and broadcast media, radio and TV as well as the current array of electronically available products.

For current intelligence purposes, media sources remain the core capability necessary for an

OSINT effort and are available from a variety of providers.

Direct wire-service feeds are available.

Commercial online premium sources discussed below all provide an array of media sources on a fee for service basis.

While not private sector information providers, the U.S. Foreign Broadcast Information Service (FBIS) and the British Broadcasting

Corporation

(BBC)

Monitoring Service each provide excellent near real-time translation of foreign media sources. In addition, an array of media analysis products supplement the direct listing of foreign broadcasts and provide useful insight into the general character of foreign media reporting on particular issues.

Internet

The Internet has, since 1994, literally exploded on to the world scene and changed forever the manner in which individuals might carry out global research.

According to Dr. Vinton Cerf, acknowledged by many to be one of the founders of the Internet, it will grow from 400 million users in November 2000, to an estimated 3.5 billion users by the year 2015.

Apart from this exponential increase in the number of human beings using the Internet, other experts project a double or triple order of magnitude increase in the use of the Internet to connect devices, from geospatial locators in vehicles, to temperature detectors in soda machines, to usage monitors in doorways. The Internet is at the very beginning of its development as a global grid of enormous value to coalition operators, logisticians, and intelligence professionals.

Figure 3

The Internet has been over-sold in the past. A study by the Community Open Source Program Office (COSPO) within the U.S. Intelligence Community concluded in 1994 that the Internet only contained roughly 450 useful substantive sites, and that 99% of the Internet was not content of intelligence value, but rather pornography, opinion, and advertising. This earlier evaluation of the intelligence potential of the Internet no longer reflects the extensive content that is now available.

Some suggest that over 250,000 databases are now available within the “deep web”, a great many of which are of potential intelligence value.

While the Internet has grown substantially in value since 1994, the intelligence professional must be very cautious about both over-reliance on the Internet, and about the source bias of materials found there.

In general, Internet sources are rarely dated, formatted, paginated, edited, filtered, or stable, even when addressing substantive topics.

The Internet is an "easy out" for operators and other consumers of intelligence. It is an attractive option for commanders and staff in a hurry.

If intelligence professionals do not demonstrate that they monitor and exploit the Internet, and/or if intelligence professionals make it too difficult for consumers to obtain usable all-source intelligence, the Internet represents a "threat" to the existing intelligence process. Increasingly, intelligence professionals must act to place information that is widely available on the Internet into its proper context – either confirming its validity or disputing the information based on classified collateral reporting.

In general, the Internet today provides two benefits to the coalition professional: first, as a means of rapidly communicating with counterparts around the world, primarily to exchange unclassified information and professional insights; and second, as a means of rapidly accessing both free and premium (fee paid for access) information sources. However, the Internet also has its dangers. Electronic mail and attached documents comprise a permanent record in cyber-space, and the sender has little control over subsequent dissemination and exploitation.

OSINT Professional Note: A number of advanced search tools are available that complement the variety of search engines that are freely available on the Internet. OSINT managers should remain abreast of developments in the field of Internet tools and integrate appropriate tools, as they become available, into their OSINT process. An example, its basic form available free or in an advanced version at a small cost, is a meta-search engine that combines the best features of multiple search engines, while also permitting subsequent searches for new information (remembering what has already been seen). Download this program from www.copernic.com.

Commercial Online Premium Sources

There are numerous commercial online premium sources, that is, sources that charge either a subscription fee or a usage fee for access to their information.

It is essential

that every professional understand the availability and the value of commercial online premium sources. They represent decades worth of editorial selection,

authentication, formatting, indexing, abstracting, and presentation management. In general, source material obtained through a commercial online premium service has been created by a reputable commercial enterprise subject to scrutiny and the judgment of the marketplace.

In Figure 4, we discuss the three best known to governments and corporations.

There are many others, some unique to Europe or Asia. Each professional is urged to consult

thorough understanding of its pricing structure. Even commands with flat-fee pricing should be aware that their next contract will be increased in price based on actual usage during the current flat-fee period. Alternatively, an option is to gain access to commercial sources via the services of a professional librarian or commercial information broker. Most professional information brokers, such as those belonging to the Association of

FACTIVA	LEXIS-NEXIS	DIALOG
www.factiva.com	www.lexis-nexis.com	www.dialog.com
Best web-based user interface, easiest means of searching all available publications. Archive of publications varies but typically provides several years worth of historical file. Includes Jane's Information Group material as well as BBC transcripts. Does not include FBIS information.	Two separate channels, one focused on legal sources including public records (primarily in the United States but very helpful in tracing real estate, aircraft, and water craft including international ships), the other focused on news sources but offering archive access, i.e. ability to reach back several years or more on any topic.	A very large collection of various commercial offerings that can be searched "by the file". Especially valuable for access to conference proceedings, academic and policy journals, dissertations, book reviews, and the Social Science Citation Index (SSCI). The latter is ideal for finding and ranking individual experts, to include identification of their official address.
Flat fee or actual cost pricing.	Flat fee, actual cost, or pay as you go credit card pricing.	Flat fee, actual cost, or pay as you go credit card pricing.

Figure 4 – Leading Commercial Online Premium Sources

his or her librarian or his or her OSINT collection manager to gain a better understanding of what their options are for high-quality commercial information relevant to their action responsibilities.

In general, and in part because of the high cost of mistakes or unnecessary retrievals, all commercial online premium services should be searched by those staff with sufficient training on the database and a

Independent Information Brokers (AIIB), specialize in either LEXIS-NEXIS or DIALOG. There are distinct advantages in contracting a searcher who has detailed familiarity with the very arcane search command characteristics of these two services. In the case of Factiva this is less vital but can still make a big difference in both the success of the searchers, and the cost of the searches.

OSINT Professional Note: Always ask for search results in electronic form, these files can more easily be shared. Copy the results into a Word document. Add pagination. Add a title page and a blank table of Sort the items into larger categories (e.g. Political, Military, Economic) and label the contents page. categories as "Heading 1". Then go through the document label each individual headline as "Heading 2". These headings are choices in the style bar at the upper left that generally says "Normal". Finally, go to the Table of Contents and use the Insert, Index and Tables, Table of Contents choices to insert a table of contents. If desired, use the Replace function to find and make bold all of the original search terms.

Other Forms of Commercial Online Information

There is a vast range of commercial sources available through direct subscription, both on the Internet and in the form of hard copy or CD-ROM publications.

Table 1 below identifies just a few sources of com