Emergent Structure and Awareness Support for Intelligence Analysis

Claus Atzenbeck, David L. Hicks and Nasrullah MemonDepartment of Software and Media Technology

Aalborg University EsbjergNiels Bohrs Vej 8, 6700 Esbjerg, Denmark{atzenbeck, hicks, nasrullah}@cs.aaue.dk

AbstractBased on the emergent nature of terrorist groups, we

propose spatial hypertext as a tool for intelligence ana-lysts for representing, reasoning upon, and communicat-ing information. Furthermore, we evaluate awareness asan important feature for analysts to realise hidden asso-ciations between various people or objects. Finally, wediscuss our prototype Socs, a general spatial structure ap-plication with high level support for awareness, under theumbrella of intelligence analysis requirements.

Keywords—intelligence analysis, emergent structure, spa-tial structure, awareness, knowledge representation

1 IntroductionThe World Wide Web has been initially envisioned as

a space where information can be seamlessly interlinkedand accessed. Consequently, research has concentrated onways to make the process of publishing, interlinking, andaccessing information on the Web semantically rich, moreeffective, and convenient. Hypertext has played an impor-tant role in this context, because it acted as the concep-tual foundation of the WWW. Many concepts, ideas, andmethods introduced in the field of hypertext can now bewitnessed in the WWW context.

The latest movement on the WWW is “Web 2.0”.The term was introduced by Tim O’Reilly [19] and de-scribes Web applications that focus (among other things)on user participation and community building. Anotherrecent trend on the WWW is working collaborativelyon shared documents. Examples include new servicesbased on Web 2.0 technology, such as Google documents(http://docs.google.com), including shared text documents,spreadsheets, and presentation documents. Another popu-lar tool for shared document creation are wikis which en-able a large number of users to edit text corpora collabo-ratively. A well-known example is Wikipedia, a free Web-based encyclopaedia with a vast number of articles avail-able in many different languages. (For example, the En-

glish version of Wikipedia currently contains over 2.3 mil-lion articles.)

The Web, however, also became an important mediumfor illicit activities, such as those of terrorist organizations,who utilize it to communicate, spread information to fol-lowers, or recruit new members. This is known as the“Dark Web” [6]. Terrorist groups gain advantages fromusing the Web as a means of communication [24]. It sup-ports various kinds of media, such as text, audio, or video,permits potential members easy and quick access to in-formation and permits them to engage in debates. Groupmembers can be easily assisted from a distance. Accord-ing to [26] “The media platform most favoured by activistsand supporters of Islamist organizations is the Internet,due to its anonymity, availability, low cost and instanta-neous communication.” Due to its extensive use by terror-ist groups, the Web acts as an important data source forintelligence services and counterterrorism efforts [21, 6].

An important methodology to analyse and track terror-ist groups and their structures is to collect and analyse anyand all relevant available data. Special tools are neces-sary to deal with the vast amount of data that is carriedover the Internet, such as e-mail communication or WWWcontent. This class of applications includes data collection(e. g., Web mining [15]) and social network analysis (SNA)[16]. The latter aims at finding and evaluating connectionsbetween members of terrorist groups in order to show howsuch a group can be destabilized.

Outputs from SNA tools are not the only source intel-ligence agents use for decision making. There are variousother sources and information elements that are not partof the repository the analysis tools use. This may includefor example personal e-mail files or information providedby the phone or in meetings, as well as any reasoning thatintelligence agents come up with.

Intelligence agents merge and organize pieces of infor-mation from different sources in order to reason about themand support their decision making process. The structureof the relationships between these pieces of information

12th International Conference Information Visualisation

1550-6037/08 $25.00 © 2008 IEEE

DOI 10.1109/IV.2008.44

326

is fragile by nature, since new information may change itsubstantially. Besides supporting the emergent nature ofincoming information, such structures should also be anappropriate medium for communicating with others. Fi-nally, their presentation should foster awareness and permitnotification services that inform the analyst about potentialunseen and non obvious connections beyond the borders ofindividual data sources.

For example, the need for associating information andcommunicating those associations is clearly stated in the9/11 report [10]. It identifies both as important factors. Forinstance, it points out that “not sharing information linkingindividuals in the Cole attack to Mihdhar” and “not linkingthe arrest of Zacarias Moussaoui, described as interestedin flight training for the purpose of using an airplane in aterrorist act, to the heightened indications of attack” [10]played a role in failing to prevent the attack.

In this paper we identify the need in the intelligenceanalysis process for supporting emergent structures thatfoster communication and awareness. We propose them aswell suited companions for intelligence agents that wouldenable them to represent and communicate their thinkingprocesses. They can serve as a means for merging, asso-ciating, and further processing information provided by avariety of sources, including SNA tools.

The paper is structured as follows: Section 2 describeshow spatial structures can serve as a means for handlingemergent information, and help solve some of the prob-lems the node–link paradigm encounters when used forthat task. Due to the emergent nature of terrorist networks,spatial structures appear to be an attractive medium for in-telligence analysis. Furthermore, we describe how they canserve as a means for communication and discuss brieflysupport for awareness and notification services. Section 3discusses our prototype and points out its benefits in thecontext of intelligence work. Finally, Section 4 concludesthe paper and briefly describes future research directions.

2 Supporting Intelligence Agents2.1 Overview

Intelligence analysts deal with information from a vari-ety of sources, all of which are important to their decisionmaking process. Additionally, as pointed out by the 9/11report [10], linking and communicating those pieces of in-formation is an important issue.

In order to deal with the increasing amount of data avail-able especially through the Internet, automatic tools areused to harvest relevant information and compute relation-ships that implicitly exist in the “raw data”. The outputis a pre-selection that helps analysts to focus on the mostrelevant parts.

Those tools, however, focus on a predefined repositoryand are limited in their structural representation. Due to

their focus on computation, most of them model relation-ships as graphs. Graphs have been well researched andthus permit the application and use of efficient mathemati-cal models and algorithms.

Even though machines are necessary to deal with thevast amount of information, final decisions, however, aretaken by humans. Analysts need support for their deci-sion making process, of which SNA tools play an impor-tant role. Special applications may support agents in thisphase by fulfilling the following requirements:

1. Supporting the emergent and fragile nature of thecreated structure and fostering its communicationamong analysts.

2. Integrating with the information sources used by theanalyst, permitting them to be represented and struc-tured in a common information space.

3. Supporting awareness of linked information beyondthe boundaries of the sources that provided it, suchthat notification becomes possible.

Supporting emergent structure as a means for knowl-edge representation, communication, integration, andawareness/notification has been and still is discussed indepth in hypertext research. Thus, we apply this researchdirection to the needs of intelligence analysts, includingstructuring and representing data delivered by web miningor SNA tools.2.2 Emergent Structure Support

Organizing and making sense of information is an im-portant task for knowledge workers and the main focus ofhypermedia research from its very beginning. Hyperme-dia systems aim at augmenting human intellect, that is “in-creasing the capability of a man to approach a complexproblem situation, to gain comprehension to suit his par-ticular needs, and to derive solutions to problems” [7].

The most widely used structure abstractions in hyper-text are nodes and links. Nodes are informational unitsthat can be connected through links. Users can traverselinks and thereby navigate through a hypertext. Nodes andlinks, however, can be criticised for a lack of support foremergent structures. In fact, this was clearly pointed outover 16 years ago when Frank Halasz in his 1991 Hyper-text Conference keynote address demanded “Ending theTyranny of the Link” [9]. He proposed so-called “supra-network hypertexts”, that are structures other than thosebased on networks. Spatial hypertext (e. g., Aquanet [12]at that time) was such an example.

Spatial hypertext is often described as being based ona card metaphor. Like cards on a table, nodes positionedin a virtual space can be moved and organized. The idea

327

is to permit the structuring of information and author-ing of associations through visual cues (e. g., colour, size,shape) and spatial proximity. Recent examples of systemsthat provide these capabilities are the Visual KnowledgeBuilder (VKB) [22] and Tinderbox [3].

Spatial hypertext was designed for and is well suitedfor dealing with emergent structures [14]. It serves “likesketch representations [8] for hypertext authoring” [25]and fosters reflections in early design phases [17]. Spatialhypertext is more than just another visualization technique:“it is also a way to take advantage of human perceptualabilities in hypertext navigation, and to provide users witha fairly intuitive medium through which they may expressnew structures and manipulate existing structures” [13].

Even though graph-like structures (e. g., nodes andlinks) are good prerequisites for the automated process-ing of networks of information and are commonly usedfor SNA (e. g., [6]), the characteristics of spatial hypertextmake it the tool of choice for the human reflection on andreasoning about emergent structures. Referring to [23],[22] argues that “[u]sing node and link hypertext to ex-press an emerging or evolving understanding means struc-turing and restructuring the material over time. This over-head makes traditional hypertexts ill-suited to many typesof analysis and design tasks – particularly, situations wherea lot of information must be interpreted rapidly or where agroup shares and restructures information in order to coor-dinate or reach consensus.”

Due to the emergent and dynamic nature of Inter-net communities (e. g., Web 2.0 online communities [5]),spatial hypertext becomes a natural tool for modellingthose. For example, virtual communities can subdivideinto smaller groups when new members join. [20] usesthe expression “ ‘fractal’ aspect of virtual communities” todescribe this process.

Intelligence analysts are supported by Web mining andSNA tools to harvest vast repositories and find implicit re-lationships in the information gathered. However, to sup-port later phases of the analysis process, such as the post-processing of this information, and also the enrichment ofit through the identification of additional relationships, spa-tial structures would be beneficial. Similar to designers,information or intelligence analysts need tools appropriateto the task at hand: “What is important is to give designersrepresentational media that allow them to externalize whatthey want to externalize in ways they like.” [17]2.3 Communication Support

As mentioned in the introduction, a major problem iden-tified in conjunction with the tragic events of 9/11 was thelack of communication among U.S. government agencies:

“The U.S. government has access to a vastamount of information. But it has a weak sys-

tem for processing and using what it has. Thesystem of ‘need to know’ should be replacedby a system of ‘need to share.’ ” [10]

Protocols are important for sharing information. Thoseinclude technical (i. e., the software and hardware infras-tructure) as well as social protocols (i. e., the social roleand interaction among team members) [4]. The structuretype used as a means of communication must foster theappropriate encoding and decoding of the passed message.

Spatial hypertext has been proposed as an alternative toformalised structures, especially for collaborative environ-ments. “Users are hesitant about formalization because ofa fear of prematurely committing to a specific perspectiveon their tasks; this may be especially true in a collabora-tive setting, where people must agree on an appropriateformalism and the conventions for encoding informationinto them” [23]. Thus, spatial hypertext is more appro-priate for communication than formalized structures (e. g.,node/link-based).2.4 Awareness and Notifications

Awareness is another important issue for collaborativeinformation spaces, even though it has not yet been imple-mented in many relevant applications, as a survey of 19Web-based collaborative systems indicates [18].

Different types of awareness have been discussed inthe research community. Those include personal (“Whoam I?”), social (“Who communicates with me?”), in-formal (“Who is doing what?”), group (“Who is in mygroup?”), and workspace awareness (“What is happen-ing?”) [11]. Analysis and notification capabilities wouldsupport collaboration among intelligence agents and fosterbetter communication. Furthermore, workspace awarenesswould enable agents to realise and understand hidden con-nections between what they have already structured and thesubjects of their investigations (i. e., terrorists and their net-works).

Structuring tools for intelligence agents should providehigh level awareness for better collaboration support aswell as workspace awareness for getting notified about im-portant implicit connections of the analyst’s current con-text (e. g., open documents) to terrorists or their organiza-tions. In order to ensure this, such tools must be integratedinto the user’s computer environment. They should haveaccess to various digital information sources that are avail-able through different applications (e. g., SNA tools or e-mail), permit associating and sharing them in a commoninformation space, and support sending notifications aboutcontext related implicit connections to the user.

3 Socs – a PrototypeIn the following we will describe our prototype Socs,

that integrates important parts of a typical computer sys-

328

tem used by knowledge workers for their daily work. Ithas been implemented in the Mac OS X environment. Thisprototype version of Socs provides a space for representingand visualizing the people with which a person interactsand collaborates. (In an upcoming version Socs will alsosupport arbitrary documents.) The security concerns of in-telligence agencies make it difficult to observe and analysein detail the work processes of intelligence analysts. There-fore the prototype will be explained from the perspectiveof knowledge workers who are authoring Wikipedia arti-cles collaboratively. This setting, however, is not uniqueand the information source can be replaced by any othersource, such as open source intelligence (OSINT) or SNAdata.

Figure 1 depicts a screenshot of the Socs prototype. Inthe centre of the Figure the so-called “social space” canbe seen, on which representations of people (e.g., thosewith which a person collaborates) can be arranged. In ad-dition to spatial attributes (i. e., proximity or alignment),colours and size can be used for expressing associationsamong collaborators. “Drop groups” are marked regionsin the background, as shown in the screenshot. They rep-resent groups. In order to address the focus–context prob-lem when a large number of objects are placed in a space,Socs features zooming techniques to switch between anoverview and a detailed view [1].

Socs integrates the system-wide address book availablein the Mac OS environment which is used by various otherapplications, such as Mail (i. e., an e-mail client) or iCal(i. e., a calender application) to find e-mail addresses oridentify meeting attendees. The user can add new “peo-ple” to the social space by dragging & dropping from theaddress book or the built-in address book browser (see Fig-ure 1). They are represented on the space by the person’spicture, name, or Wikipedia ID (whatever is available).

“Drop groups” are represented as groups in the addressbook. They get instantiated by dragging a group fromthe address book onto the space. This also adds all cur-rent members of the group on top of the newly createddrop group. If a person is removed from or moved ontoa drop group in Socs it will be synchronised immediatelyin the address book database. Even though much infor-mation expressed via the spatial structure on the space isnot represented in the address book (e. g., spatial arrange-ment or colour), the basic group-based structure is equiva-lent and usable by all applications connected to the addressbook framework. Drop groups look similar to adornmentsin Tinderbox [3], however, they have narrower semantics,since they are used primarily for grouping people.

Socs’ integrated Web-browser can be used to browse ar-bitrary Web pages. However, if Socs discovers a supportedWeb site, it will extract and display user relevant data. At

the moment Socs supports only the Wikipedia website asan information source. For any article displayed in thebrowser, Socs can display its authors. The screenshot inFigure 1 shows them listed at the Wiki Author’s window.In addition to the Wikipedia login name, the user can seethe authors’ activity count (i. e., how many changes weresubmitted), which ones are already part of the Socs socialspace (“S”), and who is part of the address book (“AB”, notshown in the screenshot). Authors of the currently viewedWikipedia article who are also on the Socs social space willbe highlighted there in red.

Records at the Wiki Author’s window can be draggedonto the space. This will create a new instance of that per-son on the space as well as one in the address book. Ifa corresponding entry exists already in the address bookdatabase, Socs will use that one as reference instead.

The linking of information about who is on the socialspace, who is in the address book, and who authored thecurrently displayed article increases various kinds of theawarenesses that were described earlier, especially groupawareness and workspace awareness. Considering otherintegrated applications that are used by intelligence ana-lysts, Socs would be able to provide awareness of teammembers and their activities as well as awareness of sub-jects of interest who can be found on the space, and arealso related to a currently viewed document or Web page.

Following spatial hypertext research, the space providedby Socs can serve as a means for communicating with oth-ers. This includes local or geographically distributed meet-ings as well as asynchronous communication. Due to easymanipulation, spatial structures are well suited for meet-ings. Changes can be seen and discussed immediately.This serves the emergent nature of many meeting situationswell.

4 Conclusion and Future WorkBased on the emergent nature of covert groups such

as terrorist organizations we propose spatial hypertext asa means for intelligence agents to support their reasoningand decision making. This solves some problems that oc-cur with explicit structures (e. g., nodes and links). Further,spatial hypertext is an appropriate medium for communica-tion and thus addresses an important aspect and core prob-lem for intelligence services that was pointed out by the9/11 report [10]. Due to their communicative capabilitiesand reduced cognitive overhead requirement (compared toformal structures), they are also an appropriate means forstructuring information collaboratively.

Various data sources, including those provided by SNAtools, can be used as input channels. By overcoming theborder of individual sources of information and permit-ting a common information space, rich structures becomepossible that can support the analysis and decision making

329

Figure 1: Screenshot of Socs (social space, web browser, wiki authors list, address book)

330

process.We propose the Socs prototype as a tool to support

knowledge workers in general and intelligence analysts inparticular. Socs provides the discussed features of a highlevel of integration into the user’s environment, varioustypes of awarenesses, support for emergent structures, andeffective communication. It provides a common informa-tion space across the borders of individual data sourcesthrough integrating services, such as the system-wide ad-dress book framework or the World Wide Web. To ourknowledge, Socs is the first application of its kind thattargets the field of intelligence analysis with a special fo-cus on social structures: It provides spatial hypertext tech-niques to author and modify social relationships and thuschanges the primary scope of existing spatial hypertext ap-plications (e. g., VKB [22] or Tinderbox [3]) from genericinformation pieces to social structure abstractions (see also[2]).

Socs is still at an early development state. We planto open Socs’s API to other applications as well as writemodules to connect to additional tools. This will increasethe potential for awareness and add further data sources.Furthermore, analysing applications used by intelligenceagents or for police investigations would be beneficial tohelp tailor Socs for their specific needs.

References[1] Claus Atzenbeck. WildDocs – Investigating Con-

struction of Metaphors in Office Work. PhD thesis,Aalborg University, Denmark, 7 2006.

[2] Claus Atzenbeck and Manolis Tzagarakis. Criteriafor social applications. In Vania Dimitrova, Mano-lis Tzagarakis, and Julita Vassileva, editors, SociUM:Adaptation and personalization in social systems:groups, teams, communities. Workshop proceedingsin conjunction with the 11th international conferenceon User Modeling, pages 45–49, 2007.

[3] Mark Bernstein. The Tinderbox Way. Eastgate Sys-tems, 2006.

[4] Uwe M. Borghoff and Johann H. Schlichter.Computer-Supported Cooperative Work. Introductionto Distributed Applications, chapter 3, pages 145–183. Springer, 2000.

[5] Susan L. Bryant, Andrea Forte, and Amy Bruck-man. Becoming Wikipedian: transformation of par-ticipation in a collaborative online encyclopedia. InGROUP ’05: Proceedings of the 2005 internationalACM SIGGROUP conference on Supporting groupwork, pages 1–10. ACM Press, 2005.

[6] Hsinchun Chen. Computer-assisted “dark web” web-site and forum collection, analysis and visualization.In Boaz Ganor, Katharina von Knop, and CarlosDuarte, editors, Hypermedia Seduction for TerroristRecruiting, volume 25 of NATO Science for Peaceand Security Series: Human and Societal Dynamics,pages 119–139. IOS Press, 2007.

[7] Douglas C. Engelbart. Augmenting human intellect:A conceptual framework. Summary Report AFOSR-3233, Standford Research Institute, 10 1962.

[8] Mark D. Gross and Ellen Yi-Luen Do. Ambiguous in-tentions: a paper-like interface for creative design. InUIST ’96: Proceedings of the 9th annual ACM sym-posium on User interface software and technology,pages 183–192. ACM, 1996.

[9] Frank G. Halasz. “Seven Issues” revisited. Keynoteaddress, Hypertext ’91 conference, San Antonio,Texas. WWW, 12 1991.

[10] Thomas H. Kean, Lee H. Hamilton, Richard Ben-Veniste, Bob Kerrey, Fred F. Fielding, John F.Lehman, Jamie S. Gorelick, Timothy J. Roemer,Slade Gorton, and James R. Thompson, editors. The9/11 Commission Report. Executive Summary. Nor-ton, 2004.

[11] Ilaria Liccardi, Hugh C. Davis, and Su White.CAWS: a wiki system to improve workspace aware-ness to advance effectiveness of co-authoring activ-ities. In CHI ’07 extended abstracts on Human fac-tors in computing systems, pages 2555–2560. ACM,2007.

[12] Catherine C. Marshall, Frank G. Halasz, Russell A.Rogers, and William C. Janssen. Aquanet: a hyper-text tool to hold your knowledge in place. In Pro-ceedings of the 3rd ACM Conference on Hypertext,pages 261–275. ACM Press, 1991.

[13] Catherine C. Marshall and Frank M. Shipman. Spa-tial hypertext: designing for change. Communica-tions of the ACM, 38(8):88–97, 8 1995.

[14] Catherine C. Marshall, Frank M. Shipman, andJames H. Coombs. VIKI: Spatial hypertext support-ing emergent structure. In Proceedings of the 1994ACM European Conference on Hypermedia Technol-ogy, pages 13–23. ACM Press, 1994.

[15] Nasrullah Memon, David Hicks, Henrik LegindLarsen, and Abdul Qadeer Khan Rajput. How inves-tigative data mining can help intelligence agencies inunderstanding and splitting terrorist networks. Stud-ies in Conflict & Terrorism, 2008.

331

[16] Nasrullah Memon and Henrik Legind Larsen. Struc-tural analysis and destabilizing terrorist networks. InProceedings of the 2006 International Conference onData Mining DMIN’06, pages 296–302, 2006.

[17] Kumiyo Nakakoji, Yasuhiro Yamamoto, ShingoTakada, and Brent N. Reeves. Two-dimensional spa-tial positioning as a means for reflection in design. InDIS ’00: Proceedings of the 3rd conference on De-signing interactive systems, pages 145–154. ACM,2000.

[18] Sylvie Noel and Jean-Marc Robert. How the web isused to support collaborative writing. Behaviour andInformation Technology, 22(18):245–262, 2003.

[19] Tim O’Reilly. What is Web 2.0? Design patterns andbusiness models for the next generation of software.WWW, 9 2005.

[20] Don Radlauer. Virtual communities as pathways toextremism. In Boaz Ganor, Katharina von Knop,and Carlos Duarte, editors, Hypermedia Seductionfor Terrorist Recruiting, volume 25 of NATO Sciencefor Peace and Security Series: Human and SocietalDynamics, pages 67–75. IOS Press, 2007.

[21] Yael Shahar. The internet as a tool for intelligenceand counter–terrorism. In Boaz Ganor, Katharina vonKnop, and Carlos Duarte, editors, Hypermedia Se-duction for Terrorist Recruiting, volume 25 of NATOScience for Peace and Security Series: Human andSocietal Dynamics, pages 140–153. IOS Press, 2007.

[22] Frank M. Shipman, Haowei Hsieh, Preetam Mal-oor, and J. Michael Moore. The Visual Knowledge

Builder: a second generation spatial hypertext. InProceedings of the 12th ACM Conference on Hyper-text and Hypermedia, pages 113–122. ACM Press,2001.

[23] Frank M. Shipman and Catherine C. Marshall. For-mality considered harmful: Experiences, emergingthemes, and directions on the use of formal represen-tations in interactive systems. Computer SupportedCooperative Work (CSCW), 8(4):333–352, 1999.

[24] Gabriel Weimann. Using the internet for terrorist re-cruitment and mobilization. In Boaz Ganor, Katha-rina von Knop, and Carlos Duarte, editors, Hyper-media Seduction for Terrorist Recruiting, volume 25of NATO Science for Peace and Security Series: Hu-man and Societal Dynamics, pages 47–58. IOS Press,2007.

[25] Yasuhiro Yamamoto, Kumiyo Nakakoji, and AtsushiAoki. Spatial hypertext for linear-information au-thoring: Interaction design and system developmentbased on the art design principle. In Proceedings ofthe 13th ACM Conference on Hypertext and Hyper-media, pages 35–44. ACM Press, 2002.

[26] Yael Yehoshua. Islamist websites as an integral partof jihad: A general overview. In Boaz Ganor, Katha-rina von Knop, and Carlos Duarte, editors, Hyper-media Seduction for Terrorist Recruiting, volume 25of NATO Science for Peace and Security Series: Hu-man and Societal Dynamics, pages 59–66. IOS Press,2007.

332