ASCEND Cognitive Task Analysis (CTA) Study - Info and Reading Options

The Internet Archive:

IARPA’s Reimagining Security with Cyberpsychology-Informed Network Defenses (ReSCIND) program aims to enhance cybersecurity by leveraging attackers’ human limitations. It focuses on developing novel defenses that exploit decision-making biases and cognitive vulnerabilities. These defenses help rebalance the asymmetry of cyber defense, imposing penalties on attackers and thwarting their efforts. IARPA’s ReSCIND program funds the ASCEND (Adaptive Security through Cognitive Exploitation for Defense) project. Led by SRI, the multidisciplinary team behind ASCEND includes researchers and experts from the Florida Institute for Human and Machine Cognition, George Mason University, RAD Science Solution, SimSpace, Two Six Technologies, the University of Florida, and Virtual Reality Medical Center, as well as independent consultants, will create a game-changing, cyberpsychology-informed cyber defense system (see http://ascend.sri.com for more information). Experiment Objectives To jumpstart and accelerate the implementation of a Model of Inferred Cognition and Knowledge (MIMICK), ASCEND’s Phase 1 technical work included Cognitive Task Analysis (CTA) Human Subjects Research (HSR) to identify attackers’ cognitive processes, decision-making factors, and actions, grounded in the Capture the Flag (CTF) task context that has been a focus of the ReSCIND Phase 1 investments. CTA is a systematic approach to achieving an understanding of knowledge, decision-making, and planning processes. Founded in the idiographic and ethnographic research traditions, it relies on domain analysis, subject matter expert (SME) input, and semi-structured interviews to produce actionable insights.  We down-selected from a broad range of CTA methods to tailor our design for maximum information value in the time and budget scope available. We selected the following methods for inclusion in this protocol: Task Diagrams, Goal-Directed Task Analysis, FCM Interviews, Recent Case Walkthroughs, and CTF tasks with concurrent think-aloud verbalizations.    Experiment Description Five SMEs each participated in two hybrid remote and in-person sessions over two nonconsecutive days (4.5 hours total for each participant), with screen and voice recordings collected via Zoom (not made publicly available due to privacy concerns). This setup ensured the capture of rich, detailed, high-quality data concerning expert cognition essential for informing MIMICK design and functionality.  On the first day, lasting 2.5 hours, activities include check-in, signing consent forms, completing demographic and professional expertise questionnaires, and engaging in Task Diagram development, Goal-Directed Task Analysis, and FCM interviews, with breaks provided as needed. The second day, lasting 2 hours, involves a review of a recent case, participating in two CTF challenge tasks with concurrent verbalization, and a debriefing session, including time for breaks. The protocol collects comprehensive participant data while adhering to Institutional Review Board (IRB) approval guidelines.  The CTA comprises multiple methods, including:  Task Diagrams. Task Diagramming aims to elicit a broad overview of how an SME thinks about the overall structure and flow of a task, in this case, a CTF. The interviewer asks the SME to decompose the task into an ordered sequence of three to six subtasks.  Goal-Directed Task Analysis. The purpose of the Goal-Directed Task Analysis is to unpack the high-level structure provided by the Task Diagram into a more detailed representation. FCM Interviews. An FCM is a graph-based representation of subjective causal beliefs: how people believe a change (i.e., an increase/decrease or a change between on and off states) in one concept impacts another. The nodes are the concepts, and the edges are the causal relationships between them. Interviews first seek to define core concepts and then identify associated concepts with relationship types. This iterates outward from the associated concepts, asking "what drives" each new concept. MentalModeler is a common software tool for constructing graphs during interviews and is used here.  Recent Case Walkthrough. The purpose of the Recent Case Walkthrough is to elicit from the SME a brief yet detailed accounting of how they handled a recent specific CTF. It will reveal what is most salient from their episodic memories of that event.  CTF with Verbal Protocol. The purpose of the CTF with concurrent verbal protocol is to understand what people think about when performing the CTF task. Participants speak only about the things they naturally think about. The verbal protocol data will provide a way to compare decision processes that participants actually engage in when performing a task versus how they represent and think about the task as elicited through the structured interviews.  Each participant completed the treatment version of two CTF challenges while concurrently thinking aloud. Participants were pseudo-randomly assigned to challenges across the five bias types (described in the subsequent section) so that each participant experienced two different biases, and so all bias types and versions were sampled across participants. The CTA study used challenges and surveys from the ECSC and EkoParty experiments.   The following cognitive vulnerabilities (CogVulns) were examined in the CTA CTFs: (Loss) Loss Aversion: The tendency for people to strongly prefer avoiding losses over acquiring gains (value functions that relate subjective to objective losses are steeper than value functions that relate subjective to objective gains).  (Rep) Representativeness: The tendency for people to judge the probability or frequency of a hypothesis by considering how much the hypothesis resembles available data.  (Anch) Anchoring: The tendency to rely too heavily on, or overly restrict one’s attention to, one trait or piece of information when making judgments. The information in question can be relevant or irrelevant to the target decision, as well as numerical or non-numerical. Includes focalism or the focusing illusion.  (Cult) Socio-Cultural: The tendency to interpret and judge phenomena in terms of the distinctive values, beliefs, and other characteristics of the society or community to which one belongs. This sometimes leads people to form opinions and make decisions about others in advance of any actual experience with them.   (Conf) Confirmation: The tendency to search for or interpret information in a way that confirms one’s preconceptions.  From each of the five SME participants, we have approximately 4.5 hours of recorded Zoom video (not made publicly available due to privacy concerns) and associated transcripts, semi-structured interview content from the Task Diagramming, Goal-Directed Task Analysis, Fuzzy Cognitive Mapping, and Recent Case Walkthrough activities, and the log files generated by SimSpace from the two CTFs they each completed.  Experimental Results  Analysis of the CTA data includes manual summarization and documentation of the knowledge, strategies, and decision-making processes reported by participants during the interviews. We also use the data to compare and contrast participant mental model descriptions of how they think about CTFs (TD, GDTA, and FCM tasks) to how they actually solve CTFs (RCW and CTF tasks), as well as comparison across participants to assess individual differences.  Separate analytical development (e.g., CTA analysis, FCM Interviews, workflow analysis) showed the potential of these methods to complement one another in representing attacker knowledge, modeling behavioral evolution, and inferring intent. FCMs proved effective for capturing and aggregating attacker knowledge and beliefs from heterogeneous data sources. Dynamic Behavior Embeddings (DBEs), scoped for future implementation, were defined to capture temporal patterns of attacker behavior and enable intent trajectory analysis. Information Foraging Theory (IFT) provided an interpretable framework for modeling attacker decision-making patterns, adapted from prior work in information-seeking behaviors. Collectively, these analytical explorations underscored the value of integrating knowledge, behavior, and decision-making to expose cognitive and tactical vulnerabilities.

Search for “ASCEND Cognitive Task Analysis (CTA) Study” downloads:

Visit our Downloads Search page to see if downloads are available.

Find “ASCEND Cognitive Task Analysis (CTA) Study” in Libraries Near You:

Read or borrow “ASCEND Cognitive Task Analysis (CTA) Study” from your local library.

• The WorldCat Libraries Catalog: Find a copy of “ASCEND Cognitive Task Analysis (CTA) Study” at a library near you.