ComplAI
UI/UX
Raleigh, USA
National Security Agency | LAS
Duration:
4 Months
Team:
Soumya Batra
Leah Tatu
Gabrielle Thorpe
My Role:
As a key contributor, I collaborated with my team to design the UI and UX of ComplAI. I conducted user research to understand analysts' pain points, developed user flows and prototypes for intuitive navigation, and created a clean, functional interface for clarity in alerts and recommendations. I also worked closely with teammates to integrate research insights and technical inputs into the design.
LAS//Research Question
The Laboratory for Analytical Sciences (LAS) operates as a collaborative initiative between government, academic, and industry professionals. Focused on innovation, LAS develops advanced technologies that strengthen national security and enhance intelligence analysis capabilities.
How might the design of an interface use the affordances of AI to enable Target Network Analysts (TDNAs) to efficiently and knowledgeably analyze data as it moves through the global communication network?
Problem
Data analysts face significant challenges in maintaining compliance with ever-changing laws and policies. They struggle to keep up with the dynamic regulatory landscape, leading to cautious and overly conservative queries that hinder creative problem-solving. Furthermore, reliance on outdated tradecraft repositories exacerbates inefficiencies, making it difficult for analysts to access reliable and up-to-date resources. This creates a pressing need for a solution that empowers analysts to stay compliant while fostering innovation and efficiency in
their workflows.
Solution
The solution is ComplAI, an AI-powered tool that leverages data from multiple sources and advanced tradecraft capabilities to assist analysts in staying compliant. It provides real-time alerts, detects potential compliance issues, and recommends actionable solutions, enabling analysts to work efficiently while adhering to evolving regulations.
The process in phases
The Scenario
A commercially made drone fitted with an IED has targeted a coffee shop in Macondo, Oceania. The resulting bombing killed two people and injured others. The BGIA counterterrorism team must find data that tells the story of what happened before, during and after the bombing to share with the Leaders of Oceania.
You Can’t Do That: An analyst queries something the BGIA does not have authority for (a seasoned TDNA).
Sub RQ1
How might AI better support the query process? New AI powered ways of searching and finding.
Sub RQ2
How might data be visualized to reveal new patterns? Visualization systems.
Sub RQ3
How might TDNAs learn tradecraft by interacting with these tools? Informal educational systems.
User Interviews
When we first met the lead analysts, we were given a brief overview of their process. As we dug deeper into their workflow and challenges, our assumptions were completely overturned during the first user interview. A key insight was that compliance held different meanings depending on the user's experience level. Additionally, tasks we perceived as difficult were ones they deeply enjoyed, as they found thrill in the complexity. This taught us that while automation could enhance efficiency, it shouldn’t replace the aspects of their work they value most.
As newcomers to AI, benchmarking proved invaluable in exploring efficient ways to leverage its capabilities. A major challenge was confidentiality, as we couldn’t access the interfaces analysts used. Benchmarking provided a crucial starting point, helping us navigate these limitations effectively.
Focus Areas
These were the key phases of the analysts' processes, with our primary focus being on the "proceed with caution" phase. This phase was critical as it emphasized careful decision-making and adherence to compliance standards, ensuring analysts could navigate complex workflows without risking errors or breaches. Understanding this focus helped us design solutions that balanced precision and efficiency.
Introducing
Ideation
We conducted iterative ideation, using mappings and whiteboarding, to identify compliance tasks suitable for automation, narrowing down to two key directions. AI cards provided valuable insights, helping us explore practical and innovative AI-driven solutions tailored to analysts' needs.
The user journey map outlines critical touch points in the compliance workflow, shedding light on pain points and areas for potential optimization. It revealed that analysts value the complexity of their tasks while requiring AI support for efficiency and accuracy. One of the key insights was the diverse interpretations of compliance among users, driven by their
experience levels.
Challenges Faced:
-
Understanding and aligning with varying user perspectives on compliance.
-
Balancing automation without removing the aspects of work analysts enjoy.
-
Navigating confidentiality constraints, which limited our access to existing tools and workflows.
Persona
Michele, who has a bachelor’s degree in Zendian, has been with the network analyst team for around 7 months, shifting from her role as a language analyst after a diversity tour.
While lacking a background in global telecommunications, Michele's extensive career in intelligence has familiarized her with high-level terms, despite not being
technologically adept.
Initial Ideas
Initially, we envisioned ComplAI as a system-level integration designed to seamlessly integrate with the settings of an analyst’s work desktop. Functioning from the control center, it leverages the system’s capabilities to provide real-time compliance support. ComplAI can detect potential compliance risks, alert users to issues, and recommend actionable solutions, all while operating discreetly in the background to enhance efficiency without disrupting the analyst’s workflow.
Critiques with the Analysts
The idea of integrating ComplAI as a system-level solution brings challenges such as ensuring compatibility across diverse desktop environments, maintaining privacy and security for sensitive data, and managing resource utilization to avoid system slowdowns. One other great feedback was that they do multiple things at the same time and thus need something that really grabs attention. Scalability to adapt to varying organizational workflows and addressing user resistance to new tools are also critical hurdles. Additionally, finding the right balance between automation and preserving the aspects of work analysts value is essential to ensure user adoption and satisfaction.
Addressed Painpoints
Struggles to keep up with the changing laws and policies
Cannot think out of the box— thus queries tend to be overly cautious
Cannot rely upon outdated tradecraft repository and data
The UI
_gif.gif)
_gif.gif)
How to use ComplAI?
AI Model
This AI model integrates a Large Language Model (LLM) with a Retrieval Augmented Generation (RAG) system to provide accurate, context-specific responses by retrieving data from Cognitive Search and AI Search systems, which pull information from separate, structured data sources. By keeping the data from Cognitive Search and AI Search distinct, the model ensures confidentiality and prevents data mixing, making it ideal for handling sensitive or classified information while delivering precise results.
Learnings
The LAS project was a significant learning experience that honed my ability to adapt as a designer. Balancing user-centered design principles with complex system requirements pushed me to refine my problem-solving and collaboration skills. One of the key challenges was managing cross-disciplinary input from stakeholders while maintaining a cohesive design vision. Additionally, working with evolving project goals taught me the importance of flexibility and iterative thinking. This experience reinforced the value of clear communication, user empathy, and the ability to adapt to dynamic environments, which are crucial for delivering effective and impactful design solutions.
Wider Implication 1
Supports analyst query formation rather than automating query structure completely.
Wider Implication 2
The goal input initially into the system enables the AI to better tailor recommendations according to the analyst’s needs.
Wider Implication 3
The testing mode which is trained on real time law and policies updates, as well as tradecraft of senior analysts, provides a risk free environment to learn and explore.