David Justh
Executive-facing decision partner who galvanizes relationships and makes calls when playbooks don’t fit
⬦ Human Factors & Ergonomics
⬦ Emergent Strategy
⬦ Design Research
Field testing military AI-piloted drone agents
This page offers 5 mini-cases. Detailed presentations with visual artifacts are available for live shareouts. Cases follow the following format ➡️ ➡️ ➡️
What? Premise, context, questions, problems
Now what? Sample of methods to glimpse strategy, actions, intention, direction
So what? Points of pride, outcomes, impacts, value adds, decisions taken
Employer: Shield AI
Domains: Aviation, defense, startup
Project features: B2B, strategy, risk assessment, agile, lean, conflict resolution, design crits, persuasion, influence
Research types: Discovery, desk, experimental, guerrilla, foundational, empirical, formative
How to build “brilliant basics” in human-machine interfaces
Applying first principles to an agentic socio-technical system headed toward its 1.0 release
WHAT? Set in a semi-regulated space, this outlines how I prevailed upon colleagues unfamiliar with how foundational human factors concepts would aid their work in the design of a multi-modal joint-optimized cognitive system focused on ISR missions (intelligence, surveillance, reconnaissance). I addressed these concepts because several instruments were becoming degraded "by design," to the point of risking failure. This addresses the design of 2 such visual instruments — the primary flight display and telemetry instrumentation — both of which merge critical flight data into small screens. It also addresses innovations in audible alarm design.
NOW WHAT? I convened meetings to socialize the following topics upward and outward. Meaning, not just to designers, but to decision makers: VPs, directors, and PMs. Topics fundamentally relate to the interplay of human factors (emphasis on human limitations) and cognitive systems engineering (emphasis on leveraging human capabilities in a complex adaptive control system):
Signal detection
Jobs to be Done
Prior knowledge
Common ground
Color psychology
Emotional valence
Occupational stress
Task & work performance
Aesthetic usability effects
Human-AI function allocation
How people do and don’t learn
Working memory and cognitive load
SO WHAT? By bringing clarity to a sophisticated socio-technical system through broadened upward communication — which increasingly represents the nucleus of design research — I had successfully made my case. Relevant leadership welcomed the no-nonsense scientific approach and designs became more congruent, as they better conformed to military standards, thus satisfying pilot, customer, and government expectations.
Client: Nuance, now Microsoft Health
Domains: Medical tech, healthcare
Project features: Product strategy, risk assessment, workplace culture
Research types: Foundational, desk, formative, experience mapping
It’s cheaper to understand your users than to apologize to your shareholders
Saving millions of dollars through the methodical evaluation of nurses’ attitudes toward AI-powered speech technology
WHAT? As frontline workers, nurses face numerous challenges. How do they communicate with colleagues and patients? How do they record and share patient information? What are their workflows and priorities? By exploring such questions, this foundational investigation of occupational culture and technology addressed a client hypothesis that its high-touch AI-powered technologies (DAX, DMO) would find product-market fits for nurses in inpatient settings: e.g., ICU, medical-surgical, rehab.
NOW WHAT? From discovery to readout, representative samples of nurses shared their attitudes, values, and lived experiences in a study both operationally and intellectually complex. Research yielded remarkably consistent themes, thanks to several advanced techniques.
-
Literature review: A necessary starting point to survey existing research on the subject. While helpful for context, it wasn’t sufficiently relevant because of the greenfield nature of the investigation, hence making this project foundational research.
Ethnomethodological workplace study: Applied interpretative phenomenological analysis to:
Contextualize lived experiences: e.g., perspectives, attitudes, values, beliefs, behavioral norms
Balance outsider (client) and insider (nurses) perspectives (a.k.a., etic vs. emic)
Understand how participants constructed, ordered, and made sense of their social realities. In other words, the nurses’ Umwelt: the unique, subjective perceptual world that someone experiences and acts within, shaped solely by their specific sensory abilities and interpretations of their surroundings, rather than the totality of the objective environment
Facilitate theoretical development
Cognitive interviews: To access multiple memory pathways that enhance recall, reconstruct context, provide detailed descriptions, elicit emotions, detect loaded language, enhance social dynamics, and reduce interviewer bias
Why code and highlight interview data in the first place?
Codes disambiguate and transform phenomena into knowledge objects that animate discourse by capturing, organizing, re-coding, and filtering diverse data through successive, iterative, recursive coding rounds
Why use different code types? Matching code types gives systematic form and flow to the data, has de-biasing effects, and helps to avoid contested vision: how different social groups use distinct discursive practices to construct competing interpretations of the same events, shaping what counts as meaningful or true within their professional worldview.
Highlight: To make specific phenomena in a complex perceptual field salient by marking them in some fashion
SO WHAT? In the end, a detailed experience map visually summarized 800+ coding instances, 40 core findings, and 24 distinct insights. This showeed that 91% of participants were disinterested or hostile to the use of such technology, while elucidating key reasons to not deviate from current workflows and practices (which relate to longstanding ironies inherent in automation technologies). Results were so consistent that this engagement revealed:
These tools represented solutions in search of problems
Technology isn’t the answer to every problem, which, in turn, led Nuance to abandon the investment, saving millions of dollars and thousands of development hours while identifying narrower areas of opportunity to consider pursuing. Put another way — and not without irony — it would have been iatrogenic (i.e., care-induced injury) for a medical technology company to have disregarded these unambiguous findings by pursuing such “solutions.”
Client: Viator / Tripadvisor
Domains: Travel, SaaS, e-commerce
Project features: B2C, product strategy, segmentation, information architecture, content design, expert review,
Research types: Pathfinding, formative
How to reduce marketing costs by creating a more targeted SEO/SEM strategy
WHAT? One of the world’s largest travel excursion sites needed to know: What motivates people to travel? Which information sources do they trust? Which devices do they prefer? What kinds of emotional bonds are formed? And how can the company reduce the high number of customers who land on product pages “sideways,” in particular from Google, to which Viator pays high sums for SEO & SEM. This issue represented significant problem-value metrics with a few root causes.
NOW WHAT? With these research questions in mind — and in a bid to make the site more navigable, relatable, and sticky — I first conducted an expert review of the company’s booking processes, which was an inexpensive way to identify IA governance issues amongst a generally muddled content design. Next, I performed a meta-analysis of 23 primary research reports involving 116 participants to systematically assess multiple dimensions surrounding travel planning.
-
Narrative review: Appropriate to assess studies that have used diverse methodologies or have examined different conceptualizations, constructs, and relationships. It’s a particularly useful way to link together studies on different topics for reinterpretation or interconnection in order to develop or evaluate new theories.
Meta-synthesis: Appropriate when the review aims to integrate qualitative research with an aim to synthesize studies on a topic to locate key themes and concepts that provide novel or more powerful explanations for the phenomenon under review. This involved reflexive thematic analysis, with inductive (grounded theory) and deductive phases.
Successive rounds of affinity mapping offered the most flexibility and economy: sample
By moving from the abstract to the concrete, the constructed themes substantively addressed issues related to:
Brand loyalty
6 influencer typologies
Threats to the company
Content design and IA governance
10 new customer archetypes through segmentation
Traveler priorities, tactics, goals, obstacles, emotions, and technologies
Biases related to participants, methodology, and, naturally, the lead researcher (myself)
Theories of value, motivation, trust, and customer identity (e.g., self-determination theory)
SO WHAT? Ultimately, the effort revealed opportunities that speak to how the company can alleviate high SEO/SEM costs through factors related to personalization and the psychology behind how people search for information. Actionable recommendations were added to the roadmap and global navigation, used by 35 million monthly users. Prime examples include:
Content funnels organized by interest
Content funnels organized by geography
A simplified product taxonomy
Enhanced representations of traveler segments
Client: GoodRx
Domains: Healthcare, e-commerce
Project features: B2C, product design
Research types: Exploratory, formative, summative
Defining a wholesale reconsideration of the discount prescription giant’s core service
WHAT? In a bid to increase customer retention and lifetime value, GoodRx sought to expand beyond its core prescription discount service by moving into a more holistic space built around a “Managing Illness” theme. In this instance, a bespoke usability study doubled as discovery in how it additionally explored a sizable investment opportunity: the wellness management arena.
NOW WHAT? This hybrid approach utilized a multi-stage protocol that leveraged competitive landscape analyses, customized heuristic evaluations (inspired by this), and a usability study with participants who:
Represented diverse sampling: They managed 11 chronic health conditions and used 16 other wellness resources for self-care
Completed randomized tasks during IDIs to mitigate demand characteristics and, ultimately, better discover how they attuned to value
Gap analyses are wonderful in that, among other things, they force an honest look at reality, make misalignments visible, identify performance shortfalls, and translate abstract goals into specific, solvable problems. In this case, such an analysis revealed:
How GoodRx does and doesn’t help numerous customer segments manage certain illnesses
Large differences in customer mental models, which variably formed against:
The crowded field of comparable wellness resources on the market
The United States’ opaque healthcare system
Several dozen findings were synthesized into 7 core themes
SO WHAT? This primary research introduced to GoodRx the first critical assessment of emotional tolls and disabilities associated with chronic illnesses, as well as the evolving self-care strategies that people form as a result. GoodRX accepted the study’s insights, which it used to inform design recommendations about how to rethink not only the existing content funnels in question, but even a wholesale reconsideration of its core service.
Employer: Shield AI
Domains: Aviation, defense, startup
Project features: B2B, management consulting, organizational ergonomics, risk assessment, strategy, resilience engineering, service design, change management, agile, lean, conflict resolution, persuasion, influence
Research types: Discovery, guerrilla, foundational, empirical, formative
The “soft” stuff is the hard stuff: Product innovation through organizational innovation
Designing stakeholder engagement to outmaneuver complexity, reduce organizational brittleness, and give executives a reason to say “yes” to a $9M test laboratory
⁉️ George Bernard Shaw might’ve put it best: “The single biggest problem in communication is the illusion that it has taken place.” To this penetrating ontological observation I would add: “And it takes a certain forbearance to unriddle, then make sense of, what did and didn’t transpire, and from whose perspectives.”
WHAT? One of the biggest challenges working in product is getting people to hear you, listen to you, and believe you. To this end, researchers twist themselves into 🥨 trying to serve as credible actors to gain influence in organizations that are often structured to deflect interpretation and understanding, especially in a startup environment, where complexities arise in parallel with growth. Yet while it’s important for ICs to recognize they’re not the organization’s top priority, sometimes the squeaky wheel gets the grease.
With this familiar context, this case describes an initiative to expose veiled warnings at the company’s first autonomous drone flight test, which amounted to a early warning call for my teams to reconsider (1) the system under test, and (2) what they thought they knew about what they thought they were learning. Software is, among other things, a coordination problem, and the information asymmetry I detected was actively undermining the social relationships necessary to support it. I concluded this needed to be openly acknowledged and sorted through.
THE PLOT THICKENS: More crucially, this flight test represented a thorny hermeneutic event: a transformative cycle of questioning, understanding, and reinterpretation of meaning within a specific situation or experience. Moreover, this cycle bore dimensions of ethnographic noticing, knowledge visibility, organizational communication, power relationships, and social learning, which are some of theflows that often represent the vectors that drive invention and innovation (versus “organizational culture,” a catch-all term that’s typically ill-defined, which tends to represent culture as deterministic).
Ultimately, this is an expository account of how I tackled an organization’s disjointed responses to gather more information before resolving conflicts of perception, communication, and expectations. I succeeded in doing this:
In an unorthodox, emergent way
Without the benefit of consulting a manager or skip-level
In a turbulent business environment and information landscape
To resist a system-level drift toward incoherence — including the self-centeredness that comes with territoriality — in order to bolster organizational decision intelligence
NOW WHAT? The show must go on, as research strategists functionally serve (1) as the glue on a team or project, and (2) as business therapists. They also tend to see things that others don't and are motivated by impact over recognition. While embracing the prefigurative notion that the best defense is a strong offense — and while recognizing that learning in systems with time delays doesn’t come naturally to some — I scheduled a large cross-functional share-out to shine a light on the flow and recombination of ideas, practices, lucky errors, and chance insights. In so doing, I formed a comprehensive view of the veiled warnings by weaving together elements of numerous fields for maximum impact:
-
Backcasting
Lean Thinking
Risk mitigation
Boundary objects
Incident response
Macroergonomics
Emergent strategy
Complexity science
Systems Convening
Three-body problems
Second-order cybernetics
Research democratization
Second-order cybernetics
Philosophy of science (e.g., post-positivism)
Non-verbal communication (i.e., paralanguage)
Underutilized rhetorical persuasion techniques
The subtle, yet inescapable presence of biases in research theater
Resilience and cognitive systems engineering, e.g., graceful extensibility, competence envelopes, and…
The Law of Stretched Systems: New technologies are invariably exploited to achieve new intensities and tempos of activity, with development competing to outpace, and take precedence over, nature. But nature cannot be fooled. When you fail to accept the reality of cumulative debts — which includes debts of morale — organizations set themselves up to be taken advantage of by:
Newfound complexities
Leading their systems (technological, informational, communications, workload) to lose the ability to conduct even their most foundational tasks, which degrades basic organizational coherence
The reality is that full control is a fantasy. There are no silver bullets.
SO WHAT? In the end, I effected a dynamic compromise, which led to the creation of a $9,000,000 testing laboratory. This, in turn
Reduced cycle times by 56%
Reduced product decision brittleness
Helped me gain cross-functional allies
Afforded adaptive capacity to the organization
Exposed a problem the organization didn’t know it had
Enabled engineers and their managers to develop solutions more reliably
Improved organizational communication by connecting knowledge across teams
Successfully launched the company’s first research service and knowledge infrastructure
Extended my teams’ abilities to mobilize and maneuver when surprises occurred when adaptive behaviors were saturated
Socialized more rigorous research on recurring bases, thus setting a foundation for more reliable and repeatable learning and, hence, decision-making
☆☆☆ Most crucially, this effort qualitatively enhanced product sensibility, coherence, and trust across pilot, product, and engineering teams. ☆☆☆
➡️ “Complex systems tend to produce complex responses (not solutions) to problems.” — John Gall ⬅️