AI-002·AI Track·Intermediate–Advanced·1–8 hrs saved

Stop Guessing Faster — Diagnose the Defect from Evidence, Not the First Plausible Theory

An AI-assisted workflow that turns scattered logs, stack traces, and bug reports into ranked hypotheses, controlled experiments, and a root cause the evidence actually supports.

3Phases

13Prompts

1–8Hours saved

8Deliverables

Executive Brief

Your Challenge

The visible symptom almost never sits where the actual cause lives. A user reports a page that won't load; the real problem is an expired token, a malformed record, a race condition, or an API response that changed last Tuesday. So you burn hours searching logs, inspecting code, reproducing conditions, and testing theories — and the intermittent failure vanishes the moment you try to catch it. You end up with "something failed" instead of "this specific condition caused this execution path to fail, and the evidence supports it."

Common Obstacles

Diagnosis goes slow and inconsistent for predictable reasons: bug reports lack reproducible steps, logs are incomplete and scattered across systems, and error messages mislead. Then human habits make it worse — teams jump straight to code changes, latch onto the first plausible explanation, blame the most recent deployment without evidence, and treat correlation as proof of causation. AI makes a tempting shortcut here, because it will happily present a plausible theory with far more confidence than the evidence supports.

The ABME Approach

This workflow enforces disciplined diagnosis in order: normalize the defect statement, separate facts from assumptions, build a timestamped timeline, reconstruct the execution path, and find the earliest divergence — not just the final exception. Only then do you generate a ranked hypothesis table where every entry carries supporting evidence, contradicting evidence, and the safest test to confirm or rule it out. You design the smallest experiment that separates competing explanations, sanitize every input before it reaches an AI platform, and state diagnostic confidence explicitly. The output is an investigation aid that ends in a demonstrable root cause, a regression test, and observability that will catch a recurrence.

Insight Summary

Effective debugging is not guessing faster. It is reducing uncertainty systematically until the failure mechanism can be demonstrated and corrected.

phase-1

The earliest divergence between expected and actual behavior is usually more useful than the final error — a timeout, null reference, or HTTP 500 is often the last result of an earlier failure.

phase-1

Separate symptom from root cause before you touch code. The exception type tells you what broke; only the mechanism tells you why the system entered the failing state.

phase-2

A good hypothesis predicts evidence and can be disproven. 'The database is broken' is a shrug; 'requests fail when the order references a soft-deleted promotion the repository filters out' is testable.

phase-2

Correlation is not causation, and a recent deployment is a candidate cause, not proof. Timeline alignment earns a hypothesis a row in the table, not a verdict.

phase-3

A fix is not verified because the exception disappeared or one request succeeded — intermittent defects hide. Prove the original reproduction now behaves correctly and that monitoring would catch a recurrence.

tactical

Never weaken security to test a hypothesis. Disabling certificate validation, turning off authorization, or logging raw auth headers trades one defect for a worse one.

The Journey

Three phases; each lists the tools you'll use there.

Normalize the Defect and Build the Evidence Base

Gather evidence, normalize the defect statement, build a timestamped timeline, and separate symptom from cause before proposing any theory.

Collect the defect report, logs, traces, code, and change history using the prerequisites list
Normalize the defect using Given / When / Then / Actual / Impact
Separate confirmed facts, reported observations, assumptions, and unknowns
Build a timeline with exact timestamps and identify first failure and last known success
Distinguish the primary failure from secondary and cascading errors

Prerequisites Checklist Defect Intake Template

Generate and Rank Hypotheses with AI

Use the prompt pack to reconstruct the execution path, produce a ranked hypothesis table, and design the minimal experiments that separate competing causes.

Run the primary prompt with sanitized evidence to produce a normalized diagnosis
Apply targeted follow-up prompts for stack traces, logs, intermittent, data-dependent, and regression cases
Build a hypothesis table with confidence, supporting and contradicting evidence, and a safe validation test
Design a minimal experiment matrix that changes one variable at a time
Actively seek disconfirming evidence to counter confirmation bias

Defect Diagnosis Prompt Pack Hypothesis Table Diagnostic Experiment Matrix

Confirm the Cause, Correct, and Verify

Demonstrate the root cause, apply the safest correction, and prove it with regression tests, deployment safeguards, and recurrence monitoring.

Write a root-cause statement covering trigger, mechanism, impact, and control gaps
Separate immediate containment, root-cause correction, and prevention actions
Verify the fix against the original reproduction and neighboring behavior
Define regression tests and a deployment and rollback plan
Run the validation checklist before accepting the diagnosis

Root-Cause Statement Template Validation Checklist Diagnostic Test Plan

What's Inside the Execution Layer

Numbered deliverables grouped by phase. Membership unlocks every tool.

2. PHASE 1Templateprotected

Defect Intake Template

A structured intake form that normalizes a defect report into consistent, analyzable fields.

Use this to

Capture a defect consistently across teams
Separate expected from actual behavior with exact timing
Feed a clean, complete record into the primary prompt

Included with membership — $20/mo

Summary

A concise description of the problem.

[...]

Expected Behavior

What should happen?

[...]

Actual Behavior

What happened instead?

[...]

Impact

Who or what is affected?

[...]

Frequency

Choose one: Always / Often / Intermittent / Once / Unknown

[...]

First Known Occurrence

Use an exact date and time where possible.

[...]

Last Known Success

Use an exact date and time where possible.

[...]

Environment

Specify: Production / Staging / Test / Development / Specific region / Specific host / Specific browser or device

[...]

Reproduction Steps

Numbered steps with exact inputs.

[...]

Evidence

Attach or reference: Error / Screenshot / Logs / Trace / Request ID / Record ID / Deployment ID

[...]

Recent Changes

Code, configuration, data, dependencies, or infrastructure.

[...]

Workaround

Any known method for avoiding or recovering from the defect.

[...]

3. PHASE 2Prompt Packprotected

Defect Diagnosis Prompt Pack

One primary diagnostic prompt plus twelve targeted follow-ups that drive evidence-based diagnosis without proposing premature fixes.

Use this to

Produce a normalized, evidence-based diagnosis
Analyze stack traces, logs, and successful-versus-failed requests
Design reproduction plans, experiment matrices, and instrumentation

Included with membership — $20/mo

Primary Prompt

Start here with the full, sanitized evidence set.

You are a senior software diagnostician investigating a software defect.

I will provide some or all of the following:
• Defect report
• Expected behavior
• Actual behavior
• Reproduction steps
• Business impact
• Timeline
• Logs
• Stack traces
• Metrics
• Distributed traces
• Relevant code
• Tests
• Recent code changes
• Deployment history
• Configuration changes
• Database information
• Infrastructure information
• Dependency information
• Environmental differences
• Prior troubleshooting attempts

Your task is to diagnose the defect using evidence-based reasoning.

Do not immediately propose a code fix.

First:

1. Normalize the defect statement using:
• Given
• When
• Then
• Actual result
• Impact

2. Separate:
• Confirmed facts
• Reported observations
• Assumptions
• Unknowns

3. Build a timeline using exact timestamps where available.

4. Identify:
• First known failure
• Last known success
• Relevant deployments
• Configuration changes
• Dependency changes
• Infrastructure events
• Data changes

5. Determine whether the failure is:
• Consistent
• Intermittent
• Data-dependent
• User-dependent
• Environment-dependent
• Time-dependent
• Load-dependent
• Sequence-dependent
• Permission-dependent

6. Reconstruct the likely execution path.

7. Identify the earliest known point at which actual behavior diverges from expected behavior.

8. Distinguish the primary failure from secondary or cascading errors.

9. Identify missing evidence that materially limits the diagnosis.

Then create a hypothesis table.

For each hypothesis, include:
• Hypothesis ID
• Proposed cause
• Confidence
• Supporting evidence
• Contradicting evidence
• Evidence still needed
• Safest validation test
• Expected result if true
• Expected result if false
• Operational risk of testing

Rank hypotheses by:
1. Evidence strength
2. Ability to explain all observed symptoms
3. Consistency with the timeline
4. Reproducibility
5. Probability
6. Testability

Review the defect across these areas:
• Application logic
• Input validation
• Null handling
• State management
• Authentication
• Authorization
• Data integrity
• Database behavior
• Transactions
• Concurrency
• Race conditions
• Caching
• Serialization
• Time zones
• Locale
• Configuration
• Feature flags
• Dependency behavior
• External API behavior
• Network behavior
• Resource exhaustion
• Timeouts
• Retries
• Queue behavior
• Deployment order
• Version compatibility
• Environmental differences
• Observability gaps

Requirements:
• Do not claim a root cause without sufficient evidence.
• Do not confuse correlation with causation.
• Do not invent log entries or code behavior.
• Cite the exact evidence supporting each conclusion.
• Clearly mark assumptions.
• Prefer experiments that distinguish multiple hypotheses.
• Recommend the least invasive diagnostic step first.
• Do not recommend disabling authentication, authorization, TLS, logging, or other security controls.
• Do not recommend broad production changes merely to gather evidence.
• Do not expose sensitive data.
• Identify when production reproduction would be unsafe.
• Consider whether prior troubleshooting changed the original conditions.
• Keep unrelated defects separate.
• Identify whether the issue may have multiple contributing causes.

After the analysis, provide:
1. Executive diagnostic summary.
2. Normalized defect statement.
3. Evidence inventory.
4. Timeline.
5. Execution-path analysis.
6. Ranked hypotheses.
7. Recommended diagnostic experiments.
8. Likely root cause, if supported.
9. Contributing factors.
10. Recommended correction.
11. Regression-test plan.
12. Deployment and rollback plan.
13. Observability improvements.
14. Remaining uncertainty.
15. Final diagnostic confidence.

Analyze a Stack Trace

When a stack trace is the primary evidence.

Analyze this stack trace in the context of the provided code.

Identify:
• The exception type
• The first application-owned frame
• The likely failing operation
• Primary versus wrapper exceptions
• Whether the stack trace is complete
• Missing context
• Candidate root causes
• The next diagnostic step

Do not assume the top frame is the root cause.

Analyze Application Logs

When correlating an ordered sequence of log events.

Analyze these logs as an ordered event sequence.

Identify:
• Correlation IDs
• Request boundaries
• First abnormal event
• Primary failure
• Secondary failures
• Retries
• Timeouts
• State transitions
• Missing expected events
• Timestamp anomalies
• Conflicting evidence

Separate facts from interpretations.

Compare Successful and Failed Requests

When you have both a passing and a failing execution to contrast.

Compare the successful and failed executions.

Create a difference table covering:
• Input
• User identity
• Permissions
• Data state
• Configuration
• Application version
• Dependency version
• Host
• Region
• Timing
• Request sequence
• Feature flags
• Response
• Logs
• External calls

Rank the differences most likely to explain the failure.

Diagnose an Intermittent Defect

When failures appear and disappear inconsistently.

Analyze this as an intermittent defect.

Focus on:
• Race conditions
• Timing
• Shared state
• Caching
• Load
• Resource exhaustion
• Retry timing
• Network instability
• Event ordering
• Stale reads
• Duplicate processing
• Clock differences

Recommend instrumentation that minimally changes timing.

Diagnose a Post-Deployment Regression

When the failure began after a release.

Compare the last known successful version with the first failing version.

Identify:
• Behavioral changes
• Dependency changes
• Configuration changes
• Database changes
• Deployment-order risks
• Compatibility changes
• Feature-flag changes
• Infrastructure changes

Do not assume the deployment caused the defect merely because the dates align.

Diagnose a Data-Dependent Failure

When only some records or inputs trigger the failure.

Identify which data characteristics distinguish failing records from successful records.

Consider:
• Nulls
• Length
• Type
• Encoding
• Duplicates
• Date values
• Time zones
• Invalid references
• Historical format
• Version
• Tenant
• Ownership
• Record state

Propose a minimized synthetic test case that reproduces the condition without exposing production data.

Diagnose a Permission Failure

When authentication or authorization is suspected.

Trace the identity used at each stage.

Document:
• User identity
• Service identity
• Delegated identity
• Token audience
• Roles
• Scopes
• Resource permissions
• Impersonation
• Credential delegation
• Tenant context

Identify the exact authorization boundary that fails.

Diagnose a Performance Defect

When the defect is latency, throughput, CPU, memory, or resource related.

Analyze the defect as a latency, throughput, CPU, memory, or resource problem.

Identify:
• Baseline
• Regression point
• Slowest phase
• External-call duration
• Query count
• Locking
• Queue depth
• Memory growth
• Garbage collection
• Thread or connection exhaustion
• Scaling behavior

Recommend measurement before optimization.

Create a Reproduction Plan

When you need a reliable, safe way to reproduce the failure.

Create the smallest safe reproduction plan.

Include:
• Required environment
• Required data
• Preconditions
• Exact steps
• Expected result
• Actual failure signal
• Logging to enable
• Cleanup
• Safety restrictions
• Success and failure criteria

Create a Minimal Experiment Matrix

When separating competing hypotheses through controlled tests.

Create a controlled experiment matrix that changes one variable at a time.

For each experiment, include:
• Variable changed
• Control condition
• Test condition
• Expected result by hypothesis
• Risk
• Required evidence
• Interpretation

Design Diagnostic Instrumentation

When you need better observability to close evidence gaps.

Recommend temporary and permanent instrumentation.

Include:
• Structured log events
• Correlation IDs
• Trace spans
• Metrics
• Timing measurements
• State-transition events
• Error classification
• Data-safe diagnostic fields
• Sampling guidance

Do not log credentials, tokens, or unnecessary personal data.

Verify a Proposed Fix

Before accepting a fix as complete.

Evaluate whether the proposed fix addresses the demonstrated root cause.

Identify:
• Which evidence it addresses
• Which symptoms it should remove
• Risks introduced
• Unaffected contributing factors
• Required regression tests
• Rollback considerations
• Monitoring needed after deployment

4. PHASE 2Matrixprotected

Hypothesis Table

A structured table that forces every candidate cause to carry its supporting and contradicting evidence and a safe way to confirm or rule it out.

Use this to

Track competing hypotheses side by side
Record contradicting evidence, not just confirming logs
Rank causes by evidence strength and testability

Included with membership — $20/mo

Reference rows from the blueprint — downloads ship as an empty skeleton

ID	Hypothesis	Confidence	Supporting Evidence	Contradicting Evidence	Validation
H1	Repository returns null for some codes	High	promotion dereferenced without check; failures limited to some codes	Log says promotion applied	Test failing code directly against repository
H2	Promotion is deleted or expires between validation and order creation	Moderate	Intermittent and code may perform multiple lookups	No timing evidence	Trace both reads and promotion version
H3	Code matching is case-sensitive in one component	Moderate	Some codes fail; validation may differ	No failing input shown	Compare exact code values and query behavior
H4	request is null	Low	Would cause a null reference	Log contains request-derived fields	Ruled out by successful request logging
H5	PromotionCode is null	Low	Could influence lookup	Failure occurs on promotion orders	Add input snapshot excluding sensitive data

RubricRank hypotheses by evidence strength, ability to explain all observed symptoms, consistency with the timeline, reproducibility, probability, and testability. For each hypothesis also capture: evidence still needed, expected result if true, expected result if false, and operational risk of testing.

5. PHASE 2Matrixprotected

Diagnostic Experiment Matrix

A controlled experiment matrix that changes one variable at a time and predicts the result under each hypothesis.

Use this to

Design tests that separate competing causes
Predict each hypothesis's outcome before running
Keep experiments reversible and low-risk

Included with membership — $20/mo

Reference rows from the blueprint — downloads ship as an empty skeleton

Experiment	H1 Prediction	H2 Prediction
Submit known active code	Succeeds	Succeeds
Submit expired code	Repository returns null	Failure only if expiration timing aligns
Normalize code casing	Previously failing mixed-case code succeeds	No change
Perform one atomic lookup	No change if lookup consistently null	Removes timing-dependent failures

RubricA strong experiment changes one important variable, has a control, predicts results for each hypothesis, produces objective evidence, is reversible, avoids production risk, uses representative conditions, and does not introduce excessive instrumentation. Column set (H1/H2) mirrors the doc's example; extend one prediction column per competing hypothesis.

6. PHASE 3Templateprotected

Root-Cause Statement Template

A labeled structure for writing a root-cause statement that captures the trigger, mechanism, impact, and why controls failed.

Use this to

Write a defensible root-cause statement
Explain the failure mechanism, not just the exception
Document why controls did not prevent or detect the defect

Included with membership — $20/mo

Triggering condition

The condition that causes the defect to occur.

[...]

Defective behavior

What the system does incorrectly.

[...]

Failure mechanism

How the defective behavior produces the failure.

[...]

User or system impact

The effect on users or the system.

[...]

Why controls did not prevent or detect it

The validation, monitoring, or process gaps involved. Example: Orders containing expired promotion codes failed because the order service performed a second repository lookup that excluded expired records and then dereferenced the null result. Input validation did not convert this condition into a business error, and monitoring grouped the resulting null-reference exceptions under a generic 500 alert.

[...]

🔒 The full execution layer — every checklist, matrix, and the prompt pack — is included with ABME membership.

Unlock Full Blueprint

Full Playbook

Overviewpublic

Software defects are often difficult to diagnose because the visible symptom occurs far from the actual cause.

A user may report that a page failed to load, while the underlying issue is:

An expired access token
A malformed database record
A race condition
A dependency timeout
An incorrect deployment variable
A caching inconsistency
A serialization mismatch
A missing permission
A recently changed API response
An unhandled null value
A resource limit
A data migration problem

Traditional troubleshooting can consume hours while engineers search logs, inspect code, reproduce conditions, and test theories.

This workflow uses AI to organize defect evidence, reconstruct the execution path, generate ranked hypotheses, identify missing diagnostic data, and recommend the smallest safe set of tests needed to isolate the root cause.

The workflow emphasizes disciplined diagnosis rather than speculative code changes.

The goal is to move from:

“Something failed”

to:

“This specific condition caused this execution path to fail, and the evidence supports the conclusion.”

AI should assist with reasoning, evidence correlation, and test design. It should not claim a root cause merely because one explanation sounds plausible.

Business Problempublic

Defect investigation frequently becomes slow and inconsistent because:

Bug reports lack reproducible steps.
Logs are incomplete or spread across systems.
Error messages are misleading.
Teams jump directly to code changes.
Engineers focus on the first plausible explanation.
Recent deployments are blamed without evidence.
Support teams and developers use different terminology.
Environmental differences are overlooked.
Production data cannot be reproduced safely.
Intermittent failures disappear during testing.
Multiple failures are grouped into one ticket.
The original error is hidden by secondary failures.
Observability does not identify the affected request.
Debugging changes alter timing and hide race conditions.
Fixes address symptoms rather than causes.

Poor diagnosis can lead to:

Repeated incidents
Unnecessary rollbacks
Failed hotfixes
New defects
Data corruption
Long outages
Increased support costs
Loss of user trust
Blame-driven incident response
Technical debt from temporary workarounds

A structured AI-assisted process helps teams investigate defects consistently while preserving engineering accountability.

Typical Use Casespublic

Use this workflow when:

Investigating a production error
Analyzing a failed background job
Troubleshooting an intermittent defect
Reproducing a customer-reported issue
Reviewing a stack trace
Correlating logs from multiple services
Comparing successful and failed requests
Investigating a post-deployment regression
Diagnosing a database-related failure
Investigating API integration problems
Analyzing a memory or performance problem
Investigating concurrency issues
Diagnosing permission or authentication failures
Preparing an escalation to a vendor
Preparing a root-cause analysis
Designing a safe hotfix

Do NOT Use This Workflow Whenpublic

This workflow is not intended to:

Declare a root cause without evidence
Replace production monitoring
Replace runtime debugging
Replace incident-response procedures
Execute suspicious code
Analyze malware by running it
Apply untested production changes
Expose customer data to an unapproved AI platform
Share active credentials, tokens, or private keys
Recommend disabling security controls
Replace subject-matter experts
Hide uncertainty
Collapse unrelated failures into one diagnosis
Treat correlation as proof of causation
Guarantee that a defect is fully resolved

The output should be treated as an investigation aid.

Expected Outcomepublic

After completing this workflow, you should have:

A normalized defect statement
A timeline of relevant events
A reproduction assessment
A symptom-versus-cause distinction
An execution-path reconstruction
A ranked hypothesis list
Evidence supporting and contradicting each hypothesis
Missing diagnostic information
A targeted reproduction plan
A minimal experiment plan
Recommended instrumentation
A likely root cause, when supported
A corrective-action recommendation
Regression-test requirements
Deployment and rollback considerations
Remaining uncertainty

🔒 The complete playbook — reference models, worked examples, and operational guidance — is included with ABME membership.

Unlock Full Blueprint

Defect Diagnosis Objectivesprotected

A complete diagnosis should answer:

What exactly failed?
Who or what was affected?
When did the failure begin?
Is the failure ongoing?
Is it reproducible?
What conditions trigger it?
What conditions do not trigger it?
What changed before the failure began?
Which component first deviated from expected behavior?
Is the visible error primary or secondary?
What evidence supports each hypothesis?
What evidence contradicts each hypothesis?
What information is missing?
What is the smallest experiment that separates competing explanations?
What is the safest correction?
How will the fix be verified?
How will recurrence be detected?

Diagnostic Confidence Modelprotected

Confirmed Root Cause

The cause has been reproduced or directly demonstrated.

A failing test
A controlled experiment
A trace showing the precise failure path
A data condition that consistently triggers the defect
A code-level explanation matching observed behavior
A fix that removes the failure while preserving other behavior

Probable Root Cause

Multiple pieces of evidence support the cause, but direct reproduction or final verification is incomplete.

Possible Cause

The explanation is technically plausible but insufficiently supported.

Unlikely Cause

Available evidence contradicts the explanation or makes it improbable.

Ruled Out

A controlled test or conclusive evidence demonstrates that the explanation is not responsible. AI should always state the confidence level of its conclusions.

Defect Severity Modelprotected

Critical

The defect causes or may cause:

Broad production outage
Data corruption
Security compromise
Authentication or authorization failure
Financial misstatement
Cross-tenant exposure
Irrecoverable state
Destructive automated behavior
Regulatory reporting failure

High

The defect causes:

Major feature failure
Significant customer impact
Persistent job failure
Incorrect business decisions
Large-scale performance degradation
Repeated operational intervention
High-risk data inconsistency

Medium

The defect affects:

A limited feature
A subset of users
An edge case
Operational efficiency
Error handling
Reporting accuracy with a workaround available

Low

The defect causes:

Minor usability problems
Cosmetic inconsistencies
Limited logging or diagnostic issues
Small workflow inconvenience
Low-impact incorrect messaging

Evidence Categoriesprotected

User-Reported Evidence

User descriptions are valuable but may reflect interpretation rather than the underlying technical behavior.

Screenshots
Error messages
Timestamps
Actions performed
Affected account
Device and browser
Input values
Frequency
Expected result
Actual result

Application Evidence

Stack traces
Structured logs
Request identifiers
Trace spans
Metrics
Health checks
Error-rate changes
Memory usage
CPU usage
Queue depth
Thread state

Infrastructure Evidence

Deployment events
Scaling events
Container restarts
Network failures
DNS changes
Certificate changes
Load-balancer behavior
Disk capacity
Resource throttling
Node health

Data Evidence

Sensitive production data should be sanitized before AI analysis.

Database records
Schema state
Migration status
Duplicate records
Null values
Data type mismatches
Referential-integrity issues
Unexpected encoding
Time-zone values

Code Evidence

Relevant functions
Recent diffs
Validation logic
Error handling
Concurrency behavior
Serialization
Dependency usage
Configuration parsing
Tests

Change Evidence

A recent change is a candidate cause, not automatic proof.

Deployment history
Feature-flag changes
Dependency upgrades
Configuration changes
Database migrations
Security-policy changes
Infrastructure updates
External API changes

Example Inputprotected

Defect Report

Users occasionally receive a 500 Internal Server Error when submitting an order.

The issue began after a release that added promotional discount codes.

It occurs only for some orders and cannot be reproduced with every discount code.

Error Log

2026-07-14T15:32:41.118Z INFO  Order request started requestId=7bd8 userId=18422 cartId=99314
2026-07-14T15:32:41.149Z INFO  Promotion applied requestId=7bd8 promotionCode=SUMMER25 discount=25
2026-07-14T15:32:41.173Z ERROR Order submission failed requestId=7bd8 error="Object reference not set to an instance of an object" at OrderService.BuildOrder(OrderRequest request) line 218

Relevant Code

public Order BuildOrder(OrderRequest request)
{
    var promotion = _promotionRepository
        .FindByCode(request.PromotionCode);
    var order = new Order
    {
        CustomerId = request.CustomerId,
        Subtotal = request.Subtotal,
        Discount = promotion.DiscountPercent,
        PromotionId = promotion.Id
    };
    if (request.PromotionCode != null)
    {
        order.Total = CalculateTotal(
            request.Subtotal,
            promotion.DiscountPercent
        );
    }
    else
    {
        order.Total = request.Subtotal;
    }
    return order;
}

Example Diagnostic Analysisprotected

The visible symptom is a NullReferenceException during order construction.

The failure is likely data-dependent because:

Only some discount-code orders fail.
The code retrieves a promotion by code.
The returned promotion value is dereferenced before its existence is checked.
An invalid, expired, differently cased, or otherwise unmatched code could cause FindByCode to return null.

The fact that the log says “Promotion applied” may indicate that:

The log occurs before the repository lookup is validated.
A different component recognized the code.
The promotion record changed between validation and order construction.
The log message is inaccurate.
Repository filtering differs between validation and order creation.

The code alone supports a likely null dereference, but it does not prove why the repository returned null.

Example Hypothesis Tableprotected

ID	Hypothesis	Confidence	Supporting Evidence	Contradicting Evidence	Validation
H1	Repository returns null for some codes	High	promotion dereferenced without check; failures limited to some codes	Log says promotion applied	Test failing code directly against repository
H2	Promotion is deleted or expires between validation and order creation	Moderate	Intermittent and code may perform multiple lookups	No timing evidence	Trace both reads and promotion version
H3	Code matching is case-sensitive in one component	Moderate	Some codes fail; validation may differ	No failing input shown	Compare exact code values and query behavior
H4	request is null	Low	Would cause a null reference	Log contains request-derived fields	Ruled out by successful request logging
H5	PromotionCode is null	Low	Could influence lookup	Failure occurs on promotion orders	Add input snapshot excluding sensitive data

Example Corrective Designprotected

public Order BuildOrder(OrderRequest request)
{
    ArgumentNullException.ThrowIfNull(request);
    Promotion? promotion = null;
    if (!string.IsNullOrWhiteSpace(request.PromotionCode))
    {
        promotion = _promotionRepository
            .FindActiveByCode(request.PromotionCode.Trim());
        if (promotion is null)
        {
            throw new InvalidPromotionException(
                "The promotion code is invalid or no longer active."
            );
        }
    }
    var discountPercent = promotion?.DiscountPercent ?? 0;
    return new Order
    {
        CustomerId = request.CustomerId,
        Subtotal = request.Subtotal,
        Discount = discountPercent,
        PromotionId = promotion?.Id,
        Total = CalculateTotal(
            request.Subtotal,
            discountPercent
        )
    };
}

This code may prevent the null dereference, but the repository lookup and promotion-validation behavior must still be confirmed.

A complete fix may also require:

Consistent code normalization
Atomic promotion validation
Expiration handling
Customer-facing error mapping
Audit or diagnostic logging
Tests for invalid and expired codes

Diagnostic Concepts and Techniquesprotected

Symptom Versus Root Cause

A symptom is what was observed.

Examples:

HTTP 500
Blank page
Queue backlog
Database timeout
Incorrect report
Job retry
Null-reference exception

A root cause explains why the system entered the failing condition.

Example:

Symptom: Null-reference exception.

Immediate cause: Code dereferenced a null promotion object.

Root cause: The order flow assumed a promotion validated earlier would always remain available, but expired promotions were filtered out during the second lookup.

Contributing factor: No explicit invalid-promotion handling.

Detection gap: Logs did not record the repository lookup result.

Stopping at the exception type may produce an incomplete diagnosis.

Timeline Analysis

A reliable timeline may include:

Time	Event	Source	Confidence
14:00	Version 3.4 deployed	Deployment system	Confirmed
14:17	First failing request	Application log	Confirmed
14:19	Error rate rises to 3%	Monitoring	Confirmed
14:25	Promotion records refreshed	Scheduled job	Confirmed
14:33	Support ticket opened	Service desk	Confirmed

Use exact timestamps and account for:

Time zones
Clock drift
Delayed log ingestion
Batch processing
Retry delay
Asynchronous events

Execution-Path Reconstruction

For each failure, reconstruct:

Request received
Authentication completed
Authorization evaluated
Input parsed
Input validated
Data retrieved
Business rules applied
State updated
External dependency called
Response created
Audit or logging completed

Identify the first step where:

Expected output is absent
Unexpected output appears
State differs
Timing becomes abnormal
An exception occurs

The earliest divergence is often more useful than the final error.

Hypothesis Management

A good hypothesis should:

Explain the observed symptoms
Match the timeline
Be technically possible
Identify an expected mechanism
Be testable
Predict evidence
Be distinguishable from alternatives

Weak:

The database is broken.

Better:

Requests fail when the order references a promotion that was soft-deleted. The repository filters deleted promotions, returns null, and the service dereferences the result.

Avoiding Confirmation Bias

Engineers may favor a theory because:

It matches a recent incident
It involves a disliked component
It was proposed by a senior engineer
It explains one prominent log
It appeared after a deployment
It is easy to test

For each leading hypothesis, actively ask:

What evidence would disprove it?
What symptom does it fail to explain?
Which competing hypothesis fits the evidence?
Did we search only for confirming logs?
Did a previous change alter the test conditions?

Diagnostic Experiment Design

A strong experiment:

Changes one important variable
Has a control
Predicts results for each hypothesis
Produces objective evidence
Is reversible
Avoids production risk
Uses representative conditions
Does not introduce excessive instrumentation

Example:

Experiment	H1 Prediction	H2 Prediction
Submit known active code	Succeeds	Succeeds
Submit expired code	Repository returns null	Failure only if expiration timing aligns
Normalize code casing	Previously failing mixed-case code succeeds	No change
Perform one atomic lookup	No change if lookup consistently null	Removes timing-dependent failures

Logging Improvements

Diagnostic logs should include:

Timestamp
Severity
Event name
Request ID
Trace ID
Operation
Component
Outcome
Duration
Error classification
Safe record identifier
Dependency name
Retry number

Do not log:

Passwords
Access tokens
Private keys
Authentication headers
Full payment data
Unnecessary personal data
Secret query parameters
Sensitive request bodies

Defect-Class Investigation Guidesprotected

Intermittent Defects

Intermittent failures often involve:

Timing
Concurrency
Resource contention
Cache state
External dependency instability
Retry ordering
Eventual consistency
Background jobs
Duplicate processing
Clock drift
Load
Network behavior

Avoid adding excessive diagnostic logging that materially changes timing.

Useful tools may include:

Correlation IDs
Distributed tracing
Sequence numbers
Monotonic timestamps
State versions
Lock timing
Queue timestamps
Thread identifiers
Request sampling

Data-Dependent Defects

Compare failing and successful records for:

Null fields
Empty values
Length
Character encoding
Date ranges
Legacy formats
Duplicate identifiers
Missing relationships
Tenant
Region
Status
Version
Ownership
Historical migrations

Create a minimized synthetic record containing only the characteristics needed to reproduce the failure.

Do not copy sensitive production records into an unapproved environment.

Environment-Dependent Defects

Compare:

Application version
Runtime version
Operating system
Architecture
Region
Configuration
Feature flags
Secrets source
Network routes
DNS
Certificates
Time zone
Locale
Dependency version
Database version
Container image
Resource limits

“Works on my machine” is evidence of an environmental difference, not evidence that the defect is invalid.

Concurrency Defects

Look for:

Shared mutable state
Read-modify-write sequences
Missing locks
Duplicate job delivery
Non-atomic updates
Stale reads
Optimistic concurrency failures
Inconsistent lock order
Thread-unsafe libraries
Singleton state
Reused request objects
Temporary file collisions

A useful experiment may increase concurrency in a controlled test environment rather than adding arbitrary delays.

Retry Defects

Retries can:

Hide the original failure
Duplicate writes
Amplify outages
Cause request storms
Change event order
Exceed rate limits
Trigger duplicate notifications

Review:

Which errors are retryable?
Is the operation idempotent?
Is backoff used?
Is jitter used?
Is there a maximum?
Is the first error retained?
Are retries observable?
Can retries outlive the caller?

Database Defects

Review:

Query parameters
Transactions
Locking
Isolation level
Connection handling
Command timeout
Deadlocks
Index changes
Schema mismatch
Nullability
Migration status
Replica lag
Read/write routing
Data type conversion
N+1 queries
Partial commits

A database timeout is a symptom. The cause could be locking, query design, resource exhaustion, network delay, or configuration.

External Dependency Defects

Document:

Endpoint
Request
Response
Timeout
Retry
Rate limit
Authentication
API version
Dependency status
Contract changes
Fallback behavior

Capture enough information to compare successful and failed calls without logging secrets or regulated data.

Time and Date Defects

Review:

UTC versus local time
Daylight-saving changes
Date-only values
Ambiguous timestamps
Expiration boundaries
Inclusive versus exclusive ranges
Clock synchronization
Serialization
Locale parsing
Leap years
Month boundaries

Time-related defects are often intermittent because they occur only near boundaries.

Correction, Verification, and Deploymentprotected

Contributing Factors

Contributing factors are not the primary cause but increased likelihood or impact.

Examples:

Missing validation
Inaccurate log message
Insufficient test coverage
Duplicate repository lookups
No contract for null behavior
Generic error mapping
Missing dashboard segmentation
Manual promotion cleanup
Deployment without canary monitoring

Corrective Actions

Separate corrective actions into:

Immediate Containment

Disable a feature flag
Reject unsafe input
Pause a failing job
Roll back a release
Isolate affected records
Add a temporary guard

Root-Cause Correction

Fix the defective logic
Correct data
Change the transaction
Repair configuration
Update dependency integration
Correct permission assignment

Prevention

Add tests
Add validation
Improve observability
Add deployment checks
Add schema constraints
Add code-review guidance
Improve runbooks

Fix Verification

A fix is not verified merely because:

The exception disappeared
One request succeeded
The code compiled
The unit test passed
Error rate temporarily dropped

Verification should demonstrate:

The original reproduction now succeeds or fails correctly
Competing cases still behave correctly
No data inconsistency remains
Regression tests pass
Performance is acceptable
Security behavior is unchanged or improved
Monitoring detects recurrence
Rollback remains possible

Regression-Test Plan

Include:

Exact original failure
Successful control case
Boundary cases
Invalid input
Missing data
Concurrent execution
Dependency failure
Retry behavior
Permission failure
Legacy data
Rollback compatibility

Each confirmed defect should produce at least one regression test where feasible.

Deployment Plan

Document:

Fix version
Components changed
Database or configuration dependency
Deployment order
Feature flag
Canary group
Validation checks
Metrics to monitor
Error threshold
Rollback trigger
Rollback steps
Data repair
Support communication

Diagnostic Metricsprotected

Teams may track:

Mean time to acknowledge
Mean time to reproduce
Mean time to isolate
Mean time to resolve
Defects reopened
Defects without regression tests
Incidents caused by repeat defects
Percentage of tickets with correlation IDs
Percentage of defects with confirmed root cause
Diagnostic experiments per defect
Fixes rolled back
Observability gaps identified

Metrics should improve diagnosis, not penalize teams for complex systems.

Governance Recommendationsprotected

Define:

Approved AI platforms
Allowed code and log classifications
Prohibited data
Incident-data handling
Human review requirements
Root-cause confidence standards
Production experiment approval
Security escalation criteria
Evidence retention
Diagnostic prompt ownership
Model quality review
False-conclusion reporting
Audit requirements

Automation Opportunitiesprotected

Bug-ticket templates
Incident-management systems
Application-support workflows
Log-analysis tools
APM platforms
CI/CD failure analysis
Error-tracking systems
Post-deployment monitoring
Customer escalation workflows
Root-cause analysis processes
A mature diagnostic workflow could: collect the defect report; retrieve correlated logs; retrieve traces and metrics; identify recent changes; compare successful and failed requests; generate ranked hypotheses; recommend controlled experiments; track evidence collected; convert confirmed defects into regression tests; feed observability gaps into the backlog.

Pro Tipsprotected

Normalize the defect before analyzing it.
Separate facts, observations, assumptions, and unknowns.
Use exact timestamps and time zones.
Find the earliest divergence, not only the final exception.
Compare successful and failed executions.
Maintain multiple hypotheses.
Record evidence that contradicts the leading theory.
Use the smallest experiment that separates competing causes.
Change one important variable at a time.
Treat recent deployments as candidates, not proof.
Preserve the original failure evidence.
Avoid diagnostic logging that changes timing excessively.
Do not test by weakening security.
Convert confirmed causes into regression tests.
Document contributing factors separately from the root cause.
Verify the correction against the original reproduction.
Add monitoring that would detect recurrence.
State diagnostic confidence explicitly.
Keep unresolved uncertainty visible.
Require human validation before declaring the defect resolved.

Common Mistakesprotected

Starting with a Fix — a code change made before the failure mechanism is understood may hide the defect without resolving it.
Treating the Last Error as the Root Cause — a timeout, null reference, or HTTP 500 may be the final result of an earlier failure.
Blaming the Most Recent Deployment — timeline correlation is useful, but evidence is required.
Changing Multiple Variables — if a test changes code, configuration, data, and infrastructure together, the result does not isolate the cause.
Ignoring Successful Cases — successful requests often reveal the variable that matters.
Ignoring Contradicting Evidence — a strong diagnosis explains both failure and success.
Overusing Production Tests — use a controlled environment when possible. Production experiments should be minimal, reversible, approved, and observable.
Logging Sensitive Data — diagnostic urgency does not override privacy and security requirements.
Using Artificial Delays as a Final Fix — a delay may reduce race frequency without correcting synchronization.
Closing the Defect When the Symptom Stops — intermittent defects may disappear temporarily. Confirm the mechanism and test the correction.
Combining Multiple Defects — different error signatures, affected populations, or timelines may indicate separate issues.

Security Considerationsprotected

Before sharing defect evidence with an AI system: remove credentials.
Remove access tokens.
Remove session cookies.
Remove private keys.
Remove connection strings.
Remove payment data.
Remove unnecessary personal data.
Sanitize customer records.
Sanitize internal URLs where required.
Confirm AI platform approval.
Confirm data-retention settings.
Follow incident-handling policy.
Do not weaken security to test a hypothesis. Unsafe examples include: disabling certificate validation; granting broad administrator access; turning off authorization; exposing a production database publicly; logging raw authentication headers; copying customer data to a personal test environment.

Brian Diamond

Founder, BrianOnAI

Twenty-five years designing, operating, and governing enterprise infrastructure — from MSP operations across dozens of client environments to enterprise infrastructure leadership. This blueprint codifies the operating model he's implemented in production, not theory.

Related Blueprints

Perform an AI-Assisted Code Review Perform an Application Architecture Review Generate Unit and Integration Tests Refactor Code Safely Generate API and Developer Documentation Identify and Prioritize Technical Debt Review a Technical Design Modernize a Legacy Application Build and Maintain Development Prompts Debug PowerShell Errors Generate Pester Tests for a PowerShell Script Add Enterprise Logging to a PowerShell Script

⚠ Normalization Warnings — 13 for review

RESTRUCTURE: 'Primary Prompt' and 'Follow-Up Prompts' (two source H1s, twelve H2 follow-ups) combined into one prompt_pack tool with 13 prompts; 'when' guidance lines are editorial additions, prompt text preserved verbatim.
CLASSIFICATION TO CONFIRM: 'Prerequisites' classified as a checklist TOOL (gather-before-start items are completable). Alternative: body/prose.
CLASSIFICATION TO CONFIRM: 'Example Hypothesis Table' kept as a body/example (the doc's worked instance) AND a separate 'Hypothesis Table' matrix TOOL was constructed from the doc's described hypothesis structure using the example rows as example_rows. Confirm the tool is warranted vs. leaving it only as an example.
CLASSIFICATION TO CONFIRM: 'Diagnostic Experiment Design' section describes an experiment matrix; a 'Diagnostic Experiment Matrix' TOOL was constructed from its example table (H1/H2 columns). The section itself is retained as body/prose under the Diagnostic Concepts group. Confirm the tool split.
CLASSIFICATION TO CONFIRM: 'Testing Recommendations' classified as a checklist TOOL named 'Diagnostic Test Plan' (nested verifiable/actionable items). Display name changed from source heading — confirm. Alternative: body/reference.
CLASSIFICATION: 'Root-Cause Statement Template' classified as a template TOOL (labeled fill-in structure with an example). Confirm.
REFERENCE VS TOOL: 'Diagnostic Confidence Model', 'Defect Severity Model', and 'Evidence Categories' classified as body/reference (consulted taxonomies with tiered definitions), not tools.
GROUPING: Many domain sections grouped into three body groups ('Diagnostic Concepts and Techniques', 'Defect-Class Investigation Guides', 'Correction, Verification, and Deployment') to avoid a flat 25+ section list. 'Diagnostic Metrics' and 'Governance Recommendations' left ungrouped as standalone prose. Confirm grouping boundaries.
PLAYBOOK: 'Security Considerations' and 'Common Mistakes' mapped to playbook flat lists (name-based tail sections) rather than body groups, since they are list-form here without deep subsection code. 'Automation Opportunities' and 'Pro Tips' mapped by name. No Quick Wins or Roadmap sections present; those playbook fields intentionally empty.
MATRIX example_rows: hypothesis-table and experiment-matrix example_rows are the doc's own tables verbatim; skeleton_rows set to 0 per empty-skeleton rule.
CODE LANGUAGE: Example Input / Corrective Design tagged as csharp (C# syntax evident); Example Analysis and Hypothesis Table are prose/table examples with no code language.
RELATED: seo.related includes both AI-track and PowerShell-track related workflows from the two Related Workflows lists.
Difficulty preserved as 'Intermediate–Advanced' from source (schema example uses single-value difficulty; source range retained).

SEO Block

Title tag: Diagnose Software Defects with AI Evidence | ABME (49 chars)
Meta: Turn scattered defect evidence into ranked hypotheses, controlled experiments, and a demonstrable root cause — without speculative fixes or exposed secrets. (156 chars)
Schema: HowTo · noindex: false
Related: ai-001, ai-003, ai-004, ai-005, ai-006, ai-007, ai-008, ai-009, ai-010, ps-003, ps-004, ps-006, ps-007, ps-008
Keywords: software defect diagnosis, root cause analysis, ai debugging, hypothesis-driven debugging, stack trace analysis, intermittent defect, post-deployment regression, execution path reconstruction, diagnostic confidence, regression test from defect

Copied

Stop Guessing Faster — Diagnose the Defect from Evidence, Not the First Plausible Theory

Executive Brief

Your Challenge

Common Obstacles

The ABME Approach

Insight Summary

The Journey

Normalize the Defect and Build the Evidence Base

Generate and Rank Hypotheses with AI

Confirm the Cause, Correct, and Verify

What's Inside the Execution Layer

Prerequisites Checklist

Defect Intake Template

Summary

Expected Behavior

Actual Behavior

Impact

Frequency

First Known Occurrence

Last Known Success

Environment

Reproduction Steps

Evidence

Recent Changes

Workaround

Defect Diagnosis Prompt Pack

Primary Prompt

Analyze a Stack Trace

Analyze Application Logs

Compare Successful and Failed Requests

Diagnose an Intermittent Defect

Diagnose a Post-Deployment Regression

Diagnose a Data-Dependent Failure

Diagnose a Permission Failure

Diagnose a Performance Defect

Create a Reproduction Plan

Create a Minimal Experiment Matrix

Design Diagnostic Instrumentation

Verify a Proposed Fix

Hypothesis Table

Diagnostic Experiment Matrix

Root-Cause Statement Template

Triggering condition

Defective behavior

Failure mechanism

User or system impact

Why controls did not prevent or detect it

Validation Checklist

Diagnostic Test Plan

Reproduce Before Fixing

Preserve the Failing Evidence — store

Add a Regression Test

Test Neighboring Behavior — a fix may affect

Test Under Representative Conditions

Full Playbook

Overviewpublic

Business Problempublic

Typical Use Casespublic

Do NOT Use This Workflow Whenpublic

Expected Outcomepublic

Defect Diagnosis Objectivesprotected

Diagnostic Confidence Modelprotected

Confirmed Root Cause

Probable Root Cause

Possible Cause

Unlikely Cause

Ruled Out

Defect Severity Modelprotected

Critical

High

Medium

Low

Evidence Categoriesprotected

User-Reported Evidence

Application Evidence

Infrastructure Evidence

Data Evidence

Code Evidence

Change Evidence

Example Inputprotected