Found one Java-level deadlock:

Q: Enforce global lock ordering by stable key

When to use: Use this when/if two or more resources are locked together in varying orders.. Deadlock requires a cycle; a total order makes cycles structurally impossible because every thread contends in the same direction. Choose keys immune to churn — database ids, not names or array indices — and centralize the comparison so no call site improvises.

Q: Break cycles with tryLock and backoff

When to use: Use this when/if ordering is impractical across legacy components.. ReentrantLock.tryLock attempts acquisition without parking; on failure, release everything held and retry after jitter. Threads may collide repeatedly but never freeze permanently — worst case degrades into bounded retry cost rather than a wedged server requiring restart.

Q: Shrink critical sections and ban alien calls under lock

When to use: Use this when/if locks are held across callback or remote invocations.. Every method invoked while holding a lock can secretly acquire more locks — listener callbacks, ORM flushes, cache loaders. Copy needed data out under the lock, release, then do alien work lock-free. Smaller critical sections also shrink contention windows regardless of deadlocks.

Q: Replace paired-lock patterns with concurrent structures

When to use: Use this when/if the deadlock guards a shared map or counter pair.. ConcurrentHashMap.compute atomizes per-key updates under one internal striping scheme, and LongAdder replaces lock-paired counters. Deleting the second lock deletes the cycle; the JDK authors already debugged the concurrency so you delete the bug class wholesale.

Q: Deploy continuous detection: watchdog plus scheduled dumps

When to use: Use this when/if you must catch regressions before customers notice a frozen feature.. A background probe polls ThreadMXBean.findDeadlockedThreads every few seconds; on discovery it captures jcmd Thread.print output, alerts, and optionally flags the offending transactions. Deadlocks then page an engineer with evidence attached instead of surfacing as mysterious stalled traffic.

🔴 The Error You're Seeing

Confirm this matches your console output. If it does, you're in the right place.

ERROR LOG

// DOC-DERIVED — canonical jstack/jcmd output shape; ids, addresses and frames vary per run
Found one Java-level deadlock:
=============================
"Thread-B":
  waiting to lock monitor 0x00007fa1bc006b00 (object 0x000000076ab62208, a java.lang.Object),
  which is held by "Thread-A"
"Thread-A":
  waiting to lock monitor 0x00007fa1bc005e00 (object 0x000000076ab62238, a java.lang.Object),
  which is held by "Thread-B"

Java stack information for the threads listed above:
===================================================
"Thread-B":
	at com.devinhyderabad.billing.Account.transferFrom(Account.java:41)
	at com.devinhyderabad.billing.TransferService.run(TransferService.java:22)

"Thread-A":
	at com.devinhyderabad.billing.Account.transferFrom(Account.java:41)
	at com.devinhyderabad.billing.TransferService.run(TransferService.java:25)

Found 1 deadlock.

⚡ Quick Fix Works 80% of the time

Impose one global acquisition order for every pair of locks — sort by a stable key such as account id before locking, and the cycle becomes impossible.

// transfer(a, b): always lock the lower-id account first
Account first  = a.id < b.id ? a : b;
Account second = a.id < b.id ? b : a;
synchronized (first) {
    synchronized (second) {
        debit(first, amount);
        credit(second, amount);
    }
}

🧠 Why this Happens

Tap to expand the deep technical explanation

The JVM’s deadlock detector builds a wait-for graph over monitors and ownable synchronizers: each node is a thread, each edge points at the owner of the resource it blocks on. A cycle means no member can ever be scheduled to release — the runtime prints the participating threads, the objects, and the holding relationships verbatim. Detection is passive: it breaks no locks, interrupts nothing, and the threads remain frozen until restart or external intervention.

The HITEC City Parking Spot Analogy:

Two cars meeting head-on in an alley too narrow to pass: each waits for the other to reverse, forever. The traffic report (thread dump) names both drivers and the alley — but tow trucks are not dispatched automatically.

🔁 How to Reproduce Confirm this is your error

Thread A locks obj1 then sleeps then locks obj2; Thread B does the reverse. Run jcmd <pid> Thread.print (or jstack -l <pid>) once both park: the section above appears with your frames. (Canonical wording; not a lab capture.)

🛠️ Solutions (5 Ways to Fix)

Solution 1✓ Most common cause

Enforce global lock ordering by stable key

👉 Use this when/if two or more resources are locked together in varying orders.

Deadlock requires a cycle; a total order makes cycles structurally impossible because every thread contends in the same direction. Choose keys immune to churn — database ids, not names or array indices — and centralize the comparison so no call site improvises.

private static int orderKey(Account a) { return Long.compare(a.id, otherId); }

void transfer(Account from, Account to, Money amt) {
    Account lower = from.id < to.id ? from : to;
    Account upper = from.id < to.id ? to : from;
    synchronized (lower) {
        synchronized (upper) {
            move(lower, upper, amt);
        }
    }
}

Solution 2

Break cycles with tryLock and backoff

👉 Use this when/if ordering is impractical across legacy components.

ReentrantLock.tryLock attempts acquisition without parking; on failure, release everything held and retry after jitter. Threads may collide repeatedly but never freeze permanently — worst case degrades into bounded retry cost rather than a wedged server requiring restart.

if (a.lock.tryLock(50, TimeUnit.MILLISECONDS)) {
    try {
        if (b.lock.tryLock(50, TimeUnit.MILLISECONDS)) {
            try { move(a, b, amt); return true; }
            finally { b.lock.unlock(); }
        }
    } finally { a.lock.unlock(); }
}
sleepWithJitter(20); // backoff, then whole sequence retried

Solution 3

Shrink critical sections and ban alien calls under lock

👉 Use this when/if locks are held across callback or remote invocations.

Every method invoked while holding a lock can secretly acquire more locks — listener callbacks, ORM flushes, cache loaders. Copy needed data out under the lock, release, then do alien work lock-free. Smaller critical sections also shrink contention windows regardless of deadlocks.

Snapshot s;
synchronized (account) {
    s = account.snapshot();      // copy primitives out
}
auditService.record(s);          // alien call outside the lock

Solution 4

Replace paired-lock patterns with concurrent structures

👉 Use this when/if the deadlock guards a shared map or counter pair.

ConcurrentHashMap.compute atomizes per-key updates under one internal striping scheme, and LongAdder replaces lock-paired counters. Deleting the second lock deletes the cycle; the JDK authors already debugged the concurrency so you delete the bug class wholesale.

balances.compute(from, (id, bal) -> bal.subtract(amt));
balances.compute(to,   (id, bal) -> bal.add(amt));
// no explicit locks anywhere in transfer()

Solution 5

Deploy continuous detection: watchdog plus scheduled dumps

👉 Use this when/if you must catch regressions before customers notice a frozen feature.

A background probe polls ThreadMXBean.findDeadlockedThreads every few seconds; on discovery it captures jcmd Thread.print output, alerts, and optionally flags the offending transactions. Deadlocks then page an engineer with evidence attached instead of surfacing as mysterious stalled traffic.

ThreadMXBean mx = ManagementFactory.getThreadMXBean();
long[] stuck = mx.findDeadlockedThreads();
if (stuck != null) {
    dumpAndAlert(mx.getThreadInfo(stuck, /*maxDepth*/ 60));
}

📋 Version Notes

Java 8

jstack -l prints this exact section; jcmd <pid> Thread.print equally available.

Java 11

Output unchanged; jcmd preferred tooling-wise.

Java 21

Virtual threads appear in dumps grouped separately from carrier platform threads; verify pinned-monitor cases with JFR events alongside traditional detection.

🛡️ How to Prevent This Next Time

Document a lock-ordering convention per subsystem, gate new nested-lock code through review, run stress tests with randomized interleavings (JCStress-style or simply high-concurrency soak tests) so cycles surface in CI, and keep the watchdog enabled in production from day one.