Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@yoff
Copy link
Contributor

@yoff yoff commented May 20, 2025

This PR introduces three queries for thread-safe classes, corresponding to three properties that such classes must possess, known as

  • P1: No escaping
  • P2: Safe publication
  • P3: Correct synchronization
    Each query is introduced in its own commit, P3 first since it comes with some definitions that make P2 and P1 simple to write. And then there is a fourth commit fixing up the query for P3, since its detection in the first version only works when there are no loops in the call graph. I chose to show both versions, since the first is quite simple to understand and the second is basically the same idea but operating on SCCs of the call graph.

Some considerations:

  • should (parts of) the module Monitors be moved into concurrency.qll?
  • should the severity be error?
  • the queries are being included in the code quality suite

@yoff yoff force-pushed the java/conflicting-access branch from 886f5b2 to d32cd70 Compare May 21, 2025 10:09
@github-actions
Copy link
Contributor

github-actions bot commented May 21, 2025

QHelp previews:

java/ql/src/Likely Bugs/Concurrency/Escaping.qhelp

Escaping

In a thread-safe class, non-final fields should generally be private (or possibly volatile) to ensure that they cannot be accessed by other threads in an unsafe manner.

Recommendation

If the field does not change, mark it as final. If the field is mutable, mark it as private and provide properly synchronized accessors.

References

java/ql/src/Likely Bugs/Concurrency/SafePublication.qhelp

Safe publication

In a thread-safe class, values must be published safely to avoid inconsistent or unexpected behavior caused by visibility issues between threads. If a value is not safely published, one thread may see a stale or partially constructed value written by another thread, leading to subtle concurrency bugs.

In particular, values of primitive types should not be initialised to anything but their default values (which for Object is null) unless this happens in a static context.

Techniques for safe publication include:

  • Using synchronized blocks or methods to ensure that a value is fully constructed before it is published.
  • Using volatile fields to ensure visibility of changes across threads.
  • Using thread-safe collections or classes that provide built-in synchronization, such as are found in java.util.concurrent.
  • Using the final keyword to ensure that a reference to an object is safely published when the object is constructed.

Recommendation

Choose a safe publication technique that fits your use case. If the value only needs to be written once, say for a singleton, consider using the final keyword. If the value is mutable and needs to be shared across threads, consider using synchronized blocks or methods, or using a thread-safe collection from java.util.concurrent.

Example

In the following example, the values of value and server_id are not safely published. The constructor creates a new object and assigns it to the field value. However, the field is not declared as volatile or final, and there are no synchronization mechanisms in place to ensure that the value is fully constructed before it is published. A different thread may see the default value null. Similarly, the field server_id may be observed to be 0.

public class UnsafePublication {
    private Object value;
    private int server_id;

    public UnsafePublication() {
        value = new Object(); // Not safely published, other threads may see the default value null
        server_id = 1; // Not safely published, other threads may see the default value 0
    }

    public Object getValue() {
        return value;
    }

    public int getServerId() {
        return server_id;
    }
}

To fix this example, we declare the field value as volatile. This will ensure that all changes to the field are visible to all threads. The field server_id is only meant to be written once, so we only need the write inside the constructor to be visible to other threads; declaring it final guarantees this:

public class SafePublication {
    private volatile Object value;
    private final int server_id;

    public SafePublication() {
        value = new Object(); // Safely published as volatile
        server_id = 1; // Safely published as final
    }

    public synchronized Object getValue() {
        return value;
    }

    public int getServerId() {
        return server_id;
    }
}

References

java/ql/src/Likely Bugs/Concurrency/ThreadSafe.qhelp

Not thread-safe

In a thread-safe class, all field accesses that can be caused by calls to public methods must be properly synchronized.

Recommendation

Protect the field access with a lock. Alternatively mark the field as volatile if the write operation is atomic. You can also choose to use a data type that guarantees atomic access. If the field is immutable, mark it as final.

References

@yoff yoff force-pushed the java/conflicting-access branch from d32cd70 to 126f974 Compare May 22, 2025 11:06
@yoff yoff force-pushed the java/conflicting-access branch from 126f974 to c2374e5 Compare May 22, 2025 11:42
@yoff yoff added the Awaiting evaluation Do not merge yet, this PR is waiting for an evaluation to finish label May 22, 2025
@yoff yoff marked this pull request as ready for review May 22, 2025 13:14
@yoff yoff requested a review from a team as a code owner May 22, 2025 13:14
@aschackmull
Copy link
Contributor

Commented offline.

@yoff
Copy link
Contributor Author

yoff commented Jun 9, 2025

Notes from offline review:

  • Everything private
  • Better lock detection, look at unreleased lock query
  • Monitor abstract toString
  • Remove char preds and fields from wrapped IPA types
  • Revert to NodeToBeDominated
  • locallySynchronizedOn not field?
    No, see example on page 550 of https://docs.oracle.com/javase/specs/jls/se25/jls25.pdf
  • Prefix should be calculated via matches or as output + _
  • getErasure -> getSourceDeclaration (needs to be on the field instead of the type)
  • getTypeDescriptor -> getPackage.getName (needs to be on the field instead of the type)
  • java.util , Collections, sync_name + sync_name.matches(“synchronized%”)
  • Define ExposedField
    - [ ] Hoist lookups out of negations, using exists
  • Pragma[inline] -> pragma[inline_late] + bindingset
  • AnnotatedAsThreadSafe, inline char pred
  • Create extension point
    - [ ] If a then b else any is often better than a implies b
  • Safe publication example is wrong.
  • Reformulate unsynchronized to find escaping paths from an exposed field access out to public methods.
    Is this not what providesAccess and "the call graph induced by a" do?
    Sure, but the reformulation is more about avoiding binary predicates and recursion through forall.
  • Split into self_conflicting and inconsistently locked
    • A publicly accessible write that is not locked is self-conflicting and can be reported right away.
    • A publicly accessible read that is not locked is conflicting with any publicly accessible write.
      This can be reported right away, if the definition of exposed field includes that there is a publicly accessible write.
    • All other cases involve two different locks.

Copilot AI review requested due to automatic review settings October 9, 2025 07:11
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces three CodeQL queries for analyzing thread-safe classes in Java, ensuring they properly implement thread safety properties. The queries correspond to the classical thread safety properties: no escaping (P1), safe publication (P2), and correct synchronization (P3).

  • Adds queries to detect data races in @ThreadSafe-annotated classes
  • Introduces queries to detect unsafe field publication and field escaping
  • Implements comprehensive monitor analysis including support for complex call graphs with loops

Reviewed Changes

Copilot reviewed 38 out of 38 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
ConflictingAccess.qll Core library providing monitor analysis and synchronization detection for thread-safe classes
ThreadSafe.ql Query detecting data races in classes marked as @ThreadSafe
SafePublication.ql Query detecting fields that are not safely published in thread-safe classes
Escaping.ql Query detecting potentially escaping fields in thread-safe classes
Test files Comprehensive test examples demonstrating various thread safety scenarios
Change notes Documentation for the three new queries
Query suite Integration of new queries into the code quality suite

yoff and others added 9 commits October 9, 2025 09:14
Co-authored-by: Raúl Pardo <[email protected]>
Co-authored-by: SimonJorgensenMancofi <[email protected]>
Co-authored-by: Bjørnar Haugstad Jåtten <[email protected]>
Our monitor analysis would be fooled by cycles in the call graph,
since it required all edges on a path to a conflicting access to be either
 - targetting a method where the access is monitored (recursively) or
 - monitored locally, that is the call is monitored in the calling method
For access to be monitored (first case) all outgoing edges (towards an access) need
to satisfy this property. For a loop, that is too strong, only edges out of the loop
actually need to be protected. This led to FPs.
Identified by triage of DCA results.
Previously, we did not use the erased type, so would not recgnize `CompletableFuture<R>`.
We now also recognize safe initializers.
- add missing qldoc
- remove use of `getErasure`
- remove use of `getTypeDescriptor`
- define `ExposedField`
@yoff yoff force-pushed the java/conflicting-access branch from 1a61230 to a1671ea Compare October 9, 2025 07:17
- favour unary predicates over binary ones
(the natural "conflicting access" is binary)
- switch to a dual solution to trade recursion through forall for simple existentials.

Co-authored-by: Anders Schack-Mulligen <[email protected]>
@yoff yoff force-pushed the java/conflicting-access branch from 8159678 to 61a3e96 Compare October 16, 2025 23:50
@yoff yoff requested a review from aschackmull October 17, 2025 00:04
yoff and others added 3 commits October 21, 2025 12:52
Co-authored-by: Anders Schack-Mulligen <[email protected]>
- do not use `getQualifiedName`
- use camelCase
- rework alert predicates
@yoff yoff force-pushed the java/conflicting-access branch from 40632d2 to f4878b3 Compare October 21, 2025 11:25
aschackmull
aschackmull previously approved these changes Oct 21, 2025
Copy link
Contributor

@aschackmull aschackmull left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM now.

@aschackmull
Copy link
Contributor

Code change looks good. If you haven't already, then it's probably good to triage a few results from a MRVA run to check that the latest version still looks good. Also, some basic performance checking is in order - let's do a dca run, but remember that it can be limited to these 3 queries to make it smaller/faster.

also format test files
@yoff
Copy link
Contributor Author

yoff commented Oct 21, 2025

Not sure if the failing test is simply unrelated?

@aschackmull
Copy link
Contributor

Not sure if the failing test is simply unrelated?

I think it's unrelated. I just tried to rerun, so let's see if that helps.

@yoff
Copy link
Contributor Author

yoff commented Oct 27, 2025

Triage of new results:

java/unreleased-lock, 21 new results
These are all due to lockInterruptibly now being correctly recognized as a locking call. This should in itself not introduce problems as the predicate failedLock handles InterruptedException.

  • several FPs in apache/geode: The code is very complex, but ultimately a function wraps a locking call and therefor does not release the lock. The query correctly detects that, but it is probably intended by the developer.
  • an FP in geode due to a ReadWriteLock (link). The reader is taken under certain conditions and the write under others. They are later released under the same conditions. A split control flow graph might have prevented this FP.
  • some FPs in hadoop where locking is wrapped in methods.
  • two FPs in google/guava due to locking being wrapped in methods.

java/safe-publication, 9 results
The FP rate for this has been scrutinized already, but with only 9 results I went throuhg them anyway.

  • 3 TPs in apache/flink, repeated in the buildless version
  • 2 TPs in gradle
  • 1 FP in gradle due to the unsafely published value being unreadable (it is an internal factory, I think it could probably be static).

java/not-threadsafe, 75 results
The FP rate for this has already been established to be good, but I went through a flock for each repo as a sanity check

  • several TPs in apache/flink SystemResourcesCounter.java (no locking at all in that file), repeated in buildless
  • several TPs in gradle, little locking used

@yoff
Copy link
Contributor Author

yoff commented Oct 27, 2025

I will do an extra DCA run on nightly for timing, since I changed the code, but otherwise I am ready to merge this :-)

@yoff
Copy link
Contributor Author

yoff commented Oct 28, 2025

Timing has actually improved with the code change: palatable/lambda is now slightly slower, but apache/commons now sees no slow-down. So the project with the very many prefixes is the only one where running the new queries is even visible.

@yoff yoff requested a review from aschackmull October 28, 2025 09:56
@yoff yoff merged commit 4461be1 into github:main Oct 28, 2025
20 of 21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Awaiting evaluation Do not merge yet, this PR is waiting for an evaluation to finish documentation Java

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants