p-org · ankushdesai · Jul 30, 2025 · Dec 10, 2024 · Mar 13, 2025 · Mar 16, 2025
diff --git a/Docs/docs/advanced/PVerifierLanguageExtensions/announcement.md b/Docs/docs/advanced/PVerifierLanguageExtensions/announcement.md
@@ -0,0 +1,23 @@
+# Announcing the New Verification Backend for P
+
+We are excited to announce the release of a new verification backend for the P programming language! This backend, which we call the P Verifier, allows you to prove that your systems behave correctly under all possible scenarios. The P Verifier is based on Mora et al. ([OOPSLA '23](https://dl.acm.org/doi/10.1145/3622876)) and uses UCLID5 (Polgreen et al., [CAV '22](https://dl.acm.org/doi/10.1007/978-3-031-13185-1_27)).
+
+Before the P Verifier, P helped users catch design-level bugs early in the development process through explicit-state model checking (Brooker and Desai, [Queue '25](https://doi.org/10.1145/3712057)). Given a formal model, specification, and a test driver describing a system configuration, the model checking backend systematically explores the given model in search of an execution that violates the given specification for the given system configuration. See the [Two-Phase Commit example tutorial](twopahsecommitverification.md) for an example model, specification, and three test drivers corresponding to three different system configurations (e.g., a configuration with three participants, one coordinator, and one client). P was used in this way to help AWS migrate [S3 (Simple Storage Service) from eventual to strong read-after-write consistency](https://aws.amazon.com/blogs/aws/amazon-s3-update-strong-read-after-write-consistency/).
+
+The new verification backend allows users to **prove** that their system design is correct for all executions over all possible system configurations. Instead of using explicit-state model checking, the new backend supports proofs by induction. Users still provide a formal model and specification. But, instead of giving a test driver, users must provide assumptions about what is possible in the system and an inductive invariant that implies that the given model satisfies the given specification under the given assumptions. The main job of the backend is to check that that given invariant is indeed inductive (all possible events in the system preserve it) and that the given invariant does indeed imply the given specification. UCLID5 accomplishes this job by way of a satisfiability modulo theories (SMT) solver, like Z3 (de Moura and Bjørner, [TACAS '08](https://doi.org/10.1007/978-3-540-78800-3_24)).
+
+Formal verification is important to AWS’s software correctness program (Brooker and Desai, [Queue '25](https://doi.org/10.1145/3712057)). Several formal tools have had successful applications within AWS in their respective domains. The new verification backend in P gives users the benefits of correctness proofs for the domain of distributed systems design, all while preserving the existing benefits of systematic testing.
+
+## Getting Started and Tutorial
+
+To start using the P Verifier, you must install P along with the verification dependencies (UCLID5 and an SMT solver like Z3). Detailed installation instructions are available [here](install-pverifier.md); simple usage instructions are available [here](using-pverifier.md).
+
+To help you get acquainted with the new verification features, we have prepared a comprehensive tutorial that walks you through the formal verification of a simplified two-phase commit (2PC) protocol. This tutorial covers the key concepts and steps of using the verification backend. You can find the tutorial [here](twophasecommitverification.md).
+
+## Industrial Application Inside Amazon Web Services
+
+The two-phase commit protocol described in the tutorial is deliberately simplified to help new users get started. In that protocol, one coordinator works with a fixed set of participants to agree on a single boolean value. Industrial systems, however, call for a number of generalizations. 
+
+We used the new verification backend to verify an industrial version of the two-phase commit protocol with the following generalizations. First, the protocol works for multiple rounds (think participants agreeing on sequences of boolean values instead of a single boolean value). Second, the participants agree on the state of a key-value store instead of boolean values. Third, the system works at the granularity of transactions (reads and writes) instead of individual values. Fourth, the key-value store is partitioned across sets of participants ("shards"). Finally, fifth, the safety property is a more general kind of consistency called [Snapshot Isolation](https://software.imdea.org/~gotsman/papers/si-podc16.pdf). We hope to release the internal details of this system and the corresponding proof of correctness at a later date.
+
+We look forward to your feedback. Happy verifying!
diff --git a/Docs/docs/advanced/PVerifierLanguageExtensions/init-condition.md b/Docs/docs/advanced/PVerifierLanguageExtensions/init-condition.md
@@ -0,0 +1,29 @@
+# Initialization Conditions
+
+Initialization conditions let us constrain the kinds of systems that we consider for formal verification. You can think of these as constraints that P test harnesses have to satisfy to be considered valid.
+
+??? note "P Init Condition Declaration Grammar"
+
+    ```
+    initConditionDecl :
+        | init-condition expression;     # P Init Condition Declaration
+    ```
+
+    `expression` is a boolean expression that should evaluate to true at initialization time.
+
+**Syntax:** `init-condition expression;`
+
+`expression` is a boolean expression that must be satisfied for the system to be considered valid. This is typically used with quantifiers to express constraints over sets of machines or values.
+
+=== "Init Condition Examples"
+
+    ``` java
+    // Ensures that there's a unique machine of type coordinator
+    init-condition forall (m: machine) :: m == coordinator() <==> m is Coordinator;
+
+    // Ensures that every machine in the participants set is a machine of type participant
+    init-condition forall (m: machine) :: m in participants() <==> m is Participant;
+
+    // Ensures that all yesVotes tallies start empty
+    init-condition forall (c: Coordinator) :: c.yesVotes == default(set[machine]);
+    ```
diff --git a/Docs/docs/advanced/PVerifierLanguageExtensions/install-pverifier.md b/Docs/docs/advanced/PVerifierLanguageExtensions/install-pverifier.md
@@ -0,0 +1,156 @@
+# Install Instructions for Amazon Linux
+
+PVerifier requires several dependencies to be installed. Follow the steps below to set up your environment.
+
+!!! success ""
+    After each step, please use the troubleshooting check to ensure that each installation step succeeded.
+
+### [Step 1] Install Java 11
+
+```sh
+sudo rpm --import https://yum.corretto.aws/corretto.key
+sudo curl -L -o /etc/yum.repos.d/corretto.repo https://yum.corretto.aws/corretto.repo
+sudo yum install java-11-amazon-corretto-devel maven
+```
+
+??? hint "Troubleshoot: Confirm that java is correctly installed on your machine."
+    ```shell
+    java -version
+    ```
+
+    If you get `java` command not found error, most likely, you need to add the path to `java` in your `PATH`.
+
+### [Step 2] Install SBT
+
+```sh
+sudo rm -f /etc/yum.repos.d/bintray-rpm.repo || true
+curl -L https://www.scala-sbt.org/sbt-rpm.repo > sbt-rpm.repo
+sudo mv sbt-rpm.repo /etc/yum.repos.d/
+sudo yum install sbt
+```
+
+??? hint "Troubleshoot: Confirm that sbt is correctly installed on your machine."
+    ```shell
+    sbt --version
+    ```
+
+    If you get `sbt` command not found error, most likely, you need to add the path to `sbt` in your `PATH`.
+
+### [Step 3] Install .NET 8.0
+
+Install .NET 8.0 from Microsoft:
+
+```sh
+wget https://dotnet.microsoft.com/download/dotnet/scripts/v1/dotnet-install.sh
+bash ./dotnet-install.sh  -c 8.0 -i $HOME/.dotnet
+
+sudo mkdir /usr/share/dotnet/
+sudo cp -r $HOME/.dotnet/* /usr/share/dotnet/
+```
+
+Then add the following line to `.zshrc` and run `source .zshrc` (or `.bashrc` if using bash):
+
+```sh
+export PATH=$HOME/.dotnet:$HOME/.dotnet/tools:$PATH
+```
+
+The purpose of copying the .NET distribution into `/usr/share/dotnet` is to make standard dotnet packages available to dotnet.   If you are uncomfortable modifying a system directory, you can add the following line to your `.zshrc` or `.bashrc` instead:
+
+```sh
+export DOTNET_ROOT=$HOME/.dotnet
+```
+
+### [Step 4] Install Z3
+
+```sh
+cd ~
+git clone https://github.com/Z3Prover/z3.git
+cd z3
+python scripts/mk_make.py --java
+cd build; make
+```
+
+Then add the following lines to your `.zshrc` and run `source ~/.zshrc` (or `.bashrc` if using bash):
+
+```sh
+export PATH=$PATH:$HOME/z3/build/
+export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/z3/build/
+```
+
+??? hint "Troubleshoot: Confirm that Z3 is correctly installed on your machine."
+    ```shell
+    z3 --version
+    ```
+
+    If you get `z3` command not found error, most likely, you need to add the path to `z3` in your `PATH`.
+
+Note:
+
+* Python: Building Z3 requires Python 3, but Python 2 is the default on Amazon Linux 2.  If `python --version` displays version 2, try
+    ```sh
+    sudo alternatives --set python `which python3`
+    ```
+    or
+    ```sh
+    alias python=python3
+    ```
+
+* G++: Building Z3 requires G++ 8 or later with support for C++20, but G++ 7 is the default on Amazon Linux 2.  If `g++ --version` displays version 7, try
+
+    ```sh
+    sudo yum install gcc10 gcc10-c++
+    ```
+
+    to install gcc10-gcc and gcc10-g++ and replace the string `gcc` with `gcc10-` in `config.mk`.
+
+### [Step 5] Install UCLID5
+
+```sh
+cd ~
+git clone https://github.com/uclid-org/uclid.git
+cd uclid
+sbt update clean compile "set fork:=true" test # should fail some tests that use cvc5 and delphi
+sbt universal:packageBin
+unzip target/universal/uclid-0.9.5.zip
+```
+
+Then add the following line to your `.zshrc` (or `.bashrc` if using bash) and run `source ~/.zshrc`:
+
+```sh
+export PATH=$PATH:$HOME/uclid/uclid-0.9.5/bin/
+```
+
+Note:
+* Tests using cvc5 and delphi are likely to fail.  If you just cut and paste the build commands into your shell, this failure may inhibit running the last two commands, so just cut and paste the last two commands into your shell again.
+
+??? hint "Troubleshoot: Confirm that UCLID5 is correctly installed on your machine."
+    ```shell
+    uclid --help
+    ```
+
+    If you get `uclid` command not found error, most likely, you need to add the path to `uclid` in your `PATH`.
+
+### [Step 6] Install PVerifier
+
+The following steps will build P with PVerifier by running the regular P build on the PVerifier branch of the repository.
+
+```sh
+cd ~
+git clone https://github.com/p-org/P
+cd P
+git checkout dev_p3.0/pverifier
+root=$(pwd)
+cd $root/Bld
+./build.sh
+dotnet tool uninstall --global P
+cd $root/Src/PCompiler/PCommandLine
+dotnet pack PCommandLine.csproj --configuration Release --output ./publish -p:PackAsTool=true -p:ToolCommandName=P -p:Version=2.1.3
+dotnet tool install P --global --add-source ./publish
+```
+
+??? hint "Troubleshoot: Confirm that PVerifier is correctly installed on your machine."
+    ```shell
+    p --version
+    ```
+
+    If you get `p` command not found error, most likely, you need to add the path to `p` in your `PATH`.
diff --git a/Docs/docs/advanced/PVerifierLanguageExtensions/outline.md b/Docs/docs/advanced/PVerifierLanguageExtensions/outline.md
@@ -0,0 +1,24 @@
+!!! tip ""
+    **We recommend that you start with the [Tutorials](tutsoutline.md) to get familiar with
+    the P language and its tool chain.**
+
+??? note "PVerifier Extension Top Level Declarations Grammar"
+
+    ```
+    topDecl:                # Top-level P Program Declarations
+    | pureFunDecl           # PureFunctionDeclaration
+    | initCondDecl          # InitConditionPredicateDeclaration
+    | invariantDecl         # InvariantDeclaration
+    | lemmaDecl             # LemmaDeclaration
+    | proofScript           # ProofScript
+    ;
+    ```
+
+A PVerifier program consists P top-level declarations along with the following:
+
+| Top Level Declarations                       | Description                                                                                                                             |
+| :------------------------------------------- | :-------------------------------------------------------------------------------------------------------------------------------------- |
+| [Pure Functions](pure.md)                    | P supports declaring pure functions that do not have side effects                                                                               |
+| [Init Conditions](init-condition.md)         | P supports declaring initial condition predicates                                                                                               |
+| [Invariants](specification.md)               | P supports declaring invariants that must hold true for the system                                                                              |
+| [Lemmas and Proofs](proof.md)                | P supports declaring lemmas and proof scripts to verify the correctness of the system                                                           |
diff --git a/Docs/docs/advanced/PVerifierLanguageExtensions/proof.md b/Docs/docs/advanced/PVerifierLanguageExtensions/proof.md
@@ -0,0 +1,97 @@
+# Lemmas and Proof Scripts
+
+Lemmas and proof scripts go hand in hand in the P Verifier. Lemmas allow you to decompose specifications and proof scripts allow you to relate lemmas to write larger proofs.
+
+## Lemmas
+
+Lemmas in P allow you to group related invariants together, which helps organize complex proofs, create smaller and more stable verification queries, and enable proof caching.
+
+??? note "P Lemma Declaration Grammar"
+
+    ```
+    lemmaDecl :
+        | Lemma iden { invariantsList }    # P Lemma Declaration
+
+    invariantsList :
+        | invariant iden: expression;
+        | invariantsList invariant iden: expression;
+    ```
+
+    `iden` is the name of the lemma or invariant, and `expression` is a boolean expression that should hold throughout system execution.
+
+**Syntax:** `Lemma lemmaName { invariant invName1: expr1; invariant invName2: expr2; ... }`
+
+`lemmaName` is the name of the lemma group, `invNameX` are the names of individual invariants, and `exprX` are the boolean expressions that should hold.
+
+=== "Lemma Declaration"
+
+    ```java
+    Lemma system_config {
+        invariant one_coordinator: forall (m: machine) :: m == coordinator() <==> m is Coordinator;
+        invariant participant_set: forall (m: machine) :: m in participants() <==> m is Participant;
+        invariant never_commit_to_coordinator: forall (e: event) :: e is eCommit && e targets coordinator() ==> !inflight e;
+        // More invariants...
+    }
+    ```
+
+## Proofs
+
+In P's verification framework, **proofs** provide a way to structure verification tasks by specifying what to verify and which lemmas to use. Proof scripts help decompose complex verification problems into smaller, more manageable parts and enable caching of intermediate results.
+
+??? note "P Proof Declaration Grammar"
+
+    ```
+    proofDecl :
+        | Proof { proofStmtList }    # P Proof Declaration
+
+    proofStmtList :
+        | proofStmt;
+        | proofStmtList proofStmt;
+
+    proofStmt :
+        | prove iden;                # Prove a lemma or invariant
+        | prove iden using iden;     # Prove using another lemma
+    ```
+
+    `iden` is the name of a lemma or invariant that should be verified.
+
+**Syntax:** `Proof { prove target1; prove target2 using helper; ... }`
+
+Where `targetN` are the names of lemmas or invariants to verify, and `helper` is an optional lemma to use during verification.
+
+=== "Basic Proof"
+
+    ```java
+    Proof {
+        prove system_config;         // Verify system_config lemma
+        prove default using system_config;  // Verify default P proof obligations using system_config
+    }
+    ```
+
+=== "Complex Proof With Dependencies"
+
+    ```java
+    Proof {
+        prove system_config;         // First prove the system configuration lemma
+        prove kondo using system_config;  // Use system_config to prove kondo
+        prove safety using kondo;    // Use kondo to prove the safety property
+        prove default using system_config;  // Verify default P obligations
+    }
+    ```
+
+=== "Using Default Keyword"
+
+    ```java
+    Proof {
+        prove lemma1;
+        prove lemma2;
+        // The special keyword "default" refers to P's built-in specifications
+        prove default using lemma1, lemma2;  // Verify using both lemmas
+    }
+    ```
+
+### Benefits of Proof Scripts
+
+1. **Organization**: Break down complex proofs into manageable parts
+2. **Verification Stability**: They enable the verifier to construct smaller, more focused queries
+3. **Caching**: Results are cached per proof step, avoiding redundant verification across runs
diff --git a/Docs/docs/advanced/PVerifierLanguageExtensions/pure.md b/Docs/docs/advanced/PVerifierLanguageExtensions/pure.md
@@ -0,0 +1,23 @@
+??? note "P Pure Function Declaration Grammar"
+
+    ```
+    pureFunctionDecl :
+        | pure iden (params)? : type;     # P Pure Function Declaration
+    ```
+
+    `iden` is the name of the pure function, `params` are the parameters of the function, and `type` is the return type of the function.
+
+**Syntax:** `pure functionName();` or `pure functionName(param1: type1, param2: type2) : returnType;`
+
+`functionName` is the name of the P pure function, `param1`, `param2`, etc. are the parameters of the function, and `returnType` is the type of the value returned by the function.
+
+=== "Pure Function Declarations"
+
+    ``` java
+    // declaration of pure functions with no parameters
+    pure participants(): set[machine];
+    pure coordinator(): machine;
+
+    // declaration of pure functions with parameters
+    pure preference(m: machine) : bool;
+    ```