MessagePiece and Message#

One of the most fundamental data structures in PyRIT is MessagePiece and Message. These classes provide the foundation for multi-modal interaction tracking throughout the framework.

        flowchart
   subgraph Conversation: 001
      subgraph Message: sequence 2
         subgraph "MessagePiece: <br>sequence: 2<br>conversation_id: 001<br>role: assistant<br>value: The image shows a wave ..."
         end
      end
      subgraph Message: sequence 1
         subgraph "MessagePiece: <br>sequence: 1<br>conversation_id: 001<br>role: user<br>value: tell me what's in this image"
         end
         subgraph "MessagePiece: <br>sequence: 1<br>conversation_id: 001<br>role: user<br>value: data/wave.png"
         end
      end
      subgraph Message: sequence 0
         subgraph "MessagePiece: <br>sequence: 0<br>conversation_id: 001<br>role: system<br>value: be a helpful assistant"
         end
      end
   end
    

SeedPrompts#

SeedPrompt objects represent the starting points of conversations. They are used to assemble and initiate attacks, and can be translated to and from MessagePieces.

Key Fields:

• value: The actual prompt text or file path

• data_type: Type of data (e.g., text, image_path, audio_path)

• name: Name of the prompt

• dataset_name: Name of the dataset this prompt belongs to

• harm_categories: Categories of harm associated with this prompt

• description: Description of the prompt’s purpose or content

• parameters: Template parameters that can be filled in dynamically

• prompt_group_id: Groups related prompts together

• role: Role in conversation (e.g., user, assistant)

• metadata: Arbitrary metadata that can be attached

SeedPrompts support Jinja2 templating, allowing dynamic prompt generation with parameter substitution. They can be loaded from YAML files and organized into datasets and groups for systematic testing.

SeedObjectives#

SeedObjective objects represent the goal or objective of an attack or test scenario. They describe what the attacker is trying to achieve and are used alongside SeedPrompts to define complete attack scenarios.

Key Fields:

• value: The objective statement describing the goal (e.g., “Generate hate speech content”)

• data_type: Always text for objectives

• name: Name identifying the objective

• dataset_name: Name of the dataset this objective belongs to

• harm_categories: Categories of harm the objective relates to

• authors: Attribution information for the objective

• groups: Group affiliations (e.g., “AI Red Team”)

• source: Source or reference for the objective

• metadata: Additional metadata about the objective

SeedObjectives support Jinja2 templating for dynamic objective generation and can be loaded from YAML files alongside prompts, making it easy to organize and reuse test objectives across different scenarios.

Relationship to SeedGroups:

SeedObjective and SeedPrompt objects are combined into SeedGroup objects, which represent a complete test case with optional seed prompts and an objective. A SeedGroup can contain:

• Multiple prompts (for multi-turn conversations)

• A single objective (what the attack is trying to achieve)

• Both prompts and an objective (complete attack specification)

Scores#

Score objects represent evaluations of prompts or responses. Scores are generated by scorer components and attached to MessagePieces to track the success or characteristics of attacks. When a prompt is scored, it is added to the database and can be queried later.

Key Fields:

• score_value: The actual score (e.g., "true", "0.75")

• score_value_description: Human-readable description of the score

• score_type: Type of score (true_false or float_scale)

• score_category: Categories the score applies to (e.g., ["hate", "violence"])

• score_rationale: Explanation of why the score was assigned

• scorer_class_identifier: Information about the scorer that generated this score

• message_piece_id: The ID of the piece/response being scored

• task: The original attacker’s objective being evaluated

• score_metadata: Custom metadata specific to the scorer

Scores enable automated evaluation of attack success, content harmfulness, and other metrics throughout PyRIT’s red teaming workflows.

AttackResults#

AttackResult objects encapsulate the complete outcome of an attack execution, including metrics, evidence, and success determination. When an attack is run, the AttackResult is added to the database and can be queried later.

Key Fields:

• conversation_id: The conversation that produced this result

• objective: Natural-language description of the attacker’s goal

• attack_identifier: Information identifying the attack strategy used

• last_response: The final MessagePiece generated in the attack

• last_score: The final score assigned to the last response

• executed_turns: Number of turns executed in the attack

• execution_time_ms: Total execution time in milliseconds

• outcome: The attack outcome (SUCCESS, FAILURE, or UNDETERMINED)

• outcome_reason: Optional explanation for the outcome

• related_conversations: Set of related conversation references

• metadata: Arbitrary metadata about the attack execution

AttackResult objects provide comprehensive reporting on attack campaigns, enabling analysis of red teaming effectiveness and vulnerability identification.