Custom Metadata Feature

Closes #4409

Overview

Introduces custom metadata feature for test cases and user keywords. The implementation allows users to define metadata for both test cases and keywords using the [CustomName] syntax.

Implementation Details

• Custom Metadata Syntax:
  • Any setting in square brackets (e.g., [Owner], [Requirement], [My Custom Metadata]) is now treated as custom metadata.
  • Custom metadata is supported in both test cases and user keywords.
  • Metadata values can be single or multi-valued, and support variable resolution.
• Parsing and Output:
  • Custom metadata is parsed and stored in the test/keyword model.
  • Custom metadata is included in the output XML and HTML reports by default.
  • Output and parsing logic was adapted to treat all unknown square bracket settings as custom metadata.
• Backwards Compatibility:
  • Test cases that previously expected errors for undefined square bracket settings (e.g., [UndefinedSetting]) will now treat these as valid custom metadata.

CLI Functionality: --custommetadata Flag

• Purpose:
  • The --custommetadata flag allows users to filter which custom metadata keys are considered "valid" for display and parsing in output files.
• Behavior:
  • No flag set: All custom metadata keys are valid and included in output and parsing.
  • Flag set with one or more keys: Only the specified custom metadata keys are considered valid and included in output and parsing. Others are ignored. Test cases and user keywords do not fail.
  • Important: The flag does not influence the execution of tests or keywords. It only affects which custom metadata is displayed in output or parsed through execution.

Test Coverage

• Acceptance Tests:
  • Acceptance tests were created to cover major scenarios:
    • Basic custom metadata parsing
    • Edge cases (empty values, special characters, Unicode, etc.)
    • Performance (many metadata entries)
    • Variable resolution and built-in expressions
    • CLI filtering with --custommetadata
    • Output and report validation
• Unit Tests:
  • Unit tests ensure parsing, model storage, and output generation for custom metadata.

Notable Changes

• Error Handling:
  • Square bracket settings that do not match known settings are now always treated as custom metadata. This means tests expecting errors for undefined settings will no longer fail; instead those settings are parsed as metadata.

Summary

This feature allows users to create tests and keywords with metadata they need. Custom metadata values support the same formatting as settings metadata (*bold*, _italic_, `code`), variable resolution, multi-line content, etc. The --custommetadata flag provides control over output, without affecting test execution.

Usage Examples

Basic Custom Metadata

*** Test Cases ***
User Login Test
    [Documentation]    Test user authentication functionality
    [Owner]             Robot
    [Requirement]      REQ-001
    [Priority]         High
    [Component]        Authentication
    Log    Testing user login

Variable Resolution

*** Variables ***
${BUILD_NUMBER}       2024.1.0
${TEST_ENVIRONMENT}   ${{'Development' if os.getenv('CI') is None else 'CI/CD'}}
${TEAM_EMAIL}         [email protected]

*** Test Cases ***
API Integration Test
    [Documentation]    Test API integration with dynamic metadata
    [Build Version]    ${BUILD_NUMBER}
    [Environment]      ${TEST_ENVIRONMENT}
    [Contact]          ${TEAM_EMAIL}
    [Created Date]     ${{'%Y-%m-%d' | strftime}}
    Log    Test API integration

Text Formatting

*** Test Cases ***
Formatted Metadata Test
    [Documentation]    Test with text formatting
    [Description]      *Critical* test for _user registration_ flow
    [Steps]       1. Navigate to registration page
    ...                2. Fill form with _valid test data_
    ...                3. Submit and verify *success message*
    [Bug Report]       BUG-123: See ``validate_user()`` function
    [Test Data]        File: ``test_users.json`` with *valid* entries
    [Requirements]     REQ-001    REQ-002    REQ-003
    Log    Test with formatted metadata

Inline Python Evaluation

*** Test Cases ***
Dynamic Metadata Test
    [Documentation]    Test with dynamically calculated metadata
    [System Info]      ${{platform.system()}} ${{platform.release()}}
    [Python Version]   ${{sys.version.split()[0]}}
    [Timestamp]        ${{datetime.now().isoformat()}}
    [File Count]       ${{len(os.listdir('/tmp'))}} files in temp
    Log    Test with dynamic metadata

Keyword Metadata

*** Keywords ***
Database Connection
    [Documentation]    Establish database connection with metadata
    [Owner]            DB Team
    [Timeout]          ${{'30s' if os.getenv('FAST_MODE') else '60s'}}
    [Dependencies]     [http://www.example.com|Example Dependency]
    [Performance]      *Optimized* for _high-load_ scenarios
    [Notes]            Connection enabled with ``max_connections=50``
    ...
    ...                Auto-retry on connection failure
    Log    Connecting to database
    No Operation

CLI Filtering Examples

# Include all custom metadata (default)
robot tests.robot

# Only include 'Owner' and 'Priority' metadata in output
robot --custommetadata Owner --custommetadata Priority tests.robot

# Only include 'Bug Report' and 'Test Data' metadata in output
robot --custommetadata "Bug Report" --custommetadata "Test Data" tests.robot