-
-
Notifications
You must be signed in to change notification settings - Fork 574
Add JSON-RPC 2.0 Transport Support to Goa #3734
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
86d1a1d to
b97d19e
Compare
- Added new templates for handling various code generation scenarios, including validation and transformation for different data types. - Refactored existing templates to improve readability and maintainability. - Updated the .golangci.yml configuration to include specific checks for static analysis. - Bumped dependencies in go.mod and updated go.sum accordingly. - Removed obsolete staticcheck configuration file. This commit enhances the code generation process and ensures better compliance with coding standards.
* Create JSON-RPC specific encoder/decoder
- Introduced `HandleStream` method for JSON-RPC WebSocket services. - Updated server templates to accommodate JSON-RPC endpoints and batch processing. - Enhanced error handling for WebSocket connections and JSON-RPC requests. - Added architecture documentation for JSON-RPC support. - Refactored existing code to integrate JSON-RPC functionality without modifying HTTP templates.
- Updated SSE client stream implementation to include a user-provided decoder function for handling complex response types. - Modified client endpoint initialization to pass the decoder to the stream. - Adjusted templates for JSON-RPC and standard endpoints to ensure compatibility with the new decoder functionality.
- Introduced Server-Sent Events (SSE) functionality for JSON-RPC, including new server and client stream implementations. - Added templates for SSE handling in both client and server contexts. - Created integration tests to validate SSE behavior and interactions. - Updated existing test scenarios to include SSE-specific cases and validation logic. - Enhanced the overall structure of the integration tests for better maintainability.
- Added integration of JSON-RPC with Server-Sent Events (SSE) and WebSocket streaming capabilities. - Updated endpoint templates to handle streaming payloads and structured responses for JSON-RPC. - Introduced new logic for determining when to generate endpoint input structs based on streaming types. - Enhanced validation and error handling for JSON-RPC methods, ensuring compliance with SSE and WebSocket protocols. - Refactored existing tests and added new scenarios to validate the updated JSON-RPC functionality.
…onse logic - Consolidated notification and response handling into a single `Send` method for SSE streams. - Updated templates to reflect the new event structure, ensuring proper handling of notifications and responses. - Enhanced error handling for SSE events, including a dedicated method for sending error responses. - Removed redundant notification and response methods to simplify the codebase and improve maintainability. - Adjusted integration tests to utilize the new `Send` method for event streaming.
- Updated integration test scenarios to improve organization and maintainability, including support for streaming and error handling. - Enhanced the framework for generating test data, allowing for more comprehensive testing of various transport protocols. - Refined templates and methods to ensure robust validation and error handling across different scenarios. - Removed deprecated files and redundant code to streamline the test suite. - Added new test cases to validate the updated JSON-RPC functionalities, ensuring thorough coverage of edge cases.
…ow; enhance golangci-lint settings and Makefile targets - Deleted the .deepsource.toml file and the report-coverage.yml workflow as part of the cleanup. - Updated .golangci.yml to include additional linters for improved code quality checks. - Modified the Makefile to include an integration-test target in the default build process. - Refactored code in various files to improve performance and maintainability, including preallocating slices and optimizing error handling.
- Updated validation test to use golden files for comparison instead of hardcoded values. - Enhanced error handling in HTTP handler functions to check for the presence of an error handler before invoking it. - Made adjustments to various test files to ensure consistent error handling and validation logic across different scenarios.
- Add missing UserTypeImports field to service.Data struct - Fix typeContext function calls (remove extra parameter) - Fix readTemplate calls to use serviceTemplates.Read pattern - Fix linting issues in convert.go (Index checks and slice append) - Rename golden files with spaces/special chars to be Windows-compatible - Update test names to match renamed golden files Fixes compilation errors and Windows CI build failures.
- Add build-goa target to Makefile that builds goa binary to GOPATH/bin - Make integration-test depend on build-goa to ensure binary is available - Update generator to check for locally built goa binary in GOPATH/bin - Add proper Windows .exe extension handling for cross-platform CI - Fallback strategy: GOA_BINARY env var -> GOPATH/bin/goa -> system PATH This ensures that JSON-RPC integration tests can find and use the goa binary during CI builds, fixing the Windows CI failure.
- Replace complex shell-based build with simple 'go install' in Makefile - Enhance getGoaBinary() to use 'go env GOBIN' for reliable binary detection - Add proper fallback chain: GOA_BINARY env var -> GOBIN -> GOPATH/bin -> PATH - Remove problematic shell command substitution that fails on Windows - Use Go's native cross-platform binary installation and detection This should resolve the Windows CI build failures by using Go's standard practices for binary management across platforms.
- Add extensive debug logging to goa binary detection process - Enhance getGoaBinary() with multiple fallback strategies: * GOBIN from 'go env' command * GOPATH from 'go env' command * Environment GOPATH variable * Windows-specific AppData locations * Default home directory go/bin - Add runtime binary verification and 'where'/'which' command fallback - Add emergency goa binary building if detection fails completely - Improve Makefile build-goa target with debug output - Handle Windows .exe extension throughout all detection paths This should provide comprehensive diagnostics for Windows CI failures and multiple robust fallback mechanisms to ensure goa binary is found.
- Remove unnecessary debug logging that cluttered output - Simplify ensureGoaBinary function without verbose logging - Keep robust binary detection and building logic - Fix unused variable error - Maintain clean, production-ready code The core functionality remains: automatically build goa binary if not found, with multiple fallback detection strategies.
Major improvements for Windows compatibility: - Enhanced binary detection with both 'where goa.exe' and 'where goa' - Use explicit 'go build -o' instead of 'go install' for predictable output - Add getGoBinDir() function with robust fallback chain - Create target directories if they don't exist (Windows permissions) - Use absolute paths for command working directories on Windows - Better error handling with detailed output for debugging This addresses Windows-specific path handling, binary naming (.exe), and permission issues that can cause CI failures.
The service.go.tpl template had incorrect indentation where tabs were replaced with inconsistent spacing, causing template parsing issues on Windows. Restored proper tab-based indentation to match the original format and maintain consistent template generation across platforms.
- Revert from go build -o back to go install for simplicity - Remove Windows-specific absolute path handling that wasn't needed - Remove unused normalizeLineEndings function - Keep core binary detection improvements The template indentation issue was the real cause of Windows CI failures, not the binary building approach. This keeps the solution clean and simple.
- Allow SSE with either ServerStreamKind or mixed results (different Result and StreamingResult types) - Fix SSE client/server template generation for mixed results - Update integration test framework to preserve test directory path for debugging - Ensure proper type references and interface generation for mixed results
Replace direct type assertion with errors.As to handle wrapped errors properly, as required by errorlint linter.
The templates were updated to support mixed results, which changed the generated code structure. This updates the golden files to match the new output.
The template changes for mixed results inadvertently broke payload decoding for regular WebSocket endpoints. This restores the original decoding logic for non-mixed streaming endpoints while preserving the new mixed results functionality.
The template fix for WebSocket endpoints changed the generated code structure, requiring updates to the golden test files.
The integration tests were failing with 'broken pipe' errors when closing WebSocket connections. This can happen when the server closes the connection before the client sends its close message, which is a normal race condition in network programming. This change ignores such errors during WebSocket close operations.
- Update codegen/sections_test.go to normalize line endings consistently - Remove redundant line ending normalization from grpc/codegen/proto_test.go as testutil.AssertString already handles it internally - Tests now properly handle Windows \r\n vs Unix \n differences
- Remove extra closing braces in validation templates that were causing double-nested if statements in generated code - Fix line ending handling in example_svc_test.go to properly trim both \r and \n - Templates now generate correct single-level nil checks for pointer fields
- Add line ending normalization (\r\n to \n) when reading template files - Ensures consistent template parsing across different platforms - Fixes Windows CI failures caused by different line ending handling in templates
- Replace direct string comparison with testutil.AssertString - Ensures consistent line ending handling across platforms - Fixes remaining Windows CI failures in WebSocket tests
- Augment PATH with GOBIN and GOPATH/bin - Explicitly pass --plugin flags for protoc-gen-go and protoc-gen-go-grpc when found - Handles Windows .exe resolution and PATH separators This prevents protoc failures on Windows runners where plugins are not on PATH by default.
The integration tests fail on Windows due to code generation issues with decode functions for WebSocket endpoints. Skipping these tests on Windows for now to unblock the PR while we investigate a proper fix.
…tforms" This reverts commit 01f53cd.
- Updated the title to reflect JSON-RPC 2.0 support. - Expanded the introduction to clarify Goa's capabilities for building RPC services. - Added a comprehensive table of contents for easier navigation. - Included detailed sections on core concepts, service definitions, transport options, and advanced features. - Provided code examples for defining services and methods, including error handling and streaming patterns. - Improved explanations of JSON-RPC message structures and request types. - Enhanced best practices and error handling guidelines for developers.
…nd vs SendAndClose; streamline Best Practices; expand batch processing
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
feat: Add JSON-RPC 2.0 Transport Support
This pull request introduces comprehensive support for JSON-RPC 2.0 in Goa, allowing services to be exposed over multiple transports including HTTP, WebSockets, and Server-Sent Events (SSE). This change is fully additive and introduces no breaking changes to existing services.
It enables developers to define RPC-style APIs using the same Goa DSL, generating type-safe clients, servers, and CLI tools with support for request/response, notifications, and bidirectional streaming.
Key Features
-32700to-32603).JSONRPCblock to services and methods, allowing it to coexist with REST in the same service.goa gencommand now generates all necessary server handlers, client code, and CLI tools for the enabled JSON-RPC transports.Design and Implementation
Single Endpoint Architecture
Each service exposes all its JSON-RPC methods on a single endpoint (e.g.,
/rpc), unlike REST which uses multiple URLs per resource. Themethodfield in the JSON-RPC request body is used for routing. This design offers several advantages:DSL Usage Example
A JSON-RPC endpoint can be added to a service with minimal changes to the design file:
Transport Selection
The framework selects the transport for a given method based on its definition:
StreamingResultthat are configured for SSE.Streaming Implementation
The generated code provides high-level interfaces for handling streams.
WebSocket (Bidirectional): The server stream interface allows for sending responses, notifications, and errors independently.
SSE (Server Push): The
Sendmakes it possible to send server initiated notifications while theSendAndClosemethod sends the final result.Testing Strategy
This feature is supported by a comprehensive test suite to ensure correctness and reliability.
Example Test Case in YAML
Backwards Compatibility
This is a purely additive change with no impact on backwards compatibility.
JSONRPC()block in their design will expose RPC endpoints.