Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@akash4393
Copy link
Contributor

@akash4393 akash4393 commented Feb 11, 2025

Static Mocking in Wiretap

Table of Contents

Overview

This feature allows static mocking of APIs in the Wiretap service by defining mock definitions in JSON files. It enables the server to match incoming requests against predefined mock definitions and return corresponding mock responses. If no match is found, the request is forwarded to the Wiretap's httpRequestHandler for further processing.

How to Enable Static Mocking

To enable static mocking, you need to set the --static-mock-dir argument to a directory path when starting Wiretap, or configure it in the Wiretap configuration file.

Example:

wiretap --static-mock-dir /path/to/mocks

When this path is set, Wiretap will expect mock definitions and response body JSON files in the following structure:

  • /path/to/mocks/mock-definitions/ — Contains the mock definition JSON files.
  • /path/to/mocks/body-jsons/ — Contains the response body JSON files.

The static mock service will start and load all the mock definitions found in /path/to/mocks/mock-definitions.

Mock Definitions

Mock definitions are JSON objects or arrays of objects that define the request and response structure. Each JSON object should contain the following keys:

  • request — Specifies the conditions for the request.
  • respose — Specifies the response that should be returned when the request matches the conditions.

Request Definition

The request definition is parsed into the following Go type:

type StaticMockDefinitionRequest struct {
	Method      string          `json:"method,omitempty"`
	UrlPath     string          `json:"urlPath,omitempty"`
	Host        string          `json:"host,omitempty"`
	Header      *map[string]any `json:"header,omitempty"`
	Body        interface{}     `json:"body,omitempty"`
	QueryParams *map[string]any `json:"queryParams,omitempty"`
}

Each field can use either a string or a regex string to match the actual request. For example, the header, body, and queryParams fields can contain regex patterns to match the incoming request.

Example Request Definition:

{
	"method": "GET",
	"urlPath": "/test",
	"header": {
		"Content-Type": "application.*"
	},
	"queryParams": {
		"test": "ok",
		"arr": ["1", "2"]
	},
	"body": {
		"test": "o.*"
	}
}

Response Definition

The response definition is parsed into the following Go type:

type StaticMockDefinitionResponse struct {
	Header           map[string]any `json:"header,omitempty"`
	StatusCode       int            `json:"statusCode,omitempty"`
	Body             string         `json:"body,omitempty"`
	BodyJsonFilename string         `json:"bodyJsonFilename,omitempty"`
}
  • BodyJsonFilename: The name of a file in the body-jsons folder, which contains the response body JSON. If this is specified, Wiretap will return the content of that file instead of using the body field.

Example Response Definition with Inline Body:

{
	"statusCode": 200,
	"header": {
		"something-header": "test-ok"
	},
	"body": "{\"test\": \"${queryParams.arr.[1]}\"}"
}

In this example, the response body uses a reference to the request's query parameters.

Example Response Definition with Body from File:

{
	"statusCode": 200,
	"header": {
		"something-header": "test-ok"
	},
	"bodyJsonFilename": "test.json"
}

In this example, Wiretap will look for a file named test.json in the body-jsons folder and return its content as the response body.

Response Generation Using Request Data

The response body can dynamically generate values based on the request. This is done by using the request's fields (such as queryParams, body, etc.) in the response body.

For example:

{
	"statusCode": 200,
	"header": {
		"something-header": "test-ok"
	},
	"body": "{\"test\": \"${queryParams.arr.[1]}\"}"
}

In this case, the response body will include the second element from the arr query parameter in the incoming request. The ${} syntax is used to refer to the request's fields.

Directory Structure

The --static-mock-dir should point to a directory that contains the following subdirectories and files:

/path/to/mocks/
  ├── mock-definitions/
  │     ├── mock1.json
  │     ├── mock2.json
  │     └── ...
  └── body-jsons/
        ├── test.json
        └── ...
  • mock-definitions/: Contains the mock request and response definitions.
  • body-jsons/: Contains the actual response body JSON files referenced by the mock definitions.

Example

  1. Directory Structure:
/mocks/
  ├── mock-definitions/
  │     ├── get-test-mock.json
  └── body-jsons/
        ├── test.json
  1. get-test-mock.json:
{
  "request": {
    "method": "GET",
    "urlPath": "/test",
    "header": {
      "Content-Type": "application.*"
    },
    "queryParams": {
      "test": "ok",
      "arr": ["1", "2"]
    },
    "body": {
      "test": "o.*"
    }
  },
  "response": {
    "statusCode": 200,
    "header": {
      "something-header": "test-ok"
    },
    "bodyJsonFilename": "test.json",
  }
}
  1. test.json:
{
  "test": "${queryParams.arr.[1]}"
}

With this configuration, when Wiretap receives a GET request to /test, it will respond with the content of test.json.

Notes

  • If no mock definition is found that matches an incoming request, Wiretap will forward the request to the wiretap's request handler and let it return a response.
  • The mock definitions can contain either a single object or an array of objects. In the case of an array, each object represents a separate mock definition.

@akash4393 akash4393 marked this pull request as draft February 11, 2025 05:39
@akash4393 akash4393 force-pushed the feat/Add-Support-Static-Mocking branch from 36076d2 to 3e3d4bc Compare February 18, 2025 23:32
@akash4393 akash4393 marked this pull request as ready for review February 19, 2025 00:43
@akash4393 akash4393 changed the title [DO NOT MERGE] WIP: Added new static mock service and basic logic for routing static mock requests Added support for static mocking (mocks defined in JSON files) Feb 19, 2025
@serranoio
Copy link
Contributor

where da tests at? 👀

@akash4393
Copy link
Contributor Author

where da tests at? 👀

Will be the next task, but need this right now to unblock the team that wants to use this feature.

Copy link
Contributor

@serranoio serranoio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. do you pinky winky promise to write tests eventually? Open source needs love <3
  2. Path level static-mocking should be implemented eventually
  3. 🤝 let's freaking go. good shit. If ur using this for whichever job you work at, then im assuming that youre not writing garbage code. This passes the eyeball test 👍

Comment on lines +99 to +100
if isPotentialRegex(patternOrStr) {
if _, err := regexp.Compile(patternOrStr); err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cant you remove line 99 if you can error out on line 100?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, line 99 is to check if it is a regex at all. Otherwise compare them as strings. If I remove line 99 then it will always try to compile the mock string as regex and even if the string was "ok" it will create a regex with that and that will match a different string with the incoming request such as "okay".

This way, if you "ok" it will not create a regex since it has no regex characters and do a string match.

Comment on lines +75 to +81
staticMockService := staticMock.NewStaticMockService(wtService, wiretapConfig.Logger)
// register Static-Mock Service
if err = platformServer.RegisterService(
staticMockService, staticMock.StaticMockServiceChan); err != nil {
panic(err)
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the ranch makes it so easy to add new features 🙏

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, this is all code for async APIs, under the covers ranch is wiring up this service against a message bus (that is what the channel is for). You can then talk to this service using pub/sub over a websocket.

Comment on lines +30 to +35
// if static-mock-dir is set, then we call the handler of staticMockService
if len(wiretapConfig.StaticMockDir) != 0 {
staticMockService.HandleStaticMockRequest(requestModel)
} else { // else call the wiretap service handler
wtService.HandleHttpRequest(requestModel)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should make static mocking path available for path globs. This way, one singular wiretap server can serve static mocks, openapi generated mocks, and also be a proxy ;)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is still a single server. This doesn't create a new server. We only add a new service that follows a different codepath when static mocking is enabled.

This way, one singular wiretap server can serve static mocks, openapi generated mocks, and also be a proxy ;)

This is already happening. :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Im having trouble understanding. According to the code, if staticMockDir is set, then no matter what, every single http request will go to the staticMockHandler. Right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but if there are no matches, it will be forwarded "back" to wiretapService.HandleHttpRequest. Look at line 195 in staticMock/handle_static_mock_request.go.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

@daveshanley daveshanley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! and you have practically written all the docs too! A few nits for you, but overall, looking great!

"github.com/pb33f/ranch/model"
"github.com/pb33f/wiretap/daemon"
"github.com/pb33f/wiretap/shared"
"github.com/pb33f/wiretap/staticMock"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: please use kebab-case for modules.

)

func handleHttpTraffic(wiretapConfig *shared.WiretapConfiguration, wtService *daemon.WiretapService) {
func handleHttpTraffic(wiretapConfig *shared.WiretapConfiguration, wtService *daemon.WiretapService, staticMockService *staticMock.StaticMockService) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have started going backwards and adding in a message based design where I can for functions that need a growing number of deps, It would make sense to wrap these items into a HandleHttpTraffic struct to contain all the things, and then it can grow without any breaking changes.

Comment on lines +75 to +81
staticMockService := staticMock.NewStaticMockService(wtService, wiretapConfig.Logger)
// register Static-Mock Service
if err = platformServer.RegisterService(
staticMockService, staticMock.StaticMockServiceChan); err != nil {
panic(err)
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, this is all code for async APIs, under the covers ranch is wiring up this service against a message bus (that is what the channel is for). You can then talk to this service using pub/sub over a websocket.


// boot the http handler
handleHttpTraffic(wiretapConfig, wtService)
handleHttpTraffic(wiretapConfig, wtService, staticMockService)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use a message based struct here, just so we can expand it later without breaking anything.

"github.com/pterm/pterm"
)

// Function to check if json is a subset of superSet
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use godoc style for function comments
(// FunctionName does something)

return data, nil
}

// Function to replace template variables in JSON path format
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same nit as others on the godoc.

// Function to replace template variables in JSON path format
func ReplaceTemplateVars(jsonStr string, vars interface{}) (string, error) {
// Regular expression to match the ${var} format (full path)
re := regexp.MustCompile(`\$\{([a-zA-Z0-9._\[\]]+)\}`)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's always a good idea to pre-compile all regex OUTSIDE of the function it's called from. Why? it's REALLY SLOW to compile regex in realtime, define this as a module variable, compile it using Init() and then every time the ReplaceTemplateVars is called, it will run s fast as possible.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I defined them as vars globally, that should compile them on load right? Let me know if that makes sense. I wasn't sure how to do the Init() method approach.


// If the BodyJsonPath is defined then set the body to contents of the file
if matchedMockDefinition.Response.BodyJsonFilename != "" {
bodyJsonFilePath := sms.wiretapService.StaticMockDir + "/body-jsons/" + matchedMockDefinition.Response.BodyJsonFilename
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would define this as a const "/body-jsons/"

return shared.IsSubset(mock.Body, incomingBody)
}

// Function to transform []string values to []interface{}(string)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

godoc nits in here (all through the file)


default:
// If it's neither an object nor an array
logger.Error("JSON not in the right format.")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we're logging the error, we should log the JSON too.

Copy link
Member

@daveshanley daveshanley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thank you very much for your contribution, this adds a lot of value to the project.

@daveshanley daveshanley merged commit 1d082b4 into pb33f:main Feb 20, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants