Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@Integralist
Copy link
Collaborator

@Integralist Integralist commented May 4, 2023

This PR introduces both breaking changes and the following features for bulk importing data into the Fastly KV Store.

The breaking changes introduced in this PR are the rename of the --key-name flag to --key to be consistent with the other 'stores' supported within the CLI.

NOTE: The first two approaches require the value to be base64 encoded as they are being streamed to a Fastly 'batch' API endpoint. The last approach (directory structure) makes individual "set-value-for-key" calls that are processed concurrently.

# stdin
echo '{"key":"example","value":"VkFMVUU="}' | fastly kv-store-entry create -s olwenufofbiklsaaz4i1l8 --stdin

# file
#
# {
#   "key": "file-example-1",
#   "value": "VkFMVUU="
# }
# {
#   "key": "file-example-2",
#   "value": "VkFMVUU="
# }
fastly kv-store-entry create -s olwenufofbiklsaaz4i1l8 --file data.json

# directory
# filename == key, file contents == value
fastly kv-store-entry create -s olwenufofbiklsaaz4i1l8 --dir ./data/

Each example is documented in pkg/app/metadata.json.

Screenshots

Example of successful insertion:

Screenshot 2023-05-09 at 15 58 25

Example of failed insertion:

Screenshot 2023-05-09 at 15 58 41

@Integralist Integralist added the enhancement New feature or request label May 4, 2023
@Integralist Integralist force-pushed the integralist/kv-bulk branch from 3f325aa to a311ecc Compare May 9, 2023 12:57
@Integralist Integralist force-pushed the integralist/kv-bulk branch from fc89164 to 81d37d4 Compare May 9, 2023 13:18
@Integralist Integralist force-pushed the integralist/kv-bulk branch from 81d37d4 to 1cc2579 Compare May 9, 2023 13:19
@Integralist
Copy link
Collaborator Author

@awilliams-fastly This is draft for the moment just while I wait for the batch API endpoint to be documented on https://developer.fastly.com/reference/api/ (there is an internal PR for that work). Once it's documented, then I can look to implement the endpoint in the go-fastly API client, and once in a published release I can replace the undocumented.Call() with an actual API client method.

But for the time being, you might be able to assist with a quick review.

Thanks!

@Integralist Integralist force-pushed the integralist/kv-bulk branch from 5f527e6 to eb79c00 Compare May 10, 2023 15:57
Copy link
Collaborator

@awilliams-fastly awilliams-fastly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

There's just a the few remaining comments, but the overall change looks good.

@Integralist
Copy link
Collaborator Author

Thanks @awilliams-fastly

I think I just need to write some tests for the new bulk import logic and then I'm done.

@Integralist Integralist force-pushed the integralist/kv-bulk branch from 348470b to 4a45932 Compare May 15, 2023 11:21
@Integralist Integralist force-pushed the integralist/kv-bulk branch 5 times, most recently from eda545d to 35e1356 Compare May 15, 2023 12:57
Was unable to get checking the attributes of a file to work on Windows
platform in the CI environment.
@Integralist Integralist force-pushed the integralist/kv-bulk branch from 35e1356 to 2233c11 Compare May 15, 2023 13:15
@Integralist Integralist force-pushed the integralist/kv-bulk branch from 48e68e9 to 2442257 Compare May 15, 2023 13:49
@Integralist Integralist merged commit 413c8c2 into main May 15, 2023
@Integralist Integralist deleted the integralist/kv-bulk branch May 15, 2023 15:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants