Adds architecture, concepts, roadmap details #2
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ef35201 to
4630a16
Compare
NNcrawler
reviewed
Apr 21, 2021
|
|
||
| Considering that kafka is set up in a clustered, cross-region, cross-zone environment, the chances of it going down are mostly unlikely. In case if it does, the amount of events lost is negligible considering it is a streaming system and is expected to forward millions of events/sec. | ||
|
|
||
| When an EventRequest proto below containing events are sent over the wire |
Member
There was a problem hiding this comment.
Suggested change
| When an EventRequest proto below containing events are sent over the wire | |
| When an [EventRequest proto](https://github.com/odpf/proton/blob/main/odpf/raccoon/EventRequest.proto) below containing events are sent over the wire |
| } | ||
| ``` | ||
|
|
||
| a corresponding EventResponse is sent by the server on the same connection that the events were consumed. |
Member
There was a problem hiding this comment.
Suggested change
| a corresponding EventResponse is sent by the server on the same connection that the events were consumed. | |
| a corresponding [EventResponse](https://github.com/odpf/proton/blob/main/odpf/raccoon/EventResponse.proto) is sent by the server on the same connection that the events were consumed. |
| ``` | ||
| EVENT_DISTRIBUTION_PUBLISHER_PATTERN=topic-%s-log | ||
| ``` | ||
| and a type such as ```type=viewed``` in the event format |
Member
There was a problem hiding this comment.
Suggested change
| and a type such as ```type=viewed``` in the event format | |
| and a type such as ```type=viewed``` in the [event](https://github.com/odpf/proton/blob/main/odpf/raccoon/Event.proto) format |
NNcrawler
pushed a commit
that referenced
this pull request
May 26, 2021
Adds architecture, concepts, roadmap details
turtleDev
added a commit
that referenced
this pull request
Sep 5, 2024
ravisuhag
pushed a commit
that referenced
this pull request
Sep 10, 2024
* publisher: log: add tests * worker: test for ack function * worker: tests: remove unused mocks * worker: add tests for error cases * worker: test: use constructor for tests * app: worker: fix flush always timing out * app: move kafka specific log messages to kafka publisher package * wip: worker metrics test * worker: add tests against instrumented metrics * worker: rename timeSource to Clock * chore: remove client example * worker: use idiomatic null device for tests * config: add tests for validation errors * config: add test for valid configuration * config: add tests for internal prepare() * config: add tests for Walk() * config: kafka: add tests for ToConfigMap() * config: kafka: add more tests for config map serializer * config: add tests for internal helper functions * refactor: extract Clock to a stand alone package * collector: add tests for TimePushed mutation * serialization: add tests for SerialiseProto * deserialization: add tests for DeserializeFunc * worker: add test for flush time out * config: refactor ack value names * tests: integration e2e tests results in coverage data * ci: test: fix coverage merge failing * ci: prospective fix for coverdata #1 * ci: prospective fix for coverdata #2 * ci: prospective fix for coverdata #3 * ci: upload test coverage as artifact * ci: refactor Docker build for release and testing * services: grpc: add tests for synchronous event handling * services: grpc: add tests to cover metric instrumentation * services: rest: remove deadcode For an incoming request, req.Body is never nil https://pkg.go.dev/net/http#Request * services: rest: add tests for error reading request body * services: rest: add tests for malformed request body * services: rest: refactor ack handling * services: rest: add tests for different error paths * ci: cache image pulls to speed up tests * ci: fix docker image caching * ci: ignore proto package during tests * ci: remove docker image caching * Revert "ci: ignore proto package during tests" This reverts commit 75d0807. * Revert "Revert "ci: ignore proto package during tests"" This reverts commit 1038f46. * Revert "Revert "Revert "ci: ignore proto package during tests""" This reverts commit a39cad6. * publisher: log: move test event to root proto package * ci: blacklist proto package during tests * hack: publisher: log: use single spaced fields * Revert "publisher: log: move test event to root proto package" This reverts commit 7f837bf. * Revert "ci: blacklist proto package during tests" This reverts commit 0856073. * Revert "hack: publisher: log: use single spaced fields" This reverts commit 5ffb846. * Revert "Revert "hack: publisher: log: use single spaced fields"" This reverts commit 67797d9. * ci: ignore proto folders when computing coverage * hack: publisher: log: use double spaced fields * ci: fix coverage data merge * ci: fix cover merge resulting in empty coverage file * publish: kinesis: add tests for stream's existence check * misc: improve test case description * publisher: kinesis: add tests for hitting rate limits and quotas * publisher: kinesis: refactor tests
ravisuhag
pushed a commit
that referenced
this pull request
Sep 12, 2024
* publisher: log: add tests * worker: test for ack function * worker: tests: remove unused mocks * worker: add tests for error cases * worker: test: use constructor for tests * app: worker: fix flush always timing out * app: move kafka specific log messages to kafka publisher package * wip: worker metrics test * worker: add tests against instrumented metrics * worker: rename timeSource to Clock * chore: remove client example * worker: use idiomatic null device for tests * config: add tests for validation errors * config: add test for valid configuration * config: add tests for internal prepare() * config: add tests for Walk() * config: kafka: add tests for ToConfigMap() * config: kafka: add more tests for config map serializer * config: add tests for internal helper functions * refactor: extract Clock to a stand alone package * collector: add tests for TimePushed mutation * serialization: add tests for SerialiseProto * deserialization: add tests for DeserializeFunc * worker: add test for flush time out * config: refactor ack value names * tests: integration e2e tests results in coverage data * ci: test: fix coverage merge failing * ci: prospective fix for coverdata #1 * ci: prospective fix for coverdata #2 * ci: prospective fix for coverdata #3 * ci: upload test coverage as artifact * ci: refactor Docker build for release and testing * services: grpc: add tests for synchronous event handling * services: grpc: add tests to cover metric instrumentation * services: rest: remove deadcode For an incoming request, req.Body is never nil https://pkg.go.dev/net/http#Request * services: rest: add tests for error reading request body * services: rest: add tests for malformed request body * services: rest: refactor ack handling * services: rest: add tests for different error paths * ci: cache image pulls to speed up tests * ci: fix docker image caching * ci: ignore proto package during tests * ci: remove docker image caching * Revert "ci: ignore proto package during tests" This reverts commit 75d0807. * Revert "Revert "ci: ignore proto package during tests"" This reverts commit 1038f46. * Revert "Revert "Revert "ci: ignore proto package during tests""" This reverts commit a39cad6. * publisher: log: move test event to root proto package * ci: blacklist proto package during tests * hack: publisher: log: use single spaced fields * Revert "publisher: log: move test event to root proto package" This reverts commit 7f837bf. * Revert "ci: blacklist proto package during tests" This reverts commit 0856073. * Revert "hack: publisher: log: use single spaced fields" This reverts commit 5ffb846. * Revert "Revert "hack: publisher: log: use single spaced fields"" This reverts commit 67797d9. * ci: ignore proto folders when computing coverage * hack: publisher: log: use double spaced fields * ci: fix coverage data merge * ci: fix cover merge resulting in empty coverage file * publish: kinesis: add tests for stream's existence check * misc: improve test case description * publisher: kinesis: add tests for hitting rate limits and quotas * publisher: kinesis: refactor tests * hygiene: reorganise package structure * refactor: merge serialization and deserialization pkg into serde * docs: fix broken dashboard link * serde: test: fix broken references * refactor: integrate middleware package into server/rest
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.