Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@GBBBAS
Copy link
Collaborator

@GBBBAS GBBBAS commented Mar 30, 2023

No description provided.

GBBBAS and others added 29 commits February 27, 2023 14:50
* fixes Add Releases Feed to Docs page #68

Signed-off-by: GBBBAS <[email protected]>

* Mkdocs test fixes

Signed-off-by: GBBBAS <[email protected]>

* Add unit tests

Signed-off-by: GBBBAS <[email protected]>

* Fix Code Smell

Signed-off-by: GBBBAS <[email protected]>

---------

Signed-off-by: GBBBAS <[email protected]>
* Upgrade Packages

Signed-off-by: GBBBAS <[email protected]>

* Upgrade Functions Packages

Signed-off-by: GBBBAS <[email protected]>

* Add logos

Signed-off-by: GBBBAS <[email protected]>

* Add logo to pypi readme

Signed-off-by: GBBBAS <[email protected]>

* Remove pydantic models

Signed-off-by: GBBBAS <[email protected]>

* Remove GraphQL

Signed-off-by: GBBBAS <[email protected]>

* Remove strawberry type

Signed-off-by: GBBBAS <[email protected]>

* Remove GraphQL

Signed-off-by: GBBBAS <[email protected]>

* Upgrade GraphQL Package

Signed-off-by: GBBBAS <[email protected]>

* Logo Size Changes

Signed-off-by: GBBBAS <[email protected]>

---------

Signed-off-by: GBBBAS <[email protected]>
* Upgrade Packages

Signed-off-by: GBBBAS <[email protected]>

* Upgrade Functions Packages

Signed-off-by: GBBBAS <[email protected]>

* Add logos

Signed-off-by: GBBBAS <[email protected]>

* Add logo to pypi readme

Signed-off-by: GBBBAS <[email protected]>

* Remove pydantic models

Signed-off-by: GBBBAS <[email protected]>

* Remove GraphQL

Signed-off-by: GBBBAS <[email protected]>

* Remove strawberry type

Signed-off-by: GBBBAS <[email protected]>

* Remove GraphQL

Signed-off-by: GBBBAS <[email protected]>

* Upgrade GraphQL Package

Signed-off-by: GBBBAS <[email protected]>

* Logo Size Changes

Signed-off-by: GBBBAS <[email protected]>

* Logo fixes

Signed-off-by: GBBBAS <[email protected]>

---------

Signed-off-by: GBBBAS <[email protected]>
Signed-off-by: GBBBAS <[email protected]>
* set up spark sessions

Signed-off-by: cching95 <[email protected]>

* adding unit tests for spark session

Signed-off-by: cching95 <[email protected]>

* remove tests for spark session

Signed-off-by: cching95 <[email protected]>

* fix code smells

Signed-off-by: cching95 <[email protected]>

* add constants in pipeline utils

Signed-off-by: cching95 <[email protected]>

---------

Signed-off-by: cching95 <[email protected]>
* add source eventhub to pipelines

Signed-off-by: cching95 <[email protected]>

* updates for documentation

Signed-off-by: cching95 <[email protected]>

* updates to mkdocs

Signed-off-by: cching95 <[email protected]>

* added unit tests

Signed-off-by: cching95 <[email protected]>

* documentation for eventhub source

Signed-off-by: cching95 <[email protected]>

* fix spelling error

Signed-off-by: cching95 <[email protected]>

* add source eventhub destination

Signed-off-by: cching95 <[email protected]>

* added destination source documentation

Signed-off-by: cching95 <[email protected]>

* remove unit tests to revisit later

Signed-off-by: cching95 <[email protected]>

* fix mkdoc errors

Signed-off-by: cching95 <[email protected]>

* fix code smells

Signed-off-by: cching95 <[email protected]>

---------

Signed-off-by: cching95 <[email protected]>
* Spark pipeline source delta and destination delta

Signed-off-by: rodalynbarce <[email protected]>

* fix mkdocs errors

Signed-off-by: rodalynbarce <[email protected]>

* docstring amendments

Signed-off-by: rodalynbarce <[email protected]>

* Added to docstrings on delta source & destination

Signed-off-by: JamesKnBr <[email protected]>

* added query type for eventhubs

Signed-off-by: cching95 <[email protected]>

---------

Signed-off-by: rodalynbarce <[email protected]>
Signed-off-by: JamesKnBr <[email protected]>
Signed-off-by: cching95 <[email protected]>
Co-authored-by: JamesKnBr <[email protected]>
Co-authored-by: cching95 <[email protected]>
* add pipeline utils unit tests

Signed-off-by: cching95 <[email protected]>

* add unit test for eventhub and delta

Signed-off-by: cching95 <[email protected]>

* Unit Test Fixes

Signed-off-by: GBBBAS <[email protected]>

* remove test spark

Signed-off-by: cching95 <[email protected]>

* add unit test for destination delta

Signed-off-by: cching95 <[email protected]>

* fix eventhub spell errors

Signed-off-by: cching95 <[email protected]>

---------

Signed-off-by: cching95 <[email protected]>
Signed-off-by: GBBBAS <[email protected]>
Co-authored-by: GBBBAS <[email protected]>
* add transform eventhub

Signed-off-by: cching95 <[email protected]>

* transformer docs set up

Signed-off-by: cching95 <[email protected]>

* documentation for transformer eventhub body

Signed-off-by: cching95 <[email protected]>

* add transformer eventhub unit tests

Signed-off-by: cching95 <[email protected]>

---------

Signed-off-by: cching95 <[email protected]>
* Dependency Injector

Signed-off-by: GBBBAS <[email protected]>

* Add Pipeline Executor

Signed-off-by: GBBBAS <[email protected]>

* Add Utilities Interface

Signed-off-by: GBBBAS <[email protected]>

* Add Pipeline Executor

Signed-off-by: GBBBAS <[email protected]>

* Add Pipeline Execute Test

Signed-off-by: GBBBAS <[email protected]>

* fixes #90

Signed-off-by: GBBBAS <[email protected]>

* Fixes for tests

Signed-off-by: GBBBAS <[email protected]>

* Step Order fix

Signed-off-by: GBBBAS <[email protected]>

* Sonarqube fixes

Signed-off-by: GBBBAS <[email protected]>

---------

Signed-off-by: GBBBAS <[email protected]>
* Add to docs site

Signed-off-by: GBBBAS <[email protected]>

* Update Delta Table Name

Signed-off-by: GBBBAS <[email protected]>

* Add Delta Table Create

Signed-off-by: GBBBAS <[email protected]>

* Add Delta Table Create Test

Signed-off-by: GBBBAS <[email protected]>

---------

Signed-off-by: GBBBAS <[email protected]>
Signed-off-by: cching95 <[email protected]>
Co-authored-by: GBBBAS <[email protected]>
Co-authored-by: Amber-Rigg <[email protected]>
Co-authored-by: NooraKubati <[email protected]>
* Add DBX Pipeline Deployment

Signed-off-by: GBBBAS <[email protected]>

* Updates for Pipeline Deployment

Signed-off-by: GBBBAS <[email protected]>

---------

Signed-off-by: GBBBAS <[email protected]>
* Pipeline Deployment Updates

Signed-off-by: GBBBAS <[email protected]>

* Update to run logic of Pipeline

Signed-off-by: GBBBAS <[email protected]>

---------

Signed-off-by: GBBBAS <[email protected]>
* eventhub destination unit test

Signed-off-by: cching95 <[email protected]>

* add unit test for spark eventhub destination

Signed-off-by: cching95 <[email protected]>

* add spark eventhub write stream unit test

Signed-off-by: cching95 <[email protected]>

* update exception handling for eventhub and delta

Signed-off-by: cching95 <[email protected]>

* update exception in utilities

Signed-off-by: cching95 <[email protected]>

---------

Signed-off-by: cching95 <[email protected]>
* streaming unit test for source eventhub

Signed-off-by: cching95 <[email protected]>

* add delta read stream unit test

Signed-off-by: cching95 <[email protected]>

* add unit test for delta destination streaming

Signed-off-by: cching95 <[email protected]>

* remove extra test

Signed-off-by: cching95 <[email protected]>

* remove code smell

Signed-off-by: cching95 <[email protected]>

* add previous line of code back

Signed-off-by: cching95 <[email protected]>

---------

Signed-off-by: cching95 <[email protected]>
* Fix for Dependency Injection

Signed-off-by: GBBBAS <[email protected]>

* Pipeline Deployment And Secrets

Signed-off-by: GBBBAS <[email protected]>

* Add System Type Docs

Signed-off-by: GBBBAS <[email protected]>

* Updates for Secrets

Signed-off-by: GBBBAS <[email protected]>

* Code Smell fix

Signed-off-by: GBBBAS <[email protected]>

* Refactor Pipeline Secret Conversion Logic

Signed-off-by: GBBBAS <[email protected]>

* import required objects

Signed-off-by: GBBBAS <[email protected]>

* Code smell fix

Signed-off-by: GBBBAS <[email protected]>

* Code Smell Fix

Signed-off-by: GBBBAS <[email protected]>

* Fix Code Smells

Signed-off-by: GBBBAS <[email protected]>

---------

Signed-off-by: GBBBAS <[email protected]>
* Added fails tests to pipeline sources

Signed-off-by: JamesKnBr <[email protected]>

* Added fails tests for pipeline destinations

Signed-off-by: JamesKnBr <[email protected]>

* Fixed code smell

Signed-off-by: JamesKnBr <[email protected]>

---------

Signed-off-by: JamesKnBr <[email protected]>
* Updates for Pipelines

Signed-off-by: GBBBAS <[email protected]>

* Update DBX Documentation

Signed-off-by: GBBBAS <[email protected]>

* Updates for Pipeline Deployment

Signed-off-by: GBBBAS <[email protected]>

* Remove package.json

Signed-off-by: GBBBAS <[email protected]>

* Fix code smells

Signed-off-by: GBBBAS <[email protected]>

---------

Signed-off-by: GBBBAS <[email protected]>
* fix twa group by dataframe return

Signed-off-by: cching95 <[email protected]>

* bug fixes for groupby and apply method

Signed-off-by: cching95 <[email protected]>

---------

Signed-off-by: cching95 <[email protected]>
@GBBBAS GBBBAS requested a review from cching95 March 30, 2023 08:53
@sonarqubecloud
Copy link

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

@GBBBAS GBBBAS merged commit d4b8d1c into main Mar 30, 2023
GBBBAS pushed a commit that referenced this pull request Apr 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants