Tags: NDAR/nda-tools
Tags
Merge pull request #97 in WSCLIENT/nda-tools-python from release/pack… …age_service_update to master Squashed commit of the following: commit d74d82bf1f366aae6eceae1768c908adacb94627 Author: Greg Magdits <[email protected]> Date: Thu Jun 3 09:56:58 2021 -0400 __init__.py edited online with Bitbucket bumped version for NPD-314 commit 8686c8db0602fc650644be6fce1bf88376ff99cd Merge: 2c9ae52 23dcfa5 Author: Greg Magdits <[email protected]> Date: Thu Jun 3 09:49:44 2021 -0400 Merge pull request #96 in WSCLIENT/nda-tools-python from feature/NDA-8282-update-nda-tools-download to release/package_service_update * commit '23dcfa503e58b360782d5aa1709551c49e77bfb8': NDA-8282 * Added step to confirm that self.path_list is not empty before getting package file ides * Updated exception handling of errors for missing data structure or missing s3 URLs to print the error and exit. NDA-8282 * Added step to confirm that self.path_list is not empty before getting package file ides * Added exception handling for POST request to get package files from list of s3 URLs, which will return a 404 if any of the s3 URLs is not contained in the package. NDA-8282 * Removed unnecessary print statements and comments * Renamed function for using a data structure * Added a function for using a text file with S3 Links * Added Windows specific exception handling for os.rename() * Updated download_from_s3link to always resume if partially downloaded file exists. fixed issues in downloadcmd.py removed default value for datastructure_name in get_manifest_and_file_elements function placed the datadictionary api in settings.cfg and corrected windows specific download path to general updated NDA tools to locate associated files from a Data Structure, then download them NDA-8234 * Updated get_packag_file_ids to support alternative download option for using file with list of s3 links. * Updated download_from_s3_link to create and resume from partially downloaded files. * Moved output to stdout into self.verbose_print so output is only displayed if the user specifies verbose option. NDA-8234 * Updated exception handling for FileExistsError when directory already exists after attempting to create the directory to use lower level OSError that is compatible with python 2 and 3. enabled the associated file download fixed 1 issue where download paths were not as expected updated to remove self.dirname, because this was causing issues when multiple threads were reading and writing to the same variable NDA-8211 * Updated download_from_s3_link * improved how requests downloads files to use streaming and downloading in chunks for memory efficiency, especially for large objects. * updated how directories are created to use the local_file; assuming local_file has the download directory + package_file.alias. * Updated setup.py to add packaging to list of packages to install commit 23dcfa503e58b360782d5aa1709551c49e77bfb8 Author: David Obenshain <[email protected]> Date: Fri May 28 01:50:27 2021 -0400 NDA-8282 * Added step to confirm that self.path_list is not empty before getting package file ides * Updated exception handling of errors for missing data structure or missing s3 URLs to print the error and exit. commit c055379665c0687d16e611af5d6f77c3c32dea9f Author: David Obenshain <[email protected]> Date: Fri May 28 01:40:13 2021 -0400 NDA-8282 * Added step to confirm that self.path_list is not empty before getting package file ides * Added exception handling for POST request to get package files from list of s3 URLs, which will return a 404 if any of the s3 URLs is not contained in the package. commit 7109427153c76eee32865b5fee5c1bb225874d6f Author: David Obenshain <[email protected]> Date: Fri May 28 01:18:50 2021 -0400 NDA-8282 * Removed unnecessary print statements and comments * Renamed function for using a data structure * Added a function for using a text file with S3 Links * Added Windows specific exception handling for os.rename() * Updated download_from_s3link to always resume if partially downloaded file exists. commit 1dbafc34fc70f4ebf6a3ec810d5c0922fde5f9bc Author: Demisew <[email protected]> Date: Thu May 27 15:49:50 2021 -0400 fixed issues in downloadcmd.py commit 86439c119196e476179b579cd1d39a26b6f326ad Author: Demisew <[email protected]> Date: Thu May 27 13:20:28 2021 -0400 removed default value for datastructure_name in get_manifest_and_file_elements function commit b7f777aa274edf335864ecde0b658b655fbc7989 Author: Demisew <[email protected]> Date: Thu May 27 12:22:14 2021 -0400 placed the datadictionary api in settings.cfg and corrected windows specific download path to general commit 9a636c155aaee16232bed43f887a04b7ebc45ef1 Author: Demisew <[email protected]> Date: Wed May 26 23:16:26 2021 -0400 updated NDA tools to locate associated files from a Data Structure, then download them commit 281e047b3ad06e7baa31647b2fbe7cf0069af83b Author: David Obenshain <[email protected]> Date: Fri May 21 13:01:57 2021 -0400 NDA-8234 * Updated get_packag_file_ids to support alternative download option for using file with list of s3 links. * Updated download_from_s3_link to create and resume from partially downloaded files. * Moved output to stdout into self.verbose_print so output is only displayed if the user specifies verbose option. commit 71f27b9c45fe0138203041720690f88308c07a05 Author: David Obenshain <[email protected]> Date: Thu May 13 16:54:32 2021 -0400 NDA-8234 * Updated exception handling for FileExistsError when directory already exists after attempting to create the directory to use lower level OSError that is compatible with python 2 and 3. commit 684e114 Author: Demisew <[email protected]> Date: Fri Apr 30 18:26:05 2021 -0400 enabled the associated file download commit 15ae6e0 Author: NIH\magditsgs <[email protected]> Date: Thu Apr 29 18:09:47 2021 -0400 fixed 1 issue where download paths were not as expected commit 43e9e12 Author: NIH\magditsgs <[email protected]> Date: Thu Apr 29 15:48:00 2021 -0400 updated to remove self.dirname, because this was causing issues when multiple threads were reading and writing to the same variable commit 958c9f5 Author: David Obenshain <[email protected]> Date: Thu Apr 29 12:49:37 2021 -0400 NDA-8211 * Updated download_from_s3_link * improved how requests downloads files to use streaming and downloading in chunks for memory efficiency, especially for large objects. * updated how directories are created to use the local_file; assuming local_file has the download directory + package_file.alias. * Updated setup.py to add packaging to list of packages to install commit 2c9ae52 Author: David Obenshain <[email protected]> Date: Tue Apr 20 13:53:11 2021 -0400 Merge pull request #84 in WSCLIENT/nda-tools-python from feature/NDA-4037-add-package-service to release/package_service_update Squashed commit of the following: commit 83e931494e50e21de9450b5821deb1f7c329814c Author: David Obenshain <[email protected]> Date: Tue Apr 20 13:50:50 2021 -0400 Updated settings.cfg template to set production endpoint for package service. commit 22c7e8136124194c3257326affca7224d809a225 Merge: e4da79a f6187da Author: David Obenshain <[email protected]> Date: Tue Apr 20 13:43:48 2021 -0400 Merge branch 'release/package_service_update' of http://git.nimhda.org/scm/wsclient/nda-tools-python into feature/NDA-4037-add-package-service � Conflicts: � NDATools/Download.py � NDATools/__init__.py commit e4da79a293e450236383ce55d7e4e240738d5f2d Author: Emmanuel Omotosho <[email protected]> Date: Thu Apr 1 10:02:07 2021 -0400 Merge pull request #82 in WSCLIENT/nda-tools-python from NDA-7141-perform-integration-tests-on-command-line-download-manager-and-updated-package-service-to to feature/NDA-4037-add-package-service Squashed commit of the following: commit 4ffee93e5854e5298430992c9c090864c29ee9ea Author: Emmanuel Omotosho <[email protected]> Date: Thu Apr 1 09:05:37 2021 -0400 Update __init__.py commit a770aa120bd919f04bc59689a1ba166fdd949ea0 Author: Greg Magdits <[email protected]> Date: Wed Mar 31 11:41:48 2021 -0400 __init__.py edited online with Bitbucket updated version for testpypi deploy commit dc489cf30eff0b425172d17e1421d3e5446ed76a Merge: 98dc159 1627e49 Author: Emmanuel Omotosho <[email protected]> Date: Mon Mar 29 09:22:58 2021 -0400 Merge branch 'master' into feature/NDA-4037-add-package-service commit 1627e49 Author: Roger <[email protected]> Date: Mon Mar 22 12:55:40 2021 -0400 Increment version. Update CHANGELOG commit 7a41953 Author: Roger Lee <[email protected]> Date: Mon Mar 22 12:47:16 2021 -0400 Merge pull request #79 in WSCLIENT/nda-tools-python from release/NPD-279-sha256-password-encoder to master Squashed commit of the following: commit bae7c5427cb873afa7f31cde97154a70d4e078f1 Author: Roger <[email protected]> Date: Tue Mar 2 16:36:17 2021 -0500 Increment version commit 56fee340970f1c49b111881ef856fc35d8c353a7 Author: Roger <[email protected]> Date: Tue Mar 2 16:35:45 2021 -0500 Remove print statement commit 07e169f205b9c1b4a0a1be7f6240700a2b3da959 Author: Demisew <[email protected]> Date: Fri Feb 26 10:22:26 2021 -0500 changed version to 0.1.56 commit 8475add220ed347f7cf2a96a128d5a3fdc5b4490 Author: Demisew <[email protected]> Date: Thu Feb 25 14:01:56 2021 -0500 changed version commit e4bfd7f13e9a6ecdb4c80cfc78be6d2182293a2a Author: Roger <[email protected]> Date: Thu Jan 7 15:29:51 2021 -0500 Create pseudo-delegating password encoding with preference to sha256 commit f444b2c Author: Roger <[email protected]> Date: Fri Aug 28 09:10:44 2020 -0400 Increment version to match PyPI. commit 6c20721 Author: NIH\magditsgs <[email protected]> Date: Tue Apr 7 13:53:29 2020 -0400 bumped version so that version transition in pypi made sense. and updated changelog commit 69c1317 Author: NIH\magditsgs <[email protected]> Date: Tue Apr 7 09:11:11 2020 -0400 release version commit 6a718b6 Author: Greg Magdits <[email protected]> Date: Tue Apr 7 09:10:05 2020 -0400 Merge pull request #73 in WSCLIENT/nda-tools-python from hotfix/NDA-6027-nda-tools-python-error-during-resume-when-incomplete-upload-parts-exist-in-s3 to master Squashed commit of the following: commit fe2dcd93b23071673af72d3131913c4179d33ed2 Author: NIH\magditsgs <[email protected]> Date: Fri Apr 3 13:38:40 2020 -0400 udpated version commit 66e10444087da550efe0460c4492e9d3fee8b4cf Author: NIH\magditsgs <[email protected]> Date: Fri Apr 3 13:37:08 2020 -0400 removed un-necessary checks that killed program execution commit 7e8642d37989e773bc5d457d34d9442ada6fd7d4 Author: NIH\magditsgs <[email protected]> Date: Fri Apr 3 13:01:30 2020 -0400 updated checking logic to only run when in production. Added .dev to indicate this is a dev release commit f95ed35bea58c7cde0bfb58f8a7748f0ec88961e Author: NIH\magditsgs <[email protected]> Date: Thu Apr 2 16:28:36 2020 -0400 updated to next available version commit 57a44e7992ae0e1e2435a276ef781c7cea112da9 Author: NIH\magditsgs <[email protected]> Date: Thu Apr 2 16:26:53 2020 -0400 updating version commit d8885b8f92bb88e37c4305ae3ad78394e9f65f22 Author: NIH\magditsgs <[email protected]> Date: Thu Apr 2 15:46:33 2020 -0400 NDA-6027 - provided required argument commit 50f9cb1 Merge: 66c48a0 e73ffa0 Author: David Obenshain <[email protected]> Date: Thu Oct 31 11:13:00 2019 -0400 Merge pull request #58 in WSCLIENT/nda-tools-python from release/Download_Manager_Refresh_2 to master * commit 'e73ffa03b8f94a4c71ae93ed388432e5cf1e3d7a': (38 commits) Removed incorrect change from changelog. Updated version for prod deploy to PyPI. Updated changelog. Merge pull request #56 in WSCLIENT/nda-tools-python from feature/NDA-5359-add-integration-tests to develop Merge pull request #54 in WSCLIENT/nda-tools-python from feature/NDA-5324-remove-duplicate-credential-handling to develop Using @patch instead of instead of mock.patch function. Removed copy of parse_local_files and replace with call to actual function. Increment Test PyPI version. Added missing parameter for skip local file check. Removed unit test classes for other modules. Added filepath function testing for Utils. NDA-5154 * Added try to catch exception for file not existing when attempting to locate the file in multiple directories. * Added progress on matching files to directories to debug log. NDA-5154 - Updating version for Test PyPI NDA-5154 - Addressing bug where parse_local_files will use local instance of Configuration class, which does not have the skip_local_file_check that is set in vtcmd.py and instance of Submission.py on resuming a submission. Added tests for configuration. Added missing Configuration variable for aws_access_key. Init test files Moved s3 resource into S3Auth. Corrected case where file_size is less than 9999 which results in invalid chunk size of 0. Moved s3 client creation to S3Auth class. Renamed Authorization to S3Authentication. Class handles s3 client session creation. Multipart sessions are still handled by Submission. Revert to getpass instead of input. ... commit e73ffa0 Author: Roger <[email protected]> Date: Wed Oct 30 15:07:42 2019 -0400 Removed incorrect change from changelog. commit 56b90c6 Author: Roger <[email protected]> Date: Wed Oct 30 13:49:21 2019 -0400 Updated version for prod deploy to PyPI. commit 62c707f Author: Roger <[email protected]> Date: Tue Oct 29 12:24:49 2019 -0400 Updated changelog. commit 7b9e63b Merge: 522e200 56b08f2 Author: Roger Lee <[email protected]> Date: Tue Oct 22 10:59:32 2019 -0400 Merge pull request #57 in WSCLIENT/nda-tools-python from develop to release/Download_Manager_Refresh_2 * commit '56b08f2ed29028d7ed59dbe53bc079850be0d348': (23 commits) Merge pull request #56 in WSCLIENT/nda-tools-python from feature/NDA-5359-add-integration-tests to develop Merge pull request #54 in WSCLIENT/nda-tools-python from feature/NDA-5324-remove-duplicate-credential-handling to develop Using @patch instead of instead of mock.patch function. Removed copy of parse_local_files and replace with call to actual function. Increment Test PyPI version. Added missing parameter for skip local file check. Removed unit test classes for other modules. Added filepath function testing for Utils. NDA-5154 * Added try to catch exception for file not existing when attempting to locate the file in multiple directories. * Added progress on matching files to directories to debug log. NDA-5154 - Updating version for Test PyPI NDA-5154 - Addressing bug where parse_local_files will use local instance of Configuration class, which does not have the skip_local_file_check that is set in vtcmd.py and instance of Submission.py on resuming a submission. Added tests for configuration. Added missing Configuration variable for aws_access_key. Init test files Moved s3 resource into S3Auth. Corrected case where file_size is less than 9999 which results in invalid chunk size of 0. Moved s3 client creation to S3Auth class. Renamed Authorization to S3Authentication. Class handles s3 client session creation. Multipart sessions are still handled by Submission. Revert to getpass instead of input. Use property decorator instead of function call. Increment version. Appends session_token to User section of settings.cfg if missing. Newly created settings.cfg has session_token option. ... commit 56b08f2 Author: Roger Lee <[email protected]> Date: Tue Oct 22 10:58:01 2019 -0400 Merge pull request #56 in WSCLIENT/nda-tools-python from feature/NDA-5359-add-integration-tests to develop Squashed commit of the following: commit 2436513bda23a2105d3bce3ebad502d439ea8402 Author: Roger <[email protected]> Date: Tue Oct 22 10:41:55 2019 -0400 Moved integration tests to the nda-tools-python-tests repo since not public-facing. commit 2aa641dc30cbf76152ce981f5a608457006bed73 Merge: f4bb0c4 4f05275 Author: Roger <[email protected]> Date: Wed Oct 16 14:10:40 2019 -0400 Merged in NDA-5358, NDA-5324 from develop. Added entry points to run unit and integration tests. Increment version. commit f4bb0c468c0fe2bcfa396865c3f8b6e010c02582 Author: Roger <[email protected]> Date: Fri Oct 11 09:53:04 2019 -0400 Emptied __init__ file in tests dir. Remove csv test file. commit fe07f52bcb877d8367461cb2174c34c1b0eaa8d2 Author: Roger <[email protected]> Date: Fri Oct 4 16:46:35 2019 -0400 parent c778c040eb23b90bdbb4a0250edb656d5b94986a author Roger <[email protected]> 1570221995 -0400 committer Roger <[email protected]> 1570800319 -0400 parent c778c040eb23b90bdbb4a0250edb656d5b94986a author Roger <[email protected]> 1570221995 -0400 committer Roger <[email protected]> 1570800258 -0400 ... commit f6187da Author: David Obenshain <[email protected]> Date: Mon Apr 12 12:00:41 2021 -0400 Merge pull request #85 in WSCLIENT/nda-tools-python from resume_flag_should_be_honored to release/package_service_update Squashed commit of the following: commit 63f9c1af53012c4753dd5e7f9ee25b2501fa872c Author: Emmanuel Omotosho <[email protected]> Date: Tue Mar 30 08:45:40 2021 -0400 resume flag should be honored commit 1b81ebc Author: David Obenshain <[email protected]> Date: Mon Apr 12 11:58:23 2021 -0400 __init__.py edited online with Bitbucket Updating dev version to resolve merge conflit with dev branches.
PreviousNext