Introducing parallel variant of add_all_ta_features. #310
+478
−3
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
+ Add add_all_ta_features_parallel parallelizing all indicators calculation. * Fix __init__.py and README.md.
There was a problem hiding this comment.
@such3r
Getting this error when running
line 609, in _job
instance = globals()[obj](**{k: target[v] if k in ohlcv_keys else v for k, v in kwargs.items()})
TypeError: __init__() got an unexpected keyword argument 'close'
The above exception was the direct cause of the following exception:
TypeError Traceback (most recent call last)
Cell In[4], line 1
----> 1 _ = add_all_ta_features_parallel(df,open="open", high="high", low="low", close="close", volume="volume", fillna=False)
File ~/utils/parallel_ta.py:1068, in add_all_ta_features_parallel(df, open, high, low, close, volume, fillna, colprefix, vectorized, n_jobs)
964 init += [
965 # Money Flow Indicator
966 (
(...)
1063 ),
1064 ]
1066 columns_order = df.columns.tolist() + [column for *_, jobs in init for column, _ in jobs]
-> 1068 results = Parallel(n_jobs=n_jobs)(delayed(_job)(df, *args) for args in init)
1070 for columns in results:
1071 for column, values in columns.items():
|
@Harsh-Maheshwari, could you, please, provide the input data you used? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Suggest simple process-based all-set indicators calculation parallelizing. Gives ~2x speedup compared to the initial implementation.