Ability to cache FeatureUnion transformers #9008
Comments
|
Couldn't this effect be obtained by wrapping the |
|
A general memoization wrapper wigs solve all sorts of things in this space
(and I've written one that has not been accepted). we have instead chosen
to make this the responsibility of the container wherein some components
are liable to change while others change in a parameter search
…On 7 Jun 2017 12:05 am, "Laurent Sorber" ***@***.***> wrote:
Couldn't this effect be obtained by wrapping the FeatureUnion
transformers in cached Pipelines? That is, assuming the full Pipeline
would be cached as suggested in #9007
<#9007>.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#9008 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAEz601lBTBkUvhBK_T3YwMVpPnajXDTks5sBVzDgaJpZM4NxWjY>
.
|
|
A general memoization wrapper wigs solve all sorts of things in this space
(and I've written one that has not been accepted).
We could reopen this discussion.
|
|
It seems reasonable to support a memory parameter to FeatureUnion like was
recently added to Pipeline (#7990).
Seems reasonnable indeed to me.
|
|
can this be assigned to me? I'm really interested in this as it should be very useful with cases described in #8960 |
|
We can't use github assignment: it only allows assignment to team members. But as far as I'm concerned, you're welcome to contribute a patch. |
|
Has there be any update on this? It seems to me that |
|
Hi Philipp, could you give a more explicit example of what you expected and
what you got?
|
|
@jnothman At the beginning I was only doing for test purposes a single FeatureUnion within a pipeline and this did not get cached. Apparently, more than one step need to be done in the pipeline, even if the FeatureUnion consists of multiple steps. Anyways, it was more of a gut feeling after following the discussion in this thread. I have some FeatureUnion operations including BOW Vectorizers inside and can't see any speed improvements with consecutive executions after using cache. I think the main reason is that, if I am correct, transforms are not cached, rather only fits. And I am not 100% sure if it works properly for FeatureUnion. By and large, I don't have clear tests on that and thus, I will get back to this thread when I have some more insights into the topic. |
|
This enhancement applies also to |
It seems reasonable to support a
memoryparameter toFeatureUnionlike was recently added toPipeline(#7990). It is valuable in the sense that parameters in some constituent transformers can be searched over while others are unchanged; those that are unchanged should not need to be re-fit from scratch.The text was updated successfully, but these errors were encountered: