MySQL batched multiPut #173
Conversation
Merge from twitter/storehaus
|
My handling of futures looks a bit clunky. Would appreciate any help with it. |
There was a problem hiding this comment.
this is not an override. That's a bit weird. It means it can't work with any code that wants a multiPut. That's a shame.
Also, batching is probably something that should be done with a combinator for all stores. See:
You want something like that for Store, right?
There was a problem hiding this comment.
Makes sense.
I'll move this to a separate batched store version, though, it looks like all the sql logic will need to go to a separate MySqlBatchedStore which extends the generic BatchedStore?
There was a problem hiding this comment.
It may extend or users can use the combinator: new BatchedStore(mySqlStore, 1000) or: mySqlStore.batched(1000) or something.
|
Decided to go with overriding Store's multiPut so we do not need a separate batched version specific to mysql. (By batching, I originally meant grouping all keys in multiPut to a single sql query.) I've created a separate pull req for BatchedStore combinator #175 |
There was a problem hiding this comment.
do you want to sequence these futures? Now the order is non-deterministic. Is that what you want?
There was a problem hiding this comment.
I'm thinking this is okay since order is not important here, and all these futures complete as part of object instantiation?
There was a problem hiding this comment.
Yes. I was dreaming of Haskell and lazy everywhere... Ian also corrected me.
|
Deleted the branch mistakenly. Re-opening. |
There was a problem hiding this comment.
why not just v => putResult.unit?
There was a problem hiding this comment.
Changed the combinator on the earlier mapCollect output to flatMap. Works now.
|
Thank you! |
|
Thanks for reviewing. |
Add a batched version of multiPut that splits the kv set into insert, update and delete queries.