add /keep empty collection mount to a-c-r#81
Open
jrandall wants to merge 2 commits intoarvados:mainfrom
Open
Conversation
Member
|
Whats the reasoning for this? |
Contributor
Author
|
This is a signal to crunch-run that it is ok to bind mount over `/keep`
(and therefore it is ok to implement the mounting of all subdirectories of
`/keep` within a single docker bind mount). This mechanism was suggested by
Tom Clegg as a simpler alternative to what I proposed.
This is a part of the fix to https://dev.arvados.org/issues/13636 - the other part (of our implementation,
which is working) is a revised crunch-run which uses the `bindmapfuse` (
https://github.com/wtsi-hgi/bindmapfuse) fuse filesystem to provide limited
access to the underlying `arv-mount` filesystem. I.e. we use the
`bindmapfuse` configuration rather than docker bind mounts to limit which
portions of `/keep` are accessible to the container.
The reason for the empty mount at `/keep` is that without that, it wouldn't
necessarily be safe for crunch-run to decide on its own to bind mount
`/keep` (i.e. because perhaps the container has some other contents under
that directory that it needs to be able to see in addition to the
subdirectories that are bind mounted in).
…On Wed, Sep 5, 2018 at 4:08 PM Peter Amstutz ***@***.***> wrote:
Whats the reasoning for this?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#81 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAb-ee01EJnwIuYRbIRdoS42EGFbV2cYks5uX-j1gaJpZM4WWjYL>
.
|
Member
|
So this only makes sense with corresponding crunch-run changes? I'm very confused by this, I think I understand the problem you're trying to solve but it seems like a very roundabout solution. |
Contributor
Author
|
It shouldn't be harmful without the crunch-run changes, so long as the
container doesn't have contents in `/keep`. The only side effect with the
current crunch-run would be that an empty collection would be explicitly
bind mounted over `/keep` in the container, meaning that if there was
anything under there it would be masked before the other more specific bind
mounts were layered over it.
That is really the purpose of it - to signal to crunch-run that it is ok to
make that entire tree into a single bind mount. The original plan in 13636
was to do that by adding functionality to arv-mount to allow specifying
specific locator paths to expose, but I am not familiar enough with that
codebase to make those changes quickly and was able to create a standalone
fuse filesystem to do it in its own layer, at least as a proof of concept.
If you are interested, the crunch-run changes that make use of the `/keep`
empty collection mount (and that we are currently running with) are in:
master...wtsi-hgi:13636-crunch-run-bindmapfuse
…On Wed, Sep 5, 2018 at 7:44 PM Peter Amstutz ***@***.***> wrote:
So this only makes sense with corresponding crunch-run changes? I'm very
confused by this, I think I understand the problem you're trying to solve
but it seems like a very roundabout solution.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#81 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAb-eb7oXgZ8jmRYm3OjSJdYFtTIx5u1ks5uYBuNgaJpZM4WWjYL>
.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.