stage1: set TERM temp, keepEnv merge in "rkt run" #1962

borqosky · 2016-01-13T11:41:36Z

Pass additional TERM to env. Needed to proper work after entering
Related to issue #28

alban · 2016-01-13T12:36:46Z

The bug was filed in intelsdi-x#28. Please file issues that are relevant to upstream on the coreos/rkt project next time. The patch needs to be rebased on upstream/master.

I think the logic in append_env() is wrong because it appends the variable in the env file each time "rkt enter" is executed. After several "rkt enter", I get:

# cat /proc/1/root/rkt/env/busybox | tr '\0' '\n'|grep ^TERM
TERM=vt220
TERM=xterm-256color
TERM=xterm-256color
TERM=xterm-256color
TERM=xterm-256color
TERM=xterm-256color

Instead of patching appexec.c, you could patch enter.c to exec appexec with an alternative env file for this case.

jellonek · 2016-01-13T13:58:08Z

We are not using enter.c and also it uses appexec which constructs environment from application environment with repeatable adding LISTEN_FDS and LISTEN_PID if they are in env in entering time...

IMO append_env should be modified so this values should be added into environment of executing process, but not into environment file.

jellonek · 2016-01-13T14:02:27Z

Maybe another parameter in appexec command line which would provide a method for getting some environment variables (their names to get from actual env, or names+values) from command line?

alban · 2016-01-13T14:03:22Z

IMO append_env should be modified so this values should be added into environment of executing process, but not into environment file.

Good idea, it seems cleaner this way.

borqosky · 2016-01-14T10:00:52Z

Argh...wasn't aware of that, that I should file issue twice. Instead of hacking in appexec, I propose set "keep env" like "inherit env" and "set env" before appexec (all envs in PrepareConfig). Then, if appexec is used during "enter" or "run" set $TERM is just temporary.

ppalucki · 2016-01-14T10:46:59Z

rkt/run.go

It can be problem here, but I'm not sure - IIRC LISTEND_FDS and LISTEN_PID are way to pass file descriptors from "internal systemd to application" rather than "external' (host) systemd.

What should work for "common" env variables, I think will not work for "socket activation" variables.

Your variables has to go through all layers like that.

# running systemd(host) -> rkt(binary) -> init -> systemd(internal) -> appexec -> app # entering rkt(binary) -> enter -> ssh(kvm)/nsenter -> appexec -> app

what I'm worried about is the LISTEN_* values are copied to early. Deeper analysis is needed.

Maybe we should, had list names of variables (without values) and each layer is responsible for passing it down (sometimes thorough file and sometimes just by inheriting and sometimes by send_env in ssh options) and then if some layer like "internal systemd" can override what it wants (but it sounds a bit like code duplication).

borqosky · 2016-01-21T10:58:36Z

yep, LISTEND_FDS and LISTEN_PID should be coupled with inner systemd. IMHO additional common keep_env file for all apps should be more clear - it s created during run, and read only in entering

alban · 2016-01-21T12:16:17Z

Thanks for updating it. It will need to be rebased on the last master.

@vcaputo do you have some time to review this?

ppalucki · 2016-01-22T11:11:10Z

My only issue with this patch is that we should differentiate:

1. applications env variables (passed by pod manifest or inherti env flag)
2, run env variables eg. LISTEN_* pair (for socket activation)
1. enter env variables eg. TERM (and eventually other passed explicit when rkt enter similar to ssh SendEnv option)

for run we should have: "app env set" + "run env set"
for enter we shoud have: "app env set" + "enter evn set"

IIUC, this change merge variables from "run" and "enter" set and IMO in some corner case can broke service under systemd eg, some can get unexpected TERM variable from host during "run" or respectively get LISTEN_* during enter

we should call appexec differently when using in run than in enter (+1 for explicit passing variables name that values should be kept to appexec)

borqosky · 2016-01-22T12:40:09Z

enter env variables eg. TERM (and eventually other passed explicit when rkt enter similar to ssh SendEnv option)

could be enhancement issue related with rkt enter command, although it could be done by simple bash command. Similar issue with docker: moby/moby#8838

in some corner case can broke service under systemd eg, some can get unexpected TERM variable from host during "run"

Well, I'm not a systemd expert, neither know that TERM could impact on systemd. If yes, there shouldn't be a problem change it quickly and don't load TERM during rkt run command

respectively get LISTEN_* during enter

LISTEN_* are being set and write to a file only during rkt run, and read it later at rkt enter.
We can use additional flags, instead of files in appexec to check whether is run or enter, but since we use ssh to kvm, there is possibility to make: ssh $IP_ADDR "/appexec ... -e=enter", or ssh $IP_ADDR "/appexec ... -e=run",so both work during runtime.

ppalucki · 2016-01-25T16:42:38Z

stage1/appexec/appexec.c

you forget to move argv offset according the number of options from optind and when you do this (with handling of '--'), you can as well handle -e NAME_OF_VARIABLE to fill the keepEnv table with variables that should be passed:
so it at the end:

run -> unit.service -> ExecStart = appexec -e LISTEN_FDS -e LISTEN_PID ...

'enter' -> appexec -e TERM

just an idea, but looks better, more clean, and more extensible solution - only if you have time for this 😸

And what with my original idea of possibility to also pass new values for variables? I mean something like -e TERM as "set value of TERM variable basing on current environ value (pass it)" and -e TERM=value as "set particular value for TERM variable"?

setenv("TERM", term, 1); is called immediately after the char *term = getenv("TERM"); here, and before the load_env() / clearenv(). It seems to be a no-op, and not what we want.

borqosky · 2016-01-29T09:42:49Z

could be enhancement issue related with rkt enter command, although it could be done by simple bash command. Similar issue with docker: moby/moby#8838

alban · 2016-02-24T15:27:31Z

stage1/appexec/appexec.c

 	size_t		n_gids;

 	exit_if(argc < 7,
 		"Usage: %s /path/to/root /work/directory /env/file uid gid[,gid...] /to/exec [args ...]", argv[0]);


Please document the new parameter -e in the usage.

alban · 2016-02-24T15:32:09Z

Can you rebase on master to fix the conflict?

iaguis · 2016-02-24T16:01:51Z

stage1/appexec/appexec.c

+ */
+static void load_env(const char *env_file, const char *keep_env_file, int entering)
+{
+	char *term = getenv("TERM"); /* usefull to keep during entering. */


Keep additional $TERM before clear envs in appexec - needed to proper work after entering. Additional flag in appexec "-e" for check entering or run. Add additional keep_env file for socket activation. Now appexec doesn't modify env_file, instead it create kee_env_file once, during run command. Related to issue #28

Two additional tests for check TERM env during entering. Document a new optional flag '-e' in appexec.

iaguis · 2016-02-25T15:13:57Z

Tested socket activation and that $TERM is set up on entering (and only once :)) and it works.

LGTM.

stage1: set TERM temp, keepEnv merge in "rkt run"

iaguis · 2016-02-25T15:22:51Z

Merged manually to remove mention of internal issue number in commit messagee.

alban added kind/enhancement component/stage1 area/usability needs/some-changes labels Jan 13, 2016

borqosky force-pushed the wborkows/fix_pass_term_env branch from e419d67 to fc8c40b Compare January 14, 2016 10:01

ppalucki reviewed Jan 14, 2016
View reviewed changes

borqosky changed the title ~~enter: pass TERM env to appexec~~ stage1: set TERM temp, keepEnv merge in "rkt run" Jan 14, 2016

jonboulle modified the milestones: v0.16.0, v1+ Jan 15, 2016

alban added the priority/P2 label Jan 18, 2016

alban modified the milestones: v1.0.0, v0.16.0 Jan 20, 2016

borqosky force-pushed the wborkows/fix_pass_term_env branch 2 times, most recently from 15756d3 to de57589 Compare January 21, 2016 10:49

alban added needs/review and removed needs/some-changes labels Jan 21, 2016

borqosky force-pushed the wborkows/fix_pass_term_env branch from de57589 to 184503a Compare January 22, 2016 08:01

borqosky force-pushed the wborkows/fix_pass_term_env branch from 184503a to 5fc5462 Compare January 25, 2016 09:41

ppalucki reviewed Jan 25, 2016
View reviewed changes

borqosky force-pushed the wborkows/fix_pass_term_env branch from 5fc5462 to 8fe17e7 Compare January 26, 2016 08:57

alban added the needs/some-changes label Feb 2, 2016

alban modified the milestones: v1.1.0, v1.0.0 Feb 2, 2016

borqosky force-pushed the wborkows/fix_pass_term_env branch 3 times, most recently from 20f674d to cb6667e Compare February 4, 2016 14:46

alban reviewed Feb 24, 2016
View reviewed changes

iaguis reviewed Feb 24, 2016
View reviewed changes

borqosky force-pushed the wborkows/fix_pass_term_env branch from cb6667e to 6f8d271 Compare February 25, 2016 08:42

tests: add functional tests for env check

d314c59

Two additional tests for check TERM env during entering. Document a new optional flag '-e' in appexec.

borqosky force-pushed the wborkows/fix_pass_term_env branch from 6f8d271 to d314c59 Compare February 25, 2016 08:43

iaguis added a commit that referenced this pull request Feb 25, 2016

Merge pull request #1962 from intelsdi-x/wborkows/fix_pass_term_env

d4db664

stage1: set TERM temp, keepEnv merge in "rkt run"

iaguis closed this Feb 25, 2016

mzylowski deleted the wborkows/fix_pass_term_env branch August 1, 2016 14:39

Uh oh!

stage1: set TERM temp, keepEnv merge in "rkt run" #1962

stage1: set TERM temp, keepEnv merge in "rkt run" #1962

Uh oh!

Conversation

borqosky commented Jan 13, 2016

Uh oh!

alban commented Jan 13, 2016

Uh oh!

jellonek commented Jan 13, 2016

Uh oh!

jellonek commented Jan 13, 2016

Uh oh!

alban commented Jan 13, 2016

Uh oh!

borqosky commented Jan 14, 2016

Uh oh!

ppalucki Jan 14, 2016

Choose a reason for hiding this comment

Uh oh!

borqosky commented Jan 21, 2016

Uh oh!

alban commented Jan 21, 2016

Uh oh!

ppalucki commented Jan 22, 2016

Uh oh!

borqosky commented Jan 22, 2016

Uh oh!

ppalucki Jan 25, 2016

Choose a reason for hiding this comment

Uh oh!

jellonek Jan 26, 2016

Choose a reason for hiding this comment

Uh oh!

alban Feb 2, 2016

Choose a reason for hiding this comment

Uh oh!

borqosky commented Jan 29, 2016

Uh oh!

alban Feb 24, 2016

Choose a reason for hiding this comment

Uh oh!

alban commented Feb 24, 2016

Uh oh!

iaguis Feb 24, 2016

Choose a reason for hiding this comment

Uh oh!

iaguis commented Feb 25, 2016

Uh oh!

iaguis commented Feb 25, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants