KubernetesJobTask bug: Pod status does not have conditions #2432
Description
I have several dozens of KubernetesJobTasks every day. And approximately twice a day this error happens:
Traceback (most recent call last):
File "/opt/conda/lib/python3.6/site-packages/luigi/worker.py", line 203, in run
new_deps = self._run_get_new_deps()
File "/opt/conda/lib/python3.6/site-packages/luigi/worker.py", line 140, in _run_get_new_deps
task_gen = self.task.run()
File "/usr/src/app/phpipeline/luigi_util.py", line 218, in run
super(PHKubernetesJobTask, self).run()
File "/opt/conda/lib/python3.6/site-packages/luigi/contrib/kubernetes.py", line 353, in run
self.__track_job()
File "/opt/conda/lib/python3.6/site-packages/luigi/contrib/kubernetes.py", line 199, in __track_job
while not self.__verify_job_has_started():
File "/opt/conda/lib/python3.6/site-packages/luigi/contrib/kubernetes.py", line 276, in __verify_job_has_started
for cond in status['conditions']:
KeyError: 'conditions'
Unfortunately, I cannot reproduce it.
But the problem is in the way status conditions are accessed:
- how it's done in
luigi:status['conditions'](line 276). - how it's done in
pykube:status.get('conditions', [])(line 329).
I guess, it can be easily fixed by using the same way of getting status conditions, as in pykube.