Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Support of hadoop delegation tokens with HADOOP_TOKEN_FILE_LOCATION env #335

@mutexed

Description

@mutexed

Hello, I see you have support of KRB ticket cache file, but in hadoop there is more lightweight type of authenticate cache. Here it is described: https://blog.cloudera.com/hadoop-delegation-tokens-explained/

The motivation is when you are using hdfs client in some big mapreduce operations (or simple map operations for transfering data from hadoop to somewhere else), each job will reauthenticate in kerberos using ticket cache, so KDC becomes a bottleneck. Moreover, while using hadoop delegation token you don't need even network connection to KDC server.

Default apache hdfs binary looks into env for HADOOP_TOKEN_FILE_LOCATION and use it to communicate with namenode if presented, is it supported in your client? If not, is it in plans to support it?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions