Bug Found when using hadoop with localFileSink #132
Description
with hadoop's configuration file in classpath, even the nested localFileSink will using hadoop's remote file system.
when deep into the code ,i found there has no judge on whether to use localFileSystem or not.
the FileWriterBase's constructor should be changed by adding a localFileSystem flag to control this.
public FileWriterBase(String codecClass, Logger log, Configuration conf,Boolean localFileSystem) {
this.conf = conf;
try {
if(localFileSystem == null) localFileSystem = false;
fs = localFileSystem ? FileSystem.getLocal(conf) : FileSystem.get(conf);
fs.setVerifyChecksum(false);
if (codecClass != null) {
codec = createCodecInstance(codecClass);
log.info("Codec:" + codec.getDefaultExtension());
} else {
codec = null;
}
} catch (Exception e) {
throw new RuntimeException(e);
}
}
i'll give a pull request for this ASAP.