当尝试从Java中读取HDFS中的文件时,“错误的FS …期望：file：//

发布时间：2020-12-14 16:25:47 所属栏目：Java 来源：网络整理

导读：我无法使用 Java从HDFS读取文件： String hdfsUrl = "hdfs://ip:port";Configuration configuration = new Configuration();configuration.set("fs.defaultFS",hdfsUrl);FileSystem fs = FileSystem.get(configuration);Path filePath = new Path(hdfsUrl +

我无法使用 Java从HDFS读取文件：

String hdfsUrl = "hdfs://<ip>:<port>";
Configuration configuration = new Configuration();
configuration.set("fs.defaultFS",hdfsUrl);
FileSystem fs = FileSystem.get(configuration);
Path filePath = new Path(hdfsUrl + "/projects/harmonizome/data/achilles/attribute_list_entries.txt.gz");
FSDataInputStream fsDataInputStream = fs.open(filePath);

SEVERE: Servlet.service() for servlet [edu.mssm.pharm.maayanlab.Harmonizome.api.DownloadAPI] in context with path [/Harmonizome] threw exception
java.lang.IllegalArgumentException: Wrong FS: hdfs://146.203.54.165:8020/projects/harmonizome/data/achilles/attribute_list_entries.txt.gz,expected: file:///
    at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:310)
    at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:47)
    at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:357)
    at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:245)
    at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:125)
    at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
    at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:356)
    at edu.mssm.pharm.maayanlab.Harmonizome.api.DownloadAPI.readLines(DownloadAPI.java:37)
    at edu.mssm.pharm.maayanlab.Harmonizome.api.DownloadAPI.doGet(DownloadAPI.java:27)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:622)
...

我没有设置我们的HDFS,所以我不知道我不知道什么.任何帮助表示赞赏.

解决方法

试试这个：

Configuration configuration = new Configuration();
FileSystem fs = FileSystem.get(new URI(<url:port>),configuration);
Path filePath = new Path(<path/to/file>);
FSDataInputStream fsDataInputStream = fs.open(filePath);
BufferedReader br = new BufferedReader(new InputStreamReader(fsDataInputStream));

请参考http://techidiocy.com/java-lang-illegalargumentexception-wrong-fs-expected-file/

解决了类似的问题.

（编辑：李大同）

【声明】本站内容均来自网络，其相关言论仅代表作者个人观点，不代表本站立场。若无意侵犯到您的权利，请及时与联系站长删除相关内容!