登录到有无效块的机器后操作。
hdfs fsck / 后面的路径是指定检查哪个目录下有无效块，根据丢失文件目录可以细化，以提高检查速度
该命令会返回
FSCK started by hdfs (auth:SIMPLE) from /xxx.xxx.xxx for path / at Fri Jul 26 14:37:29 CST 2019................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................/.../c9ddcb18-51c0-4fa7-bdab-daa5079bc094/rdd-1061487/_partitioner: Under replicated BP-770033680-172.20.36.31-1562142528050:blk_1073943459_202678. Target Replicas is 3 but found 2 live replica(s), 0 decommissioned replica(s), 0 decommissioning replica(s).. /.../c9ddcb18-51c0-4fa7-bdab-daa5079bc094/rdd-1061487/part-00000: Under replicated BP-770033680-172.20.36.31-1562142528050:blk_1073943450_202669. Target Replicas is 3 but found 2 live replica(s), 0 decommissioned replica(s), 0 decommissioning replica(s).. /.../c9ddcb18-51c0-4fa7-bdab-daa5079bc094/rdd-1061487/part-00001: Under replicated BP-770033680-172.20.36.31-1562142528050:blk_1073943456_202675. Target Replicas is 3 but found 2 live replica(s), 0 decommissioned replica(s), 0 decommissioning replica(s).. /.../c9ddcb18-51c0-4fa7-bdab-daa5079bc094/rdd-1061487/part-00002: Under replicated BP-770033680-172.20.36.31-1562142528050:blk_1073943452_202671. Target Replicas is 3 but found 2 live replica(s), 0 decommissioned replica(s), 0 decommissioning replica(s).
. . .
. . .
. . .
Total size: 64294925737 B (Total open files size: 672396467 B) Total dirs: 2743 Total files: 2439 Total symlinks: 0 (Files currently being written: 22) Total blocks (validated): 2752 (avg. block size 23362981 B) (Total open file blocks (not validated): 26) Minimally replicated blocks: 2752 (100.0 %) Over-replicated blocks: 0 (0.0 %) Under-replicated blocks: 22 (0.7994186 %) Mis-replicated blocks: 0 (0.0 %) Default replication factor: 3 Average block replication: 2.9920058 Corrupt blocks: 0 Missing replicas: 22 (0.26647288 %) Number of data-nodes: 3 Number of racks: 1FSCK ended at Fri Jul 26 14:37:29 CST 2019 in 48 milliseconds
根据打印的块名称，找到文件系统中存储的块文件和块meta信息，将快文件和meta信息删除掉。
hadoop debug recoverLease -path $filePath -retries 10 恢复filePath对应的文件，如：/.../c9ddcb18-51c0-4fa7-bdab-daa5079bc094/rdd-1061487/part-00000
由于恢复块太多，使用脚本批量执行
#!/bin/bash file_name=$1 path=$2 if [ -z "${file_name}" ];then echo "Pls input file path" exit 1 fi cat ${file_name}| while read line do if [ -z $line ]; then continue fi hadoop debug recoverLease -path $line -retries 10 done
完成恢复后需要重启该节点后，cdh界面无效块告警消除
转载于:https://www.cnblogs.com/liwutao/p/11265908.html
