Monday, 27 July 2015

Deleting the file older than certain data in HDFS

The following command will delete the files before certain date  
hadoop fs -ls shakespeare | tail -n+2 | xargs -n 8 | awk '{  if ( ($(date)-$(date -d $6)) == 0) 
            cmd="hadoop fs -rm " $8; 
           system(cmd) 
          } 

1 comment: