Permissions: Note that files listed by hdfs dfs are owned by the hdfs user on the Linux system. That means that files on HDFS must be written / removed / modified by user hdfs. Using root will not work.
Removing files will happen in 6 hours by default:
15/09/08 17:42:20 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion interval = 360 minutes, Emptier interval = 0 minutes. Moved: 'hdfs://node1:8020/user/root/test1' to trash at: hdfs://node1:8020/user/root/.Trash/Current
List a directory's contents
If no argument is given, and the /user/username directory exists, it will display the user's directory.
hdfs dfs -ls
hdfs dfs -ls /
Found 8 items drwxrwxrwx - yarn hadoop 0 2015-09-08 13:56 /app-logs drwxr-xr-x - hdfs hdfs 0 2015-09-08 13:58 /apps drwxr-xr-x - hdfs hdfs 0 2015-09-08 13:53 /hdp drwxr-xr-x - mapred hdfs 0 2015-09-08 13:52 /mapred drwxr-xr-x - hdfs hdfs 0 2015-09-08 13:52 /mr-history drwxr-xr-x - hdfs hdfs 0 2015-09-08 13:53 /system drwxrwxrwx - hdfs hdfs 0 2015-09-08 13:57 /tmp drwxr-xr-x - hdfs hdfs 0 2015-09-08 17:29 /user
Create a directory
As user hdfs:
hdfs dfs -mkdir /foodir
Remove a directory
As user hdfs:
hdfs dfs -rm -R /foodir
View the contents of a file
hdfs dfs -cat /foodir/myfile.txt
Recursively view the contents of a directory
hdfs dfs -ls -R
Put a file onto the filesystem
hdfs dfs -put sample.txt
Copy a file on the HDFS filesystem to another HDFS filename
hdfs dfs -cp sample.txt sample-copy.txt
Copy a file from HDFS to local filesystem
hdfs dfs -get sample.txt /tmp
cat a file
hdfs dfs -cat sample.txt
tail the last 20 lines of a file
hdfs dfs -tail sample.txt
Add a file with a specific number of replication
In this example, set replication to 2 instead of default of 3.
hdfs dfs -D dfs.replication=2 -put sample.txt sample-rep2.txt
And verify the replication value:
hdfs fsck sample-rep2.txt
Average block replication: 2.0
Immediately empty trash
hdfs dfs -expunge
Disable safe mode
HDFS will enable safe mode if the set number of possible nodes to fail have failed. See dfs.namenode.safemode.threshold-pct in hdfs-site.xml.
hdfs dfsadmin -safemode leave