In this topic, we described about the below sections -
During the first time logon to the configured HDFS file system, open name node (HDFS server) and execute the following command to format it.
$ hadoop namenode -format
After formatting, need to start the distributed file system. The command to start the name node and the data nodes is -
Listing Files in HDFS
To list of files in a directory, status of a file using ‘ls’. Below syntax of ls to list all files from a directory or a filename as an argument.
$HADOOP_HOME/bin/hadoop fs -ls <args>
hadoop fs -mkdir:
It will take path uri’s as argument and creates directory or directories.
$HADOOP_HOME/bin/hadoop fs -mkdir <paths>
Space utilization in a HDFS dir.
hadoop fs -du:
Displays sizes of files and directories contained in the given directory or the size of a file if it’s just a file.
$HADOOP_HOME/bin/hadoop fs -du URI
Copy single src file, or multiple src files from local file system to the Hadoop data file system.
$HADOOP_HOME/bin/hadoop fs -put <localsrc> ... <HDFS_dest_Path>
- <localsrc> -- Local source path
- <HDFS_dest_Path> -- HDFS destination path
Copies/Downloads files to the local file system.
$HADOOP_HOME/bin/hadoop fs -get <hdfs_src> <localdst>
- <hdfs_src> -- HDFS source path
- <localdst> -- Local Destination to copy
Help command to get list of commands supported by Hadoop Data File System(HDFS).
$HADOOP_HOME/bin/hadoop fs -help
Inserting Data into HDFS
The file called filename.txt in the local system to be saved in the hdfs file system. Follow the steps given below to upload the required file in the Hadoop file system.
Create an input directory.
$HADOOP_HOME/bin/hadoop fs -mkdir /usr/input
Transfer and store a data file from local systems to the Hadoop file system using the put command.
$HADOOP_HOME/bin/hadoop fs -put /home/filename.txt /usr/input
Verify the file using ls command.
$HADOOP_HOME/bin/hadoop fs -ls /usr/input
Retrieving Data from HDFS
File in HDFS called outfile required to download from Hadoop file system. Given below are the steps to do it.
Initially, view the data from HDFS using cat command.
$HADOOP_HOME/bin/hadoop fs -cat /usr/output/outfile
Get the file from HDFS to the local file system using get command.
$HADOOP_HOME/bin/hadoop fs -get /usr/output/ /home/download/
Shutting Down the HDFS
Shut down the HDFS by using the following command.