This cheat sheet outlines some of the main Hadoop commands that we’ve found useful while building our Cloudways alternative hosting service.
Generic
- hadoop fs -ls <path> list files in the path of the file system
- hadoop fs -chmod <arg> <file-or-dir> alters the permissions of a file where <arg> is the binary argument e.g. 777
- hadoop fs -chown <owner>:<group> <file-or-dir> change the owner of a file
- hadoop fs -mkdir <path> make a directory on the file system
- hadoop fs -put <local-origin> <destination> copy a file from the local storage onto file system
- hadoop fs -get <origin> <local-destination> copy a file to the local storage from the file system
- hadoop fs -copyFromLocal <local-origin> <destination> similar to the put command but the source is restricted to a local file reference
- hadoop fs -copyToLocal <origin> <local-destination> similar to the get command but the destination is restricted to a local file reference
- hadoop fs -touchz create an empty file on the file system
- hadoop fs -cat <file> copy files to stdout
Yarn commands
- yarn node -list list nodes in the yarn cluster
- yarn node -status <node id> status of a node (memory used, free, number of containers, etc) for <node id> (first column from command above)
- yarn application -list list of Yarn applications and their state
- yarn logs -applicationId <appid> dump the logs for a particular application
Configuration commands
- hdfs getconf return various configuration settings in effect
- hdfs getconf -namenodes namenodes in the cluster
- hdfs getconf -confkey <a.value> return the value of a particular setting (e.g. dfs.replication)
HDFS commands
- hdfs dfsadmin -safemode get find out if you’re in safemode
- hdfs dfsadmin -report find out how much disk space us used, free, under-replicated, etc.
Want to speed up your web site?
WP Bolt makes it easy and affordable to have a High Performance WordPress VPS server.