Basic Linux Commands II
In this section, we will review more commands to get you comfortable do basic things on HYAK. This section has sampled some of the data and exercises for this tutorial were sampled from The Unix Shell by Software Carpentry, but have been tailored to fit most HYAK users. Sampled materials are under the Copyright of Software Carpentry and are made available under the Creative Commons Attribution license (CC BY 4.0).
head
, tail
, more
, less
#
Other ways to view file contents, great for viewing smaller chunks of longer files.
By default, the head and tail commands display the first and last 10 lines of a file. To view a specific number of lines use -n
followed by the desired number of lines.
cp
#
cp
#
"copy" files with Move to top of working directory.
Copy animals.csv
to your current directory using the shorthand .
to mean "here"
Copy a directory with all its contents using recursive copy
mv
#
mv
#
"move" file to a new name (rename) with rm
#
rm
#
"remove" a file with warning
rm
permanently deletes a file. This action is irreversible.
Remove a directory with recursive rm.
touch
#
touch
#
Create an empty file with touch
is also useful for updating the timestamp of a file#
?
and []
#
#
Wildcards are special characters used as a shorthand*
wc
#
wc
#
apply "word count" with wc -l
counts lines#
>
#
>
#
"redirect" output to a file with caution
If the file already exists, it will be overwritten.
>>
#
>>
#
"append" output to a file with First, change to the animal-counts/
directory and print the contents with the cat
command
The espected output is as follows:
Using wc -l
to count the lines of the above output
To create a subset of animals.csv
, we can combine head
, tail
, >
, and >>
commands.
animals-subset.csv
should now contain the first 3 lines of animals.csv
:
Because we do not want to override animals-subset.csv
file contents, use >>
to append rather than >
:
Now, the subset reads as follows:
Counting the lines once more:
Expected output:
sort
#
sort
#
"sort" numbers with The expected output:
The sort
command by itself will sort a text file's contents in ascending order:
Notice how the output is sorted in lexicographical order:
To sort the numbers based on their numerical value, use the -n
option
Now, the file contents will be interpreted as numbers rather than strings and numbers.txt
will be sorted in ascending numerical order:
tip
The sort
command will sort lines alphabetically or numerically. Lines with numbers and letters are sorted as follows:
- Numbers
- Capital letters
- Lowercase letters
Other sorting options include -r
which sorts lines in reverse order and -u
which removes duplicate lines.
Going back to the alkanes
directory
Expected output:
Create a sorted list of the alkane lengths and check the first output line:
caution
Empty file resulting from >
. The following code will cause the contents of lengths.txt
to be deleted:
Notice how lengths.txt
is now an empty file. To get lengths.txt
back, backtrack to the original command:
|
#
#
Use a "pipe" to string commands togetherAlternatively, the alkane lengths can sorted by using one line of code:
The output should be the same at sorted-lengths.txt
:
grep
#
grep
#
Search for a PATTERN in a FILE with grep
finds and prints lines in files that match a pattern
Only the lines containing "not" will print out. Now try the follwing grep
command:
Notice how longer words containing "The" are also printed out. Use the -w option to ensure that grep is only matching whole words:
grep
also works with multiple words at a time:
Other options include -n which displays line numbers and -r which recursivly searches for patterns:
Combining options -n and -w:
The -r (recursive) option:
This searches for the word Yesterday in your current directory (writing).
history
#
history
#
View your history of commands with Use history
, |
, and grep
together to find all times the cat
command was used.
find
#
file
#
"find" files or files matching a pattern with This will print out the location of numbers.txt :
Use *
to find all .txt files starting in your current ditectory :
Use grep
and find
togther to search for files with specific words or phrases:
The output shows the file location and the lines with the word "searching" :
scp
#
klone
with "server copy" or scp
#
Transfer your data to #
From your local computerFind a file you would like to transfer. Let's say it is called text.txt. Use the following to transfer text.txt from your local computer to klone
. We will transfer it to our working directory in gscratch/scrubbed
klone
to your local computer with "server copy" or scp
#
Transfer data from #
To your local computerrsync
#
Similarly, data can be transferred using the rysnc
command
rsync
and scp
can be paired with other options such as -v
(verbose) which provides a detailed output of the transfer process or -z
( -c
for scp
) to compress data during transfers.
important concept: scp
vs rsync
rsync
is generally used for larger file transfers and file synchronization. Unlike scp
which always transfers the entire file, rsync
will only transfer the parts of the file that changed. rsync
can also resume aborted transfers from lost internet connections. scp
works well for quick file transfers that do not require the additional features rsync provides.
#
Well done!You made it through the tutorial! If you have any questions about the content covered here or feedback to pass along about this tutorial, please open a ticket by emailing help@uw.edu with "Hyak" in the subject line.