The grep command is a hugely powerful way to search through files. Like many command line utilities, once you're comfortable using it, you will discover that it is surprisingly fast and accurate.

However, many Linux users only bother to learn one or two grep options and then use them as a kind of one-size-fits-all approach to searching. A little time spent learning what grep can do will pay dividends – and there's nothing more satisfying than knowing exactly how to use a command to find something in a jiffy.

We'll start out with the basics and build up a repertoire of ways to use grep, before finishing with some things to check out if you're still hungry for more.

The examples that follow use the OpenBSD calendar files. These are already installed by default on Ubuntu, Mint, and even the Mac, along with many other distros.

However if you are a Fedora user, and you want to follow along, you can add them manually by selecting the System > Administration > Add/Remove Software option from your desktop. Search for calendar, check the box next to Reminder Utility and click Apply. Or, just run yum install calendar on the command line. Now for the important stuff.

Basic searches

The grep command looks for the following things:

1. Any options you might use to tailor your search.
2. The string (or pattern) you are looking for.
3. A location in which to search – either a file or a directory.

As a quick example, try:

grep first /usr/share/calendar/calendar.history

Let's examine exactly what's happening here. We've asked grep to find all instances of the string 'first' in the file calendar.history.

Bear in mind that grep is case sensitive – compare the different set of results you get if you run:

grep First /usr/share/calendar/calendar.history

If you want your search to return everything, regardless of capitalisation, use the -i option to ignore case.

grep -i first /usr/share/calendar/calendar.history

All good so far.

Now, what if you want to search in more than one file? There are many calendar files we might want to look in, so change the file path to a directory and then use the -r option so that grep searches recursively through all the files and subdirectories it finds under the specified directory:

grep -ir first /usr/share/calendar

Each line of output is now prefixed with the name of the file. Note that the order in which you add options after the '-' character does not matter.

Sometimes the examples above are exactly what you need; if you're looking for a letter you wrote to someone called Don Jenkins, and you know that it's somewhere in your home directory, you can probably find your file with:

grep -r Jenkins /home/faye

However, what if you run your grep command, but you don't get the results you expect, or you get so many matches that you can't tell one line from another?

Customise your output

Let's make it a bit easier to see what we're doing by turning on grep's highlighting option. This time we'll search for 'war'.

grep -ri --color=auto war /usr/share/calendar/

That's better, now you can see the actual string you are searching for (if you are running Linux Mint, highlighting is turned on by default).

grep highlighting

BRIGHTEN YOUR SEARCH: Highlighting your grep output brightens your terminal and allows you to see the wood for the trees

Next, let's make sure we aren't picking up any substrings, since grep will list all matches even if they are part of another word. For example, our search for war is also picking up a line about Rod Stewart.

You can prevent this by requesting whole word matches only, with the -w option. This reduces the number of matches by two thirds. We know this because you can use grep to tally up instances, rather than report them directly, by adding the -c option:

grep -riwc war /usr/share/calendar

You can see that grep outputs the number of matches for each file it searches. If you were searching a single file, it would simply return a single number.

One last thing before we move on: if you want to search for more than one word, you need to use single quotes to retain the whitespace:

grep -ri 'civil war' /usr/share/calendar

That's all well and good, but when I search for something like "$20", it all goes horribly wrong. What's happening? OK, there's something else about grep that you need to know, and that's the fact that it's designed to match patterns with regular expressions.

grep regular expression

SEARCH EXPRESSIONS: The $ sign is being interpreted here as part of a regular expression. One small adjustment and I can easily find the transactions we're looking for

This is a huge topic all of its own, but here's a heads-up. Characters like $*.?+ have their own special meaning, and enable you to search for complex and precise patterns in files. If you are searching for something that contains one of these special characters, you need to 'escape' it using a backslash directly before the character:

grep -ri \$20 /home/faye/statement.txt

If you don't do this, grep will interpret the character as more than just a literal, and the output you get may, or may not, return the results you were hoping for.

Finally, if you want to save a search, you can redirect your grep output into a file as follows:

grep -riw first /usr/share/calendar > /home/faye/search.txt

Beware that if you are saving the file in the same directory, or subdirectory, as the one in which you are recursively searching, you can end up stuck in a loop with grep returning output from the file it is creating. If you do this by accident, just Ctrl+C out of the loop, and make sure that you delete the output file, as it will be massive.

You'll find grep indispensable as you become a seasoned Linux user, but if you really want to master this command there are a couple of further steps you need to take.

Firstly, make use of the man page (press Q to exit once you've finished reading):

man grep

This will give you a detailed reference to all grep's options. You should also try using the help command:

grep --help

which is perfect for a quick reminder.

They can tell you how to show line numbers (brilliant for source files), display filenames only, print contextual lines above and below matches, and even return everything that doesn't match what you're searching for.

Second, if you can invest some time learning how to use regular expressions, you will reap the benefits tenfold – and then the grep world really will be your oyster.

-------------------------------------------------------------------------------------------------------

First published in Linux Format Issue 133

Liked this? Then check out 6 of the best desktop search tools for Linux

Sign up for TechRadar's free Weird Week in Tech newsletter
Get the oddest tech stories of the week, plus the most popular news and reviews delivered straight to your inbox. Sign up at http://www.techradar.com/register

Follow TechRadar on Twitter * Find us on Facebook