unix grep utility cs465. the grep utility grep stands for globally search for a regular expression...
Post on 20-Dec-2015
238 Views
Preview:
TRANSCRIPT
The grep utility• grep stands for globally search for a regular expression
and print the results
• It is one of the most used Unix tools. It has even added to the Unix user's vocabulary:
– Verb: “Grep through the files to see what should be changed.”
– Adjective: “The projx file is grepped source code.”
– Noun: “Grepping is the best way to find that information.”
grep commands
• grep is actually a family of commands– fgrep– grep– egrep
• All three search files for strings which match specified patterns:
fgrep – pattern must be a fixed string
grep – pattern can include regular expressions
egrep – pattern can include extended regular expressions
Simplest grep
– Searches all files in the file-list
– Displays the filenames of the files which contain the fixed-pattern, along with the line in the file that the pattern was found on.
– If you list only ONE filename in the file-list, fgrep will NOT include the filename in the results
• The simplest grep command is fgrep:
$ fgrep fixed-pattern [file-list]
fgrep Example
• Search for all files in the current directory that contain the string "main".
• Example:$ fgrep main *
memo: The main point is that the
new.c:main()
prog1.c:main()
$
More fgrep examples
Display all lines in file prog.c containing “num”:$ fgrep num prog.cnum = 0;while ( num < 5 ) {num = num + 1;$
Display info on all users lines containing “small”:$ fgrep small /etc/passwdsmall000:x:1164:102:Faculty – Pam Smallwood:/export/home/small000:/bin/ks
$
grep format$ grep [options] pattern [filelist]
– Search for specified pattern in each line of specified files – Send lines containing pattern (or other info) to the standard
output (i.e. display them)
• Options:-c display only count of matching lines
-h outputs matched lines but not filenames-i ignore case when matching-l display names of files only (no matching lines)-n display line numbers
-s suppresses error messages for nonexistent or unreadable files
-v display only non-matching lines
-w restricts pattern to matching a whole word
grep examples
$ cat soccer.txt
In Soccer,
There are no time outs.
There are no helmets,
no shoulder pads,
no commercial breaks,
no warm dugouts,
no halftime extravaganzas.
So if that’s what you need,
play another sport.
$ grep –n are soccer.txt
2:There are no time outs.
3:There are no helmets,
$ grep are soccer.txt
There are no time outs.
There are no helmets,
$ grep –c are soccer.txt
2
grep examples
$ cat soccer.txt
In Soccer,
There are no time outs.
There are no helmets,
no shoulder pads,
no commercial breaks,
no warm dugouts,
no halftime extravaganzas.
So if that’s what you need,
play another sport.
Are you ready?
$ grep –v no soccer.txt
In Soccer,
So if that’s what you need,
Are you ready?
$ grep Are soccer.txt
Are you ready?
$ grep –vc no soccer.txt
3
$ grep –i Are soccer.txt
There are no time outs.
There are no helmets,
Are you ready?
grep examples
$ cat soccer.txt
In Soccer,
There are no time outs.
There are no helmets,
no shoulder pads,
no commercial breaks,
no warm dugouts,
no halftime extravaganzas.
So if that’s what you need,
play another sport.
Are you ready?
$ grep –vw no soccer.txt
In Soccer,
So if that’s what you need,
Play another sport.
Are you ready?
$ grep –l soccer *
$ grep –l Soccer *
soccer.txt
$ grep –li soccer *
soccer.txt
grep examples
$ cat team1
Rob Murray
Scott Stewart
Martin Jones
Scott Smith
$ cat team2
Scott Jones
Richard Shepard
Doug Stringfellow
John English
$ grep Scott team1 team2
team1:Scott Stewart
team1:Scott Smith
team2:Scott Jones
$ grep –l Scott team1 team2
team1
team2
$ grep –h Scott team1 team2
Scott Stewart
Scott Smith
Scott Jones
grep examples
$ cat team1
Rob Murray
Scott Stewart
Martin Jones
Scott Smith
$ cat team2
Scott Jones
Richard Shepard
Doug Stringfellow
John English
$ grep Scott team1 taem2
team1:Scott Stewart
team1:Scott Smith
grep: can’t open taem2
$ grep –s Scott team1 taem2
team1:Scott Stewart
team1:Scott Smith
grep examples
$ cat team1
Rob Murray
Scott Stewart
Martin Jones
Scott Smith
$ cat team2
Scott Jones
Richard Shepard
Doug Stringfellow
John English
$ grep Scott team*
team1:Scott Stewart
team1:Scott Smith
team2:Scott Jones
$ grep –c Do team* | grep ":0"
team1:0
$ grep –c Doug team*
team1:0
team2:1
More grep examples
Print non-commented lines in file myfile (i.e. lines that do NOT start with the string "#")$ egrep -v "^#" myfilename=billecho $name$
Search files in sub subdirectory for string “test” (ignore case)$ grep –i test `ls /sub`ltr: Test for todaymbox:Subject: testmakefile: test1: test1.cmakefile: gcc test1.c -o test1$
More grep example
Determine number of users in the projectX group:$ grep projectX /etc/group
projectX:x:507:Plin_9318,Fyusuf_9287,Rlee_8656,Rdeich_1254,Njuwal_5960,Mmelto_8858,Wbucki_6698,Tespin_0604,Psmallwo_000
$
-c shows matching count-c shows matching count-c shows matching count-c shows matching count-c shows matching count-c shows matching count
$ grep -c 507 /etc/passwd
9
$
Searching for Multiple Strings
-f option– If you have multiple strings that you want to
search for, you can put all the strings into a string file, and use:
$ cat stringfilepattern1pattern2$ grep –f stringfile filelist
grep examples$ cat soccer.txt
In Soccer,
There are no time outs.
There are no helmets,
no shoulder pads,
no commercial breaks,
no warm dugouts,
no halftime extravaganzas.
So if that’s what you need,
play another sport.
Are you ready?
$ cat words
Soccer
dugouts
helmets
$ grep –f words soccer.txt
In Soccer,
There are no helmets,
no warm dugouts,
$ grep -v –f words soccer.txt
There are no time outs.
no shoulder pads,
no commercial breaks,
no halftime extravaganzas.
So if that’s what you need,
play another sport.
Are you ready?
grep Exercises
• Display all lines of file “test” that do not contain the string “and”, ignoring case
$ grep -iv "and" test
• Display a count of all of the lines of each “.c” file in current directory that contain the strings “num” or “number”
• Answer:$ cat stringsnumnumber$ grep -c –f strings *.c
Advanced grep
• grep is much more powerful when used with regular expressions to match more complex strings.
• Regular expressions are strings of characters and special symbols that are used to match other strings.
Regular Expressions• A pattern matching string is called a regular
expression (RE)
• grep (and other Unix utilities) can use REs
• Regular Expression Metacharacters:
. (period) match any single character, except newline (similar to wildcard ?)
* (asterisk) match any number (including zero) of the preceding character
.* match any number of any character
Not Filename Expansion!
• Although there are similarities to the metacharacters used in filename expansion
• this is different!
• Filename expansion is done by the shell.
• Regular expressions are used by commands (programs).
More RE Metacharacters
^ (caret) match start of line
$ (dollar) match end of line
Character Sets:
[ ] match any of enclosed characters
[^ ] match anything BUT the enclosed
NOTE: If the caret (^) is anywhere inside a character set except right after the opening bracket, it has no special meaning
Character Set Ranges
• The hyphen (-) character can be used with the square brackets to indicate a range of characters:
[0-9] is the same as [0123456789]
[a-z] is the same as [abcd...wxyz]
• If the hyphen is placed at the beginning or end of the character set, it has no special meaning (and will match a hyphen in the string)
Other Characters• Any character other than a metacharacter will accept
one of itself:
– A single letter a in a regular expression will accept a single letter a in a string
– This is assumed to be case-sensitive; lowercase a doesn't accept uppercase A
• Use a backslash to turn OFF metacharacter processing (i.e. match a metacharacter to its real value)
• Use quotes around Regular Expressions to prevent SHELL metacharacter interpretation.
Example Pattern Matches
• Some sample regular expressions and what they match:"abc" matches the string abc
"^abc" abc at the beginning of a line
"abc$" abc at the end of a line
"^abc$" abc as the entire line
"[Aa]bc" abc or Abc
"a[aeiuo]c" a, lowercase vowel, c
"a[^aeiou]c" a, not lowercase vowel, c
Example Pattern Matches• More regular expressions and what they match:
"[x-z]" matches x or y or z
"[x\-z]" matches x or - or z
"[xz-]" matches x or - or z (same)
"[.c]" matches any character followed by a c
"[\.c]" matches .c
"[a-zA-Z0-9]" matches any letter or digit
"[^0-9]" match any non-digit
"[^\^]" match any single character except ^
More Pattern Matches
"[Pp][Aa][Mm]"
– Matches “Pam" or “pam" or “pAM“
– Does not match "am" or “pa“
"[abc]*"
– matches "aaaaa" or "acbca“
"0*10"
– matches "010" or "0000010" or "10"
grep examples$ cat pattern
Background is black, and white.
I love red,
and I love blue,
but not yellow.
$ grep –i "^b" pattern
Background is black, and white.
but not yellow.
$ grep –i b pattern
Background is black, and white.
And I love blue,
but not yellow.
$ grep "^b" pattern
but not yellow.
$ grep "\." pattern
Background is black, and white.
But not yellow.
$ grep "." pattern
Background is black, and white.
I love red,
and I love blue,
but not yellow.
grep examples$ cat pattern
Background is black, and white.
I love red,
and I love blue,
but not yellow.
$ grep "," pattern
Background is black, and white.
I love red,
and I love blue,
$ grep ",$" pattern
I love red,
And I love blue,
grep examples$ cat pattern2
rd
reed
red
reef
ref
reep
$ grep "re[df]" pattern2
red
ref
$ grep "re*d" pattern2
rd
reed
red
$ grep "f$" pattern2
reef
ref
$ grep "re*[dp]" pattern2
rd
reed
red
reep
More grep examples (using RE)
Display names of all files in this directory that refer to Unix (or unix)
$ grep -l "[Uu]nix" *mboxmyfile.txtscript2$
List soft-linked files only:$ ls –l | grep "^l"lrwx------ 2 small000 faculty 512 Jun 4 13:04 t1 -> t
lrwx------ 2 small000 faculty 512 Jun 2 13:43 t2 -> t
$
grep in a script
Display long list of files, then the number of "old" files (files last accessed in 2007):
$ cat oldfiles.ksh#! /bin/kshls -l > listfilenum=`grep 2007 listfile | wc -l`echo Number of old files: $numrm listfileexit 0$
Extended Regular ExpressionAdditional Metacharacters(available only with egrep)
+ match any number (greater than zero) of preceding character
? match zero or one instances of preceding character
| combines REs with either-or matching
( ) groups pattern matching characters
egrep examples$ cat pattern2
rd
reed
red
reef
ref
reep
$ egrep "re+d" pattern2
red
reed
$ egrep "re?d" pattern2
rd
red
$ egrep "re+[dp]" pattern2
red
reed
reep
$ egrep "re?[df]" pattern2
rd
red
ref
grep/egrep examples$ cat pattern
Background is black, and white.
I love red,
and I love blue,
but not yellow.
$ egrep "I|a" pattern
Background is black, and white.
I love red,
and I love blue,
$ grep "red|blue" pattern
$ egrep "red|blue" pattern
I love red,
and I love blue,
$ egrep "^I|^a" pattern
I love red,
and I love blue,
Extended RE Examples
[abc]+d matches "aaaaad" or "acbcad "
but does NOT match "d"
0+10 matches "010" or "0000010"
but does NOT match "10"
x[abc]?x matches "xax", "xbx", "xcx" or "xx"
A[0-9]?B matches "A8B" or "AB“
but does NOT match "a8b" or "A123B"
Grouping
• The parentheses special characters ( and ) can be used to group several subexpressions together and apply a suffix to them as a group:
ba+d accepts bad, baad, baaad, etc
(ba)+d accepts bad, babad, bababad, etc
(ba)+(cd)+ accepts bacd, babacd, bacdcd, bacdcdcdcd, etc
Alternatives• The (|) "either-or" choice can also be "grouped"
aa|bb will accept either aa or bbfr(ie|ei)nd will accept friend or freind, and
nothing else
• There can be any number of choices:m(a|e|ai|oo)n will accept man, men, main, or moon
• If all the choices are single characters, then you might as well use a character set:p(a|e|i)n will accept pan, pen, or pin, and is
equivalent to p[aei]n
More egrep Examples
• You must use egrep in order to have access to the EXTENDED regular expressions:
$ egrep 'ab+c?d' filematch lines with a followed by any number of
b’s and and optional c followed by a d
$ egrep '(ab)+c?(de)+' filematch lines with any number of ab’s and optional c
followed by any number of de’s
top related