{"id":6840,"date":"2020-02-11T09:08:31","date_gmt":"2020-02-11T15:08:31","guid":{"rendered":"https:\/\/www.poweradmin.com\/blog\/?p=6840"},"modified":"2019-12-06T16:21:39","modified_gmt":"2019-12-06T22:21:39","slug":"how-to-maximize-on-your-usage-of-awk","status":"publish","type":"post","link":"https:\/\/www.poweradmin.com\/blog\/how-to-maximize-on-your-usage-of-awk\/","title":{"rendered":"How To Maximize On Your Usage Of Awk"},"content":{"rendered":"

\"\"<\/a><\/p>\n

Abbreviated from the names of its developers \u2013 Aho, Weinberger, and Kernighan \u2013 Awk is a\u00a0 scripting language used in Unix or Linux environments for manipulating data and generating reports.<\/span><\/p>\n

\u00a0<\/p>\n

The awk command programming language requires no compiling, and allows the user to employ variables, numeric functions, string functions, and logical operators. In this article, we\u2019ll be looking at some examples demonstrating its many and diverse uses.<\/span><\/p>\n

Some Awk Basics<\/span><\/h2>\n

In essence<\/a>, the awk command scans a file line by line, splits each input line into fields, compares input lines or fields to a specified pattern, then performs one or more actions on the matched lines.<\/span><\/p>\n

\u00a0<\/p>\n

Awk operations are especially useful in transforming data files, and producing formatted reports.<\/span><\/p>\n

Selecting Specific Fields<\/span><\/h2>\n

The easiest and most common usage of awk<\/a> is selecting specific fields from files, or from data that is fed into the awk command line. You use white space (any sequence of blanks and tabs) as a field separator by default, and a simple example at the Linux $ prompt would be:<\/span><\/p>\n

\u00a0<\/p>\n

$ echo one two three four five | awk \u2018{print $4}\u2019<\/span><\/p>\n

four<\/span><\/p>\n

$ who | awk \u2018{print $1}\u2019<\/span><\/p>\n

jdoe<\/span><\/p>\n

fhenry<\/span><\/p>\n

\u00a0<\/p>\n

In the example above, awk is extracting just the fourth and first fields from the data provided.<\/span><\/p>\n

Using Awk To Pull Text From Files<\/span><\/h2>\n

By adding the name of the file after the awk command, you can use the mechanism to pull specific text from files. For example, to pick out the first, fifth and last words in the single line of text from the file \u201cHelenKellerQuote\u201d:<\/span><\/p>\n

\u00a0<\/p>\n

$ awk \u2018{print $1,$5,$NF}\u2019 HelenKellerQuote<\/span><\/p>\n

\u00a0<\/p>\n

The beautiful heart.<\/span><\/p>\n

\u00a0<\/p>\n

Specifying $NF in the command pulls out the last piece of text on each line. Here, NF represents the number of fields in the line (23) while $NF represents the value of that field (\u201cheart.\u201d). The period is included because it\u2019s part of the final text string.<\/span><\/p>\n

Using Awk To Print Text From Linux Files<\/span><\/h2>\n

The awk command may be used to print fields from a file in any specified order. In the example below, awk is rearranging the fields in a date command output:<\/span><\/p>\n

\u00a0<\/p>\n

$ date | awk \u2018{print $4,$3,$2}\u2019<\/span><\/p>\n

\u00a0<\/p>\n

2019 Nov 22<\/span><\/p>\n

\u00a0<\/p>\n

Note that if you leave out the commas between the field designators in an awk command, the output will be pushed into a single string.<\/span><\/p>\n

\u00a0<\/p>\n

$ date | awk \u2018{print $4 $3 $2}\u2019<\/span><\/p>\n

\u00a0<\/p>\n

2019Nov21<\/span><\/p>\n

\u00a0<\/p>\n

If you happen to replace the commas with hyphens, awk will attempt to subtract one field from another:<\/span><\/p>\n

\u00a0<\/p>\n

$ date | awk \u2018{print $4-$3-$2}\u2019<\/span><\/p>\n

\u00a0<\/p>\n

1997<\/span><\/p>\n

\u00a0<\/p>\n

Here, awk is subtracting 22 (the day of the month) from the year (2019) and simply ignoring \u201cNov\u201d.<\/span><\/p>\n

\u00a0<\/p>\n

If you want the output to be separated by something other than white space, you can specify an output separator with the OFS (output field separator) modifier like this:<\/span><\/p>\n

\u00a0<\/p>\n

$ date | awk \u2018{OFS=\u201d-\u201c; print $4,$3,$2}\u2019<\/span><\/p>\n

\u00a0<\/p>\n

2019-Nov-22<\/span><\/p>\n

\u00a0<\/p>\n

As part of a script, you can use awk to simply display some relevant text. In this example, we\u2019re using awk to display a line of text to label a set of data so that you can identify what you\u2019re looking at:<\/span><\/p>\n

\u00a0<\/p>\n

$ who | awk \u2018BEGIN {print \u201cCurrent logins:\u201d} {print $1}\u2019<\/span><\/p>\n

\u00a0<\/p>\n

Current logins:<\/span><\/p>\n

\u00a0<\/p>\n

shs<\/span><\/p>\n

\u00a0<\/p>\n

nemo<\/span><\/p>\n

Specifying Field Separators<\/span><\/h2>\n

If your text is separated by some character other than white space (commas, colons, semicolons, etc.), you can inform awk by using the -F (input separator) option as shown here:<\/span><\/p>\n

\u00a0<\/p>\n

$ cat testfile<\/span><\/p>\n

\u00a0<\/p>\n

a:b:c,d:e<\/span><\/p>\n

\u00a0<\/p>\n

$ awk -F : \u2018{print $2,$3}\u2019 testfile<\/span><\/p>\n

\u00a0<\/p>\n

b c,d<\/span><\/p>\n

\u00a0<\/p>\n

To specify a separator and pull a field from the colon-separated \/etc\/passwd file, you could use a syntax like this:<\/span><\/p>\n

\u00a0<\/p>\n

\"\"<\/a><\/p>\n

(Image source: NetworkWorld.com<\/a>)<\/span><\/p>\n

Evaluating Fields Using Awk<\/span><\/h2>\n

You can also use the awk command to take a deeper look into the contents of a data file. If for example, you want to list only the user accounts in \/etc\/passwd, you can include a test for the 3rd field. In the screenshot we\u2019re only interested in any UIDs that are 1000 and above:<\/span><\/p>\n

\u00a0<\/p>\n

\u00a0<\/p>\n

\"\"<\/a><\/p>\n

(Image source: NetworkWorld.com<\/a>)<\/span><\/p>\n

\u00a0<\/p>\n

To add a title for the listing, you can add a BEGIN clause:<\/span><\/p>\n

\u00a0<\/p>\n

\u00a0<\/p>\n

\"\"<\/a><\/p>\n

(Image source: NetworkWorld.com<\/a>)<\/span><\/p>\n

\u00a0<\/p>\n

You can use the \u201c\\n\u201d (newline characters) modifier to separate your intended output lines, if you want more than one line in your title:<\/span><\/p>\n

\u00a0<\/p>\n

\u00a0<\/p>\n

\"\"<\/a><\/p>\n

(Image source: NetworkWorld.com<\/a>)<\/span><\/p>\n

Using Awk For Math Calculations<\/span><\/h2>\n

The awk command includes several mathematical capabilities, and is able to calculate square roots, logs, tangents, and other functions. So for example:<\/span><\/p>\n

\u00a0<\/p>\n

$ awk \u2018BEGIN {print sqrt(2019)}\u2019<\/span><\/p>\n

\u00a0<\/p>\n

44.9333<\/span><\/p>\n

\u00a0<\/p>\n

This returns the square root of 2019. The syntax below calculates the logarithm of 2019:<\/span><\/p>\n

\u00a0<\/p>\n

$ awk \u2018BEGIN {print log(2019)}\u2019<\/span><\/p>\n

\u00a0<\/p>\n

7.61036<\/span><\/p>\n

\u00a0<\/p>\n

Creating Scripts With Awk<\/span><\/h2>\n

It\u2019s possible to use awk for creating standalone scripts. The example below builds on the user accounts case given earlier, but also counts the number of users with accounts on the system.<\/span><\/p>\n

\u00a0<\/p>\n

\u00a0<\/p>\n

\"\"<\/a><\/p>\n

(Image source: NetworkWorld.com<\/a>)<\/span><\/p>\n

\u00a0<\/p>\n

The BEGIN section is run only when the script starts, and provides a heading, dictates the field separator, and sets up a counter to start with 0. The script also includes an END section which only runs after all the lines in the text provided to the script have been processed. It displays the final count of lines which meet the specification in the middle section.<\/span><\/p>\n

\u00a0<\/p>\n

The awk command has a long tradition in Unix, and as we\u2019ve demonstrated, it still provides a range of uses and functionality for Linux environments.<\/span><\/p>\n

\u00a0<\/p>\n

\u00a0<\/p>\n

\u00a0<\/p>\n

\u00a0<\/p>\n","protected":false},"excerpt":{"rendered":"

Abbreviated from the names of its developers \u2013 Aho, Weinberger, and Kernighan \u2013 Awk is a\u00a0 scripting language used in Unix or Linux environments for manipulating data and generating reports. \u00a0 The awk command programming language requires no compiling, and allows the user to employ variables, numeric functions, string functions, and logical operators. In this […]<\/p>\n","protected":false},"author":10,"featured_media":6849,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[447,6,9],"tags":[],"class_list":["post-6840","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-linux","category-tech","category-technical"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/www.poweradmin.com\/blog\/wp-json\/wp\/v2\/posts\/6840","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.poweradmin.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.poweradmin.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.poweradmin.com\/blog\/wp-json\/wp\/v2\/users\/10"}],"replies":[{"embeddable":true,"href":"https:\/\/www.poweradmin.com\/blog\/wp-json\/wp\/v2\/comments?post=6840"}],"version-history":[{"count":5,"href":"https:\/\/www.poweradmin.com\/blog\/wp-json\/wp\/v2\/posts\/6840\/revisions"}],"predecessor-version":[{"id":6843,"href":"https:\/\/www.poweradmin.com\/blog\/wp-json\/wp\/v2\/posts\/6840\/revisions\/6843"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.poweradmin.com\/blog\/wp-json\/wp\/v2\/media\/6849"}],"wp:attachment":[{"href":"https:\/\/www.poweradmin.com\/blog\/wp-json\/wp\/v2\/media?parent=6840"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.poweradmin.com\/blog\/wp-json\/wp\/v2\/categories?post=6840"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.poweradmin.com\/blog\/wp-json\/wp\/v2\/tags?post=6840"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}