AWK Cheat Sheet

Krishna Neupane

AWK Cheat Sheet

Introduction to AWK

AWK is a powerful text processing language, ideal for manipulating data files, extracting and formatting data, and producing reports.

  • Purpose: Text processing, data extraction, and reporting.

AWK Commands

Print All Lines

  • awk '{print}' file.txt
  • Prints all lines in file.txt.

Print Specific Field

  • awk '{print $2}' file.txt
  • Prints the second field of each line in file.txt.

Sum a Column

  • awk '{sum += $1} END {print sum}' file.txt
  • Sums up the numbers in the first column of file.txt.

Print Lines Matching a Pattern

  • awk '/pattern/ {print}' file.txt
  • Prints lines that match 'pattern'.

Print Lines with More than N Fields

  • awk 'NF > N' file.txt
  • Prints lines with more than N fields.

Print Line Number with Each Line

  • awk '{print NR, $0}' file.txt
  • Prints each line preceded by its line number.

Print Last Field of Each Line

  • awk '{print $NF}' file.txt
  • Prints the last field of each line.

Replace or Substitute Text

  • awk '{gsub(/pattern/, "replacement"); print}' file.txt
  • Replaces 'pattern' with 'replacement' in file.txt.

Print Lines Where a Field Matches a Pattern

  • awk '$N ~ /pattern/' file.txt
  • Prints lines where the Nth field matches 'pattern'.

Split a Line into Array

  • awk '{split($0, array, ":"); print array[1]}' file.txt
  • Splits each line by ':' and prints the first element.

Complex Data Processing

Pass Shell Variable to AWK

  • awk -v var="$shell_var" '{print var, $0}' file.txt
  • Passes a shell variable shell_var to AWK.

Perform Arithmetic Operations

  • awk '{print $1 * $2}' file.txt
  • Multiplies the first and second fields.

Print Lines If a Field is Within a Range

  • awk '$1 > 5 && $1 < 10' file.txt
  • Prints lines if the first field is between 5 and 10.

Conditional Statements

  • awk '{if ($1 > 10) print $1; else print $0}' file.txt
  • Prints the first field if it's greater than 10, otherwise prints the whole line.

Loop Through Fields in a Line

  • awk '{for (i=1; i<=NF; i++) print $i}' file.txt
  • Prints each field of a line on a new line.

Advanced Text Processing

Print Lines Whose Field Matches a Regular Expression

  • awk '$1 ~ /^[a-zA-Z]+$/' file.txt
  • Prints lines where the first field contains only letters.

Aggregate Data Based on a Field

  • awk '{arr[$1] += $2} END {for (i in arr) print i, arr[i]}' file.txt
  • Sums the second field grouped by the first field.

Print Every Nth Line

  • awk 'NR % N == 0' file.txt
  • Prints every Nth line.

Print Lines Longer Than N Characters

  • awk 'length($0) > N' file.txt
  • Prints lines longer than N characters.

Convert CSV to Tab-Delimited

  • awk 'BEGIN {FS=","; OFS="\t"} {print}' file.csv
  • Converts a CSV file to tab-delimited format.

Tips for Using AWK

  • Regular Expressions: Mastering regular expressions enhances AWK's power.
  • Script Files: For complex operations, write AWK commands in a script file.
  • Testing: Test AWK commands on sample data before applying to important files.

Follow me on : Medium Linkedin Researchgate