UNIX Command Line

Input file is comma separated.

$ cat file.txt
hiddenhausen,99.60,y
herbstein,99.021,n
bangalore,98.82,y
golm,98.8,y
para,98.82,n
bogen,98.61,n
saintandre,98.5,n
delhi,98.61,y
hyderabad,99.02,y

Lets try to format the above file:

$ awk '{ printf "%-15s%-8s%s\n",$1,$2,$3}' FS=\, file.txt

hiddenhausen   99.60   y
herbstein      99.021  n
bangalore      98.82   y
golm           98.8    y
para           98.82   n
bogen          98.61   n
saintandre     98.5    n
delhi          98.61   y
hyderabad      99.02   y

So its printing first field ($1) as a string of 15 characters that are left-justified, 2nd field ($2) as 8 characters left-justified and then third field ($3)

With variable number of fields in each line, a generic solution would be:

$ awk '
{
  for(i=1;i<=NF;i++)
     printf("%-15s%c", $i, (i==NF) ? ORS : "")
}' FS=, file.txt

hiddenhausen   99.60          y
herbstein      99.021         n
bangalore      98.82          y
golm           98.8           y
para           98.82          n
bogen          98.61          n
saintandre     98.5           n
delhi          98.61          y
hyderabad      99.02          y

Related post:
-Formatting fields into columns in awk

Input file:

$ cat cnr.txt
HIDDENHAUSEN|99.60
FIEBERBRUNN|99.07
MELLENDORF|99.04
HERBSTEIN|99.02
ACHTERWEHR|98.82
GOLM|98.82
PARA|98.82
BOGEN|98.61
SAINTANDRE|98.55
OLSZTYN|98.61
HYDERABAD|99.02

Output required: Print those lines for which 2nd field has occurred more than once. i.e. required o/p:

HERBSTEIN|99.02
ACHTERWEHR|98.82
GOLM|98.82
PARA|98.82
BOGEN|98.61
OLSZTYN|98.61
HYDERABAD|99.02

Awk solution:

$ awk 'NR==FNR && a[$2]++ {b[$2];next} $2 in b' FS="|" cnr.txt cnr.txt

Related posts:
- Difference between awk NR and FNR variables
- Posts on awk NR==FNR
- Awk FNR variable usage example
- Remove duplicates based on field using awk
- Remove duplicates from file without sorting using awk

Input string:

1234 5678

o/p required:

1
2
3
4

5
6
7
8

Using -w option of Linux fold (wrap each input line to fit in specified width)

$ echo "1234 5678" | fold -w1

Awk solution : Make field separator (FS) as NULL so that each character represents a field

$ echo "1234 5678" | awk 'BEGIN{FS=""}{for(i=1;i<=NF;i++)print $i}'

Using bash:

#!/bin/sh

str="1234 5678"
while [ -n "$str" ]
        do
                printf "%c\n" "$str"
                str=${str#?}
done

Using sed (Using & as matched string)

$ echo "1234 5678" | sed 's/[0-9 ]/&\n/g'

vimdiff
a powerful file comparison tool (edit two or three versions of a file with Vim and show differences)

Two important window commands to close windows while performing vimdiff on multiple files. (Also applicable when you open multiple files using :split or :vsplit)

ctrl-w c - Close this window
ctrl-w o - close all Other windows (mnemonic - Only)

Two of vimdiff important commands:

do - (diff-obtain) obtain differences from other window into this one
dp - (diff-put) push differences in current window to the other window

In order to remove all duplicate lines (rows) from a file which is opened in vi editor,

Use the following command

:sort u

which will sort all lines and remove duplicates and will keep only unique lines.
Other solutions for removing duplicate lines from vi editor can be found here

Related posts:
- Remove duplicate consecutive fields or lines from file
- Remove duplicate blank lines using awk in bash
- Remove duplicates based on fields using awk
- Remove duplicate lines without sorting file using awk