UNIX Command Line

Assume you have a requirement of running two commands on 10 hosts.

In this post I would like to show you how you can:

1) Run multiple commands on remote host using ssh.

2) Also instead of working on one host at a time serially, how can we execute the commands on multiple hosts in parallel to save the overall execution time.

Example:
Find the number of files under '/sdb/_read/' and '/sdb/_write/' directories on following 10 hosts.

$ cat host.txt 
192.19.66.203
192.19.66.204
192.19.66.205
192.19.66.206
192.19.66.207
192.19.66.208
192.19.66.209
192.19.66.210
192.19.66.211
192.19.66.212

Serial execution: 
We can write a bash for loop and work on each host like this:

$ for ip in $(cat host.txt); do echo -n "$ip|"; ssh -2 root@$ip 'R=$(ls /sdb/_read/ | wc -l ) ; D=$(ls /sdb/_write/ | wc -l) ; echo "$R|$D"'; done

Time taken:

real 0m50.084s
user 0m0.052s
sys 0m0.032s

Parallel execution:
I have already blogged about Unix xargs parallel execution option here, using that:

$ cat host.txt  | xargs -n 1 -P 5 -i ssh -2 root@{} 'I=$(hostname -i);R=$(ls /sdb/_read/ | wc -l ) ; D=$(ls /sdb/_write/ | wc -l)  ; echo "$I|$R|$D"'

Time taken:

real 0m10.437s
user 0m0.044s
sys 0m0.036s

10 Seconds vs 50 Seconds, wow !

Output:

192.19.66.205|0|4
192.19.66.204|0|0
192.19.66.203|0|0
192.19.66.207|5|0
192.19.66.206|0|0
192.19.66.208|0|0
192.19.66.209|0|7
192.19.66.210|0|0
192.19.66.211|2|1
192.19.66.212|9|0

Note: Utilities like GNU parallel or Parallel SSH are always the best tool in this job.

Related posts:

1. Unix Xargs paralel execution
2. Unix - Execute script on remote host using SSH without copying the script.
3. Diff remote files using SSH in Unix
4. Bash and SSH some examples