Table of Contents

1 Shell Script Examples

Shell script is powerful if you could remember all the weird syntax. Here is the example to show how to use Array in Shell script. Note: readarray only works in Bash 4.x, check your version: bash –version

  • Array in Shell script 1

    string='Paris, France, Europe';
    echo $string
    readarray -td, array <<<"$string"; declare -p array;
    ## declare -a array=([0]="Paris" [1]=" France" [2]=$' Europe\n')
    
    country='Paris/Frence/Europe/Sweden/Croatia';
    echo $country
    readarray -td/ array <<<"$country"; declare -p array;
    for item in "${array[@]}" 
    do
      echo "$item" 
    done      
    
  • count number of argument

    echo "$#"
    var="string"
    
  • array in bash

    arr=(                            
    "dog"                            
    "cat"                            
    "rat"                            
    )                                
    for var in ${arr[*]}             
    do                               
    echo "var=$var"                  
    done                             
    
  • if statement, one liner

    var="$#" && if [ "$var" -eq 2 ]; then echo "two arguments"; else echo "not two argument"; fi
    
  • Iterate all files with one line shell command

    for name in $(ls *); do echo $name; done
    
  • String matches

    for name in $(ls *); do if [[ $name == *"txt"* ]]; then echo $name; else echo ""; fi 
    
  • Use regex match =~

    if [[ $name =~ "txt" ]]; then echo $name; else echo ""; fi
    
  • Repeat shell command with watch If you want to get constantly update information or stream data shell commands then you can use watch repeats your commands For Exmple, I want to monitor my system disk usage

    watch df -lah  # default 2 seconds interval
    

    watch shell commands with pipe, e.g. ps aux | grep --color httpd with double quotes

    watch "ps aux | grep --color httpd"
    
  • Check a variable is empty or not in shell script, check empty string

    if test -z "$1" 
    then
       echo "\$1=$1 is empty"
    else
       echo "\$1=$1 is not empty"
    fi
    # or
    if [ -z "$1" ]
    then
       echo "\$1=$1 is empty"
    else
       echo "\$1=$1 is not empty"
    fi
    

2 grep Examples

  • grep content of file with specified file name extension, show full file name

    grep --include="*.java" -Hnris pattern $PWD
    
  • cat and grep

    cat /tmp/file.txt | grep string
    
  • grep with color

    cat /tmp/file.txt | grep --color string  
    
  • grep print NUM line of trailing context file after matching lines, two line after

    cat file | grep -A 'dog'
    
  • grep print NUM lines of leading context file before matching lines, two line before

    cat file | grep -B 'dog'
    
  • grep inversion

    # find all non java file  
    grep -v -E '.java$'       
    
  • grep string that contains "txt", grep matches string

    if echo $name | grep -q "txt"; then echo $name; else echo ""; fi  
    
  • grep string its suffix is "txt", e.g. "file.txt"

    if echo $name | grep -q "txt$" then echo $name; else echo ""; fi  
    
  • grep string its prefix is "file", e.g. "file.txt"

    # grep -q => no output                                           
    if echo $name | grep -q "^file" then echo $name; echo echo ""; fi
    
  • grep in POSIX basic and extended regular expression

    # basic regex                                                    
    ps aux | grep --color 'wai\|filewatcher'                         
    # extended regex                                                 
    ps aux | grep --color -E 'wai|filewatcher'                       
    
  • kill all mysqld in MacOS, combine grep and awk

    ps aux | grep --color mysqld | awk '{print $2}' | line 'x -> kill -9 x'
    
  • grep colo escape sequence color

    grep -E --color=always -Hnris 'begin|end' *.tex | more -R
    
  • awk get the next line after match

    whois bbc.co.uk | awk -F: '/Registrar:/ && $0 != "" {getline; print $0}'
    # => British Broadcasting Corporation [Tag = BBC]
    
  • awk as filter, match line contains root or www

    cat /etc/passwd | awk '/root|www/{print}'
    

3 sed Example

// list file from line 3 to 9
sed -n 3,9p file.txt 

4 sed does not work on MacOS, sort of

  • sed on MacOS is BSD version, it does not implement regular expression extension: GNU regex Extension
  • Character class in GNU regex Extension.

    \w [a-bA-Z_] word echo "dog cat" | sed 's/\w/X/g' => "XXX XXX"
    \W   non whitespace echo "dog cat" | sed 's/\W/X/g' => "dogXcat"
    \s   whitespace echo "dog cat" | sed 's/\s/X/g' => "dogXcat"
    §   non whitespace  
    \b   match a word boundary echo "dog cat" | sed 's/\b/X/g' => "XdogX XcatX"
    \<   match the beginning of a word  
    \>   match the end of a word  
  • \d is NOT in the GNU regex expression, but it is in POSIX regex expression Extension
  • gsed can be installed on your MacOS which is GNU sed.
# does not work on MacOS or FreeBSD
echo "dog cat " | sed -E 's/\s*//g'  => "dog cat "
echo "dog cat " | gsed   's/\s*//g'  => "dogcat"
# it seems gsed does not need flag -E

5 Vim regex extension

  • Add more here

6 POSIX basic regular expression.

  • What is that?

7 POSIX extended regular expression.

Character class ASCII example desc
[[:alpha:]] [a-zA-Z] [a-zA-Z]  
[[:digit:]] [0-9] [0-9]  
[[:space:]] [ \n\r\t\v\f] whitespace  
[[:alnum:]] [0-9a-zA-Z]    
[[:print:]]     Visiable char and space char
[[:lower:]] [a-z]    
[[:upper:]] [A-Z]    
[[:graph:]]     Visiable char
[[:xdigit:]]     hexadecimal
[[:blank:]] [ \t]   Space and tab
[[:cntrl:]] contrl char    

8 rsync examples

  • rysnc file with specified extensions such png, svg etct.

    • include and exclude files.
    # rsync svg, png and jpg files excluding other files.
    rsync -rvzu --include='*'{svg,png,jpf} --exclude "*"  source  dest
    # rsync html file excluding other files.
    rsync -rvzu --include='*.html' --exclude "*"  source  dest
    # rsync folder
    rsync -artv folder  source dest
    
    # rsync local folder with remote folder
    rsync -artv local_folder user@xfido.com:/var/www/html
    

9 If and Else in Shell Script

String comparision Description
Str1 = Str2 Return true if strings are equal.
Str1 != Str2 Return true if string are NOT equal
-n Str Return true if string is NOT null
-z Str Return true if string IS null
Numeric Comparision Description
   
expr1 -eq expr2 Return true if expressions are equal
expr1 -ne expr2 Return true if expressions are NOT equal
expr1 -gt expr2 Return true if expr1 is greater than expr2
expr1 -lt expr2 Return true if expr1 is less than expr2
expr1 -ge expr2 Return true if expr2 is greater than or equal to expr2
expr1 -le expr2 Return true if expr2 is less than or equal to expr2
File Conditionals Description
-d file Return true if file is a directory
-e file Return true if file exists
-f file Return true if the provided string is a file.
-g file Return true if the group id is set on a file.
-r file Return true if file is readable.
-s file Return true if file is a non-zero size.
-u Return true if the user id is set on a file.
-w Return true if the file is writable.
-x Return true if the file is executable.
   

10 Execute remote shell script Execute remote shell

wget -O - https://bitbucket.org/zsurface/publicfile/raw/46cc8426ac08e2d575b60312fdc01f6ab6e3656c/test1.sh | bash

11 Check whether a command is in bash

command -v curl
if [ $? ]; then echo "curl exists"; else echo "curl does not exist"; fi

12 FreeBSD install ports without prompt make install clean BATCH=yes

  • ports installation is not working.
cd /usr/ports/redis && make install clean BATCH=yes

13 Install Redis On FreeBSD

pkg install -y redis

14 Enable Redis on FreeBSD

# add redis_enable="YES" to /etc/rc.conf

15 Run Redis on FreeBSD

/usr/local/etc/rc.d/redis start

16 How to avoid all the ssh-agent and ssh-add BS.

  • Add follwoing two lines to your ~/.profile, it works in FreeBSD.

    eval $(ssh-agent -s)
    ssh-add ~/.ssh/myprivate_id_rsa
    

17 Here are all the public keys:

18 Add global shared profile under /etc/profile in FreeBSD and Ubuntu.

19 In Haskell, if check whether a file is modified or not, use modificationTime

  • If the path is directory, the time will be changed if file is added or deleted.
  • See filewatcher.hs or watchDir in AronModule.hs
modificationTime :: FileStatus -> EpochTime

20 Execute remote shell command

ssh user@xfido.com  'cd user; ls;'

21 Shell single and double quotes Single and Double Quotes

22 Number of argument is wrong in shell script.

$scr/try_argument_count.sh 1 2
# => "$#" => 2
# inside Vim script
:term java_compile.sh c %
# => "$#" => 3

Author: aaa

Created: 2021-09-22 Wed 17:54

Validate