Table of Contents
- 1. Shell Script Examples
- 2. grep Examples
- 3. sed Example
- 4. sed does not work on MacOS, sort of
- 5. Vim regex extension
- 6. POSIX basic regular expression.
- 7. POSIX extended regular expression.
- 8. rsync examples
- 9. If and Else in Shell Script
- 10. Execute remote shell script Execute remote shell
- 11. Check whether a command is in bash
- 12. FreeBSD install ports without prompt make install clean BATCH=yes
- 13. Install Redis On FreeBSD
- 14. Enable Redis on FreeBSD
- 15. Run Redis on FreeBSD
- 16. How to avoid all the ssh-agent and ssh-add BS.
- 17. Here are all the public keys:
- 18. Add global shared profile under
/etc/profile
in FreeBSD and Ubuntu. - 19. In Haskell, if check whether a file is modified or not, use modificationTime
- 20. Execute remote shell command
- 21. Shell single and double quotes Single and Double Quotes
- 22. Number of argument is wrong in shell script.
1 Shell Script Examples
Shell script is powerful if you could remember all the weird syntax. Here is the example to show how to use Array in Shell script. Note: readarray only works in Bash 4.x, check your version: bash –version
Array in Shell script 1
string='Paris, France, Europe'; echo $string readarray -td, array <<<"$string"; declare -p array; ## declare -a array=([0]="Paris" [1]=" France" [2]=$' Europe\n') country='Paris/Frence/Europe/Sweden/Croatia'; echo $country readarray -td/ array <<<"$country"; declare -p array; for item in "${array[@]}" do echo "$item" done
count number of argument
echo "$#" var="string"
array in bash
arr=( "dog" "cat" "rat" ) for var in ${arr[*]} do echo "var=$var" done
if statement, one liner
var="$#" && if [ "$var" -eq 2 ]; then echo "two arguments"; else echo "not two argument"; fi
Iterate all files with one line shell command
for name in $(ls *); do echo $name; done
String matches
for name in $(ls *); do if [[ $name == *"txt"* ]]; then echo $name; else echo ""; fi
Use regex match =~
if [[ $name =~ "txt" ]]; then echo $name; else echo ""; fi
Repeat shell command with watch If you want to get constantly update information or stream data shell commands then you can use watch repeats your commands For Exmple, I want to monitor my system disk usage
watch df -lah # default 2 seconds interval
watch shell commands with pipe, e.g.
ps aux | grep --color httpd
with double quoteswatch "ps aux | grep --color httpd"
Check a variable is empty or not in shell script, check empty string
if test -z "$1" then echo "\$1=$1 is empty" else echo "\$1=$1 is not empty" fi # or if [ -z "$1" ] then echo "\$1=$1 is empty" else echo "\$1=$1 is not empty" fi
2 grep Examples
grep content of file with specified file name extension, show full file name
grep --include="*.java" -Hnris pattern $PWD
cat and grep
cat /tmp/file.txt | grep string
grep with color
cat /tmp/file.txt | grep --color string
grep print NUM line of trailing context file after matching lines, two line after
cat file | grep -A 'dog'
grep print NUM lines of leading context file before matching lines, two line before
cat file | grep -B 'dog'
grep inversion
# find all non java file grep -v -E '.java$'
grep string that contains "txt", grep matches string
if echo $name | grep -q "txt"; then echo $name; else echo ""; fi
grep string its suffix is "txt", e.g. "file.txt"
if echo $name | grep -q "txt$" then echo $name; else echo ""; fi
grep string its prefix is "file", e.g. "file.txt"
# grep -q => no output if echo $name | grep -q "^file" then echo $name; echo echo ""; fi
grep in POSIX basic and extended regular expression
# basic regex ps aux | grep --color 'wai\|filewatcher' # extended regex ps aux | grep --color -E 'wai|filewatcher'
kill all mysqld in MacOS, combine grep and awk
ps aux | grep --color mysqld | awk '{print $2}' | line 'x -> kill -9 x'
grep colo escape sequence color
grep -E --color=always -Hnris 'begin|end' *.tex | more -R
awk get the next line after match
whois bbc.co.uk | awk -F: '/Registrar:/ && $0 != "" {getline; print $0}' # => British Broadcasting Corporation [Tag = BBC]
awk as filter, match line contains root or www
cat /etc/passwd | awk '/root|www/{print}'
3 sed Example
// list file from line 3 to 9 sed -n 3,9p file.txt
4 sed does not work on MacOS, sort of
- sed on MacOS is BSD version, it does not implement regular expression extension: GNU regex Extension
Character class in GNU regex Extension.
\w [a-bA-Z_] word echo "dog cat" | sed 's/\w/X/g' => "XXX XXX" \W non whitespace echo "dog cat" | sed 's/\W/X/g' => "dogXcat" \s whitespace echo "dog cat" | sed 's/\s/X/g' => "dogXcat" § non whitespace \b match a word boundary echo "dog cat" | sed 's/\b/X/g' => "XdogX XcatX" \< match the beginning of a word \> match the end of a word \d
is NOT in the GNU regex expression, but it is in POSIX regex expression Extension- gsed can be installed on your MacOS which is GNU sed.
# does not work on MacOS or FreeBSD echo "dog cat " | sed -E 's/\s*//g' => "dog cat " echo "dog cat " | gsed 's/\s*//g' => "dogcat" # it seems gsed does not need flag -E
5 Vim regex extension
- Add more here
6 POSIX basic regular expression.
- What is that?
7 POSIX extended regular expression.
- POSIX Extend Regular Extend
- What is that?
Character class | ASCII | example | desc |
---|---|---|---|
[ [:alpha:]] |
[a-zA-Z] | [a-zA-Z] | |
[ [:digit:]] |
[0-9] | [0-9] | |
[ [:space:]] |
[ \n\r\t\v\f] | whitespace | |
[ [:alnum:]] |
[0-9a-zA-Z] | ||
[ [:print:]] |
Visiable char and space char | ||
[ [:lower:]] |
[a-z] | ||
[ [:upper:]] |
[A-Z] | ||
[ [:graph:]] |
Visiable char | ||
[ [:xdigit:]] |
hexadecimal | ||
[ [:blank:]] |
[ \t] | Space and tab | |
[ [:cntrl:]] |
contrl char |
8 rsync examples
rysnc file with specified extensions such png, svg etct.
- include and exclude files.
# rsync svg, png and jpg files excluding other files. rsync -rvzu --include='*'{svg,png,jpf} --exclude "*" source dest # rsync html file excluding other files. rsync -rvzu --include='*.html' --exclude "*" source dest # rsync folder rsync -artv folder source dest # rsync local folder with remote folder rsync -artv local_folder user@xfido.com:/var/www/html
9 If and Else in Shell Script
String comparision | Description |
---|---|
Str1 = Str2 | Return true if strings are equal. |
Str1 != Str2 | Return true if string are NOT equal |
-n Str | Return true if string is NOT null |
-z Str | Return true if string IS null |
Numeric Comparision | Description |
expr1 -eq expr2 | Return true if expressions are equal |
expr1 -ne expr2 | Return true if expressions are NOT equal |
expr1 -gt expr2 | Return true if expr1 is greater than expr2 |
expr1 -lt expr2 | Return true if expr1 is less than expr2 |
expr1 -ge expr2 | Return true if expr2 is greater than or equal to expr2 |
expr1 -le expr2 | Return true if expr2 is less than or equal to expr2 |
File Conditionals | Description |
-d file | Return true if file is a directory |
-e file | Return true if file exists |
-f file | Return true if the provided string is a file. |
-g file | Return true if the group id is set on a file. |
-r file | Return true if file is readable. |
-s file | Return true if file is a non-zero size. |
-u | Return true if the user id is set on a file. |
-w | Return true if the file is writable. |
-x | Return true if the file is executable. |
10 Execute remote shell script Execute remote shell
wget -O - https://bitbucket.org/zsurface/publicfile/raw/46cc8426ac08e2d575b60312fdc01f6ab6e3656c/test1.sh | bash
11 Check whether a command is in bash
command -v curl if [ $? ]; then echo "curl exists"; else echo "curl does not exist"; fi
12 FreeBSD install ports without prompt make install clean BATCH=yes
- ports installation is not working.
cd /usr/ports/redis && make install clean BATCH=yes
13 Install Redis On FreeBSD
pkg install -y redis
14 Enable Redis on FreeBSD
# add redis_enable="YES" to /etc/rc.conf
15 Run Redis on FreeBSD
/usr/local/etc/rc.d/redis start
16 How to avoid all the ssh-agent and ssh-add BS.
Add follwoing two lines to your ~/.profile, it works in FreeBSD.
eval $(ssh-agent -s) ssh-add ~/.ssh/myprivate_id_rsa
17 Here are all the public keys:
- MacOS MacOS Public Key
- FreeBSD FreeBSD Public Key
18 Add global shared profile under /etc/profile
in FreeBSD and Ubuntu.
19 In Haskell, if check whether a file is modified or not, use modificationTime
- If the path is directory, the time will be changed if file is added or deleted.
- See
filewatcher.hs
orwatchDir
inAronModule.hs
modificationTime :: FileStatus -> EpochTime
20 Execute remote shell command
ssh user@xfido.com 'cd user; ls;'
21 Shell single and double quotes Single and Double Quotes
22 Number of argument is wrong in shell script.
$scr/try_argument_count.sh 1 2 # => "$#" => 2 # inside Vim script :term java_compile.sh c % # => "$#" => 3