Unix Tools
==========


The Find Tool
-------------

An interesting tool to pattern match files and their content, and execute actions for each


.. code-block:: bash

    find /etc/ -maxdepth 2 -name nginx 

    ls -la $(find /etc/ -maxdepth 2 -name nginx)

    # with xargs
    find /etc/ -maxdepth 2 -name nginx | xargs ls -la

    # find files and remove them with with xargs and rm
    find /tmp -name core -type f -print0 | xargs -0 /bin/rm -f

    find /etc/ -maxdepth 2 -name nginx | xargs ls -la


The grep tool
-------------

Check existence of text / regex in file, and take action upon the result

.. code-block:: bash

    target_line='/dev/disk/by-id/scsi-0Linode_Volume_zvol /nfs/zvol ext4 defaults,noatime,nofail 0 2'
    target_file='/etc/fstab'

    line_found=$(grep -rnw "$target_file" -e "$target_line")

    if [[ ! "$line_found" ]] ; then
        echo "appending the line";
        echo "$target_line" | sudo tee -a "$target_file" ;
    else
        echo "line exists" ;
    fi


.. code-block:: bash

    target_line="mk ALL=(ALL) NOPASSWD:ALL"
    target_file='/etc/sudoers'

    line_found=$(grep -rnw "$target_file" -e "$target_line")

    if [[ ! "$line_found" ]] ; then
        echo "appending the line";
        echo "$target_line" | sudo tee -a "$target_file" ;
    else
        echo "line exists" ;
    fi


Text Manipulation
-----------------


The sed tool
************

https://www.gnu.org/software/sed/manual/sed.html

sed - stream editor for filtering and transforming text.

String - Regex Substitution
^^^^^^^^^^^^^^^^^^^^^^^^^^^

``sed -i 's/original/new/g' file.txt``

* **-i** inplace
* **s** Substitute
* **original** regex
* **new** text
* **g** global means all occurrences


String replacement examples

.. code-block:: bash

    # replace white space with newlines
    hostname -I | sed 's/ /\n/g'

    # replace colon with a space
    echo $PATH | sed 's/:/ /g'

    # replace colo with a newline
    echo $PATH | sed 's/:/\n/g'

    # replace a pattern within a given lines range (address selection)
    seq 10 | sed '3,6s/^.*/number/g'

    # negation of address selection
    seq 10 | sed '3,6!s/^.*/number/g'

    # allow currently installed packages to accept pip upgrades
    pip freeze | sed 's/==/>=/g' > requirements.txt
    pip install -r requirements.txt --upgrade


Text Filtering - Selection
^^^^^^^^^^^^^^^^^^^^^^^^^^

Text selection can be done with this pattern

``sed -n '/regex/p'``

* option ``-n`` removes all stream lines by default
* the ``p`` command is to print specific lines that match the regex

.. code-block:: bash

    # all user profiles who's default shell is bash
    sed -n '/bash$/p' /etc/passwd

    # Match everything that starts with a b, third letter is a d
    printf "%s\n" abode bad bed bit bid byte body | sed -n '/^b.d/p'

    # Match all 3 letter words that starts with a b, third letter is a d
    printf "%s\n" abode bad bed bit bid byte body | sed -n '/^b.d$/p'

    printf "%s\n" abode bad bed bit bid byte body | sed '/^b.d$/d'


Text filtering can be done with this command

sed '/regex/d'

.. code-block:: bash

    # print users whose default shell is not bash
    sed '/bash$/d' /etc/passwd


the tr tool
***********

Translate or delete a single character from a text stream

This tool is limited to replacing a single character, which is very limited in comparison to ``sed``

.. code-block:: bash

    # replace spaces with a newline
    hostname -I | tr ' ' '\n'

    # replace spaces with a comma
    hostname -I | tr ' ' ','


the awk Tool
************

https://www.gnu.org/software/gawk/manual/gawk.html

.. code-block:: bash

    # text stream passthrough
    cat /etc/passwd |  awk '{print}'

    # or printing the zeroth field of each line, which is all the fields
    cat /etc/passwd |  awk '{print $0}'

    # specify a field separator (FS) and print the first field of each line
    cat /etc/passwd |  awk 'BEGIN {FS = ":"} {print $1}'

    # find usernames that use the bash shell with a regex /bash/, and print the username with the user id
    cat /etc/passwd |  awk 'BEGIN {FS = ":"} /bash/ {print $1,$3}'

    # specify the input and output output field separator
    cat /etc/passwd |  awk 'BEGIN {FS = ":" ; OFS = ":" } /bash/ {print $1,$3}'

the cut Tool
************

Select specific fields in a stream

.. code-block:: bash

    # select the first and fifth field in the passwd file
    cut -d: -f1,5 /etc/passwd