Bash Functions with Flags

How to write useful bash functions with short flags and long flags

When it comes to creating reusable scripts for pipeline workflows, flexibility and adaptability are key. By structuring your scripts as functions with configurable flags, you can significantly enhance their reusability. These reusable scripts can be seamlessly integrated into your pipeline containers or sourced from a dedicated scripts folder in your repository. In this article, we will explore the benefits of writing reusable scripts and how to leverage configurable flags to create flexible and maintainable pipeline workflows.

Developing reusable scripts can be challenging, particularly when it comes to creating scripts that are compatible with pipeline workflows. However, by structuring your scripts as functions with configurable flags, you can significantly enhance their reusability. These reusable scripts can be easily integrated into pipeline containers or sourced from a dedicated scripts folder within your repository.

When scripts are designed as functions with configurable flags, they offer a level of flexibility that enables them to be reused across different scenarios and pipelines. This reusability simplifies the maintenance process and promotes consistency in your pipeline workflows. Additionally, it enables you to leverage the power of configuration, allowing you to tailor the behavior of your scripts to specific use cases.

Consider the following example, where reusable scripts are utilized within a pipeline configuration:

├── scripts/
│   ├── utils.sh
│   ├── script1.sh
│   └── script2.sh
└── .gitlab-ci.yml

pipeline_job:
    before_script:
      - source ./scripts/utils.sh
      - util_login_vault --url "$VAULT_URL" --role "$VAULT_ROLE"
    script:
      - util_deploy_aws --flag-a "option-a"

In this example, the util_login_vault and util_deploy_aws functions are sourced from a scripts folder and incorporated into the pipeline configuration. The flags passed to these functions allow for customization and adaptability, ensuring that the scripts align with the specific requirements of the pipeline.

Lets Create a demo bash function!!

Initialize File

Lets create a .sh file for our bash script.

touch do_something.sh

Every file needs a shebang line. It tells the shell which interpreter to use, in this case its Bash.

#!/usr/bin/env bash

We will also need to name our function and create a function code block

#!/usr/bin/env bash

function do_something(){

}

do_something

To make it easy to test i will invoke the function at the end so i can just run the script. You can comment it out to leave in as example command or delete.

Set up Special Environment Vars

First we need to determine what flags we will be using and have the option to set defaults and ENV vars to read from first

For example you can add a special prefix_ to the env vars so you can run this function without flags. Flags will override the env variables

#!/usr/bin/env bash

function do_something(){
  local OPTION_A="${PREFIX_OPTION_A-default_value}"
  local OPTION_B="${PREFIX_OPTION_B-nil}"
  local OPTION_C="${PREFIX_OPTION_C-1}"
  local OPTION_D="${PREFIX_OPTION_D-false}"
  local OPTION_E="${PREFIX_OPTION_E}"


}

do_something

Option	Default Value	Environment Variable
OPTIONA	default_value	PREFIX_FLAG1
OPTION2	nil	PREFIX_FLAG2
OPTION3	1	PREFIX_FLAG3
OPTION4	false	PREFIX_FLAG4
OPTION5		PREFIX_FLAG5

Add Flags

Now we are going to setup our flags using getopt and have them set the variables we made at the beginning of our function

#!/usr/bin/env bash

function do_something(){
  local OPTION_A="${PREFIX_OPTION_A-default_value}"
  local OPTION_B="${PREFIX_OPTION_B-nil}"
  local OPTION_C="${PREFIX_OPTION_C-1}"
  local OPTION_D="${PREFIX_OPTION_D-false}"
  local OPTION_E="${PREFIX_OPTION_E}"

  # ignore `\` its just to prevent hugo from replacing with an emoji
  # Parse command-line options using getopt
  TEMP=$(getopt -o 'a:\b:c:e:d' --long 'option-a:,option-b:,option-c:,option-e:,option-d' -- "$@")
  eval set -- "$TEMP"
  # Process each option
  while true; do
    case $1 in
      -a|--option-a)
        OPTION_A=$2
        shift 2
        ;;
      -b|--option-b)
        OPTION_B=$2
        shift 2
        ;;
      -c|--option-c)
        OPTION_C=$2
        shift 2
        ;;
      -d|--option-d)
        OPTION_D="true"
        shift 
        ;;
      -e|--option-e)
        OPTION_E=$2
        shift 2
        ;;
      --)
        shift
        break
        ;;
    esac
  done

  # Print the values of the flags
echo "OPTION_A: $OPTION_A"
echo "OPTION_B: $OPTION_B"
echo "OPTION_C: $OPTION_C"
echo "OPTION_D: $OPTION_D"
echo "OPTION_E: $OPTION_E"

}

do_something \
    --option-d \
    --option-a value1 \
    --option-b value2 \
    --option-c "three hundred" \
    --option-e "dd"

Option	Default Value	Environment Variable	Flag Option
OPTIONA	default_value	PREFIX_FLAG1	-a, --option-a
OPTION2	nil	PREFIX_FLAG2	-b, --option-b
OPTION3	1	PREFIX_FLAG3	-c, --option-c
OPTION4	false	PREFIX_FLAG4	-d, --option-d
OPTION5		PREFIX_FLAG5	-e, --option-e

Understanding the getopt Command: Parsing Command-Line Options in Shell Scripts

When developing shell scripts, it is essential to handle command-line options and arguments efficiently. One powerful tool that simplifies this task is the getopt command. By utilizing getopt, you can define a set of options and flags that users can pass to your script during execution, enhancing its flexibility and usability.

How does getopt work?

The getopt command parses command-line options and arguments, making it easier to extract and process them within your shell script. Here's a step-by-step breakdown of how it works:

1. Invocation and Arguments: The getopt command is invoked with the option specifications and the command-line arguments ("$@").

2. Parsing Options: getopt examines the provided option specifications and the command-line arguments to identify and extract the specified options and their corresponding values.

3. Storage of Parsed Options: The output of getopt is stored in a designated variable, commonly named TEMP. This variable contains the parsed options and their values, ready for further processing within your script.

4. Setting Positional Parameters: The eval set -- "$TEMP" command assigns the parsed options and values to the positional parameters ($1, $2, etc.), making them easily accessible for subsequent operations.

5. Option Processing: Your script utilizes a while loop to iterate over the positional parameters and handle each option individually.

6. Case Statement for Option Handling: Within the loop, a case statement is employed to identify and process each option. Both the short form (e.g., -t, -a) and the long form (e.g., --table-name, --table-arn) can be utilized to specify options.

7. Assigning Values to Variables: For every option encountered, the corresponding variable is set to the provided value using the $2 parameter. The shift 2 command then shifts the positional parameters by two positions, preparing the loop for the next option.

8. Loop Continuation: The loop continues until all options have been processed, ensuring that your script handles each specified option appropriately.

9. Non-Option Arguments: The special -- option acts as a delimiter, indicating the end of the options. Any arguments following -- are treated as non-option arguments and can be accessed using the $1, $2, etc. parameters.

By employing the getopt command, your shell script gains the ability to accept a wide range of command-line options and their corresponding values. Additionally, default values can be set within the script for options that are not explicitly provided by users, enhancing the script's versatility and usability.

Handling Boolean Flags

When using the getopt command to parse command-line options, you may encounter scenarios where you have boolean flags that don't require a following value. These flags are commonly used to enable or disable certain features or behaviors in your script.

To handle boolean flags in getopt, you can define them without a colon (:) after the short option or without an equal sign (=) after the long option. This indicates that the flag is a standalone boolean option.

For example, let's consider a script with a boolean flag -d and its corresponding long option --option-d. Here's how you can define them in getopt:

-o 'a:\b:c:e:d' --long 'option-a:,option-b:,option-c:,option-e:,option-d'

In this case, the -d short option and --option-d long option are defined without a colon or an equal sign. This configuration signifies that they are boolean flags and do not require a following value.

When running the script, you can use the -d flag or --option-d flag to set the associated variable to true. If you omit the flag, the variable will retain its default value, typically false.

To process the options correctly, the shift operation is used after each boolean flag is encountered. This operation shifts the positional parameters by one position, discarding the option itself, since no value is expected. However, if an option requires an argument, such as -a valueA, shift 2 is used instead to discard both the option and its corresponding value.

Here's an example code snippet illustrating the usage:

#!/usr/bin/env bash
OPTION_A=${OPTION_A=string_abcdefg}
OPTION_D=false

TEMP=$(getopt -o 'a:\b:c:e:d' --long 'option-a:,option-b:,option-c:,option-e:,option-d' -- "$@")
eval set -- "$TEMP"

while true; do
  case $1 in
    -a|--option-a)
      OPTION_A=$2
      shift 2
      ;;
    -d|--option-d)
      OPTION_D="true"
      shift
      ;;
    --)
      shift
      break
      ;;
  esac
done

echo "OPTION_A: $OPTION_A"
echo "OPTION_D: $OPTION_D"

In this example, -a valueA sets the value for option-a using shift 2 to discard both the option and its value. -d sets the boolean flag for option-d to true using shift to discard the flag itself.

By understanding how to handle boolean flags in getopt and utilizing the appropriate shift or shift 2 operation, you can effectively enable or disable specific features in your scripts based on user input.

Full Script

#!/usr/bin/env bash
function do_something() {
  local OPTIONA="${PREFIX_FLAG1-default_value}"
  local OPTION2="${PREFIX_FLAG2-nil}"
  local OPTION3="${PREFIX_FLAG3-1}"
  local OPTION4="${PREFIX_FLAG4-false}"
  local OPTION5="${PREFIX_FLAG5}"

  # ignore `\` its just to prevent hugo from replacing with an emoji
  # Parse command-line options using getopt
  TEMP=$(getopt -o 'a:\b:c:e:d' --long 'option-a:,option-b:,option-c:,option-e:,option-d' -- "$@")
  eval set -- "$TEMP"
  # Process each option
  while true; do
    case $1 in
      -a|--option-a)
        OPTIONA=$2
        shift 2
        ;;
      -b|--option-b)
        OPTION2=$2
        shift 2
        ;;
      -c|--option-c)
        OPTION3=$2
        shift 2
        ;;
      -d|--option-d)
        OPTION4="true"
        shift 
        ;;
      -e|--option-e)
        OPTION5=$2
        shift 2
        ;;
      --)
        shift
        break
        ;;
    esac
  done
  # Print the values of the flags
  echo "OPTION-A: $OPTIONA"
  echo "OPTION2: $OPTION2"
  echo "OPTION3: $OPTION3"
  echo "OPTION4: $OPTION4"
  echo "OPTION5: $OPTION5"
}
###
# to run in terminal
###
# source ./flag_starter.sh or /path/to/your/script.sh
# Call the function with flags
do_something \
    --option-a value1 \
    --option-b value2 \
    -b value3 \
    -c '333' \
    --option-c "three hundred" \
    -d "true" \
    --option-e "dd"