CIS120Linux Fundementals

Strings and Numbers

Computer programs are all about working with data. In past chapters, we have focused on processing data at the file level. However, many programming problems need to be solved using smaller units of data such as strings and numbers. In this chapter, we will look at several shell features used to manipulate strings and numbers. The shell provides a variety of parameter expansions that perform string operations.

Parameter Expansion

Though parameter expansion came up earlier in this course, we did not cover it in detail because most parameter expansions are used in scripts rather than on the command line. We have already worked with some forms of parameter expansion, such as shell variables. The shell provides many more. It's always good practice to enclose parameter expansions in double quotes to prevent unwanted word splitting, unless there is a specific reason not to. This is especially true when dealing with filenames, as they can often include embedded spaces and other assorted nastiness.

Basic Parameters

The simplest form of parameter expansion is reflected in the ordinary use of variables. For example, $a becomes whatever the variable a contains when expanded. Simple parameters may also be surrounded by braces, as in ${a}. This has no effect on the expansion but is required if the variable is adjacent to other text, which may confuse the shell. For example, attempting to create a filename by appending the string _file to the contents of the variable a will result in nothing because the shell will try to expand a variable named a_file rather than a. This problem can be solved by adding braces around the “real” variable name, as in ${a}_file.

Example:

a="foo"
echo "$a_file"      # Outputs nothing because it looks for a variable named a_file
echo "${a}_file"    # Outputs foo_file

Expansions to Manage Empty Variables

Several parameter expansions are intended to deal with nonexistent and empty variables. These expansions are handy for handling missing positional parameters and assigning default values to parameters.

• ${parameter:-word}: Results in the value of word if parameter is unset or empty. If parameter is not empty, the expansion results in the value of parameter.

Example:

foo=
echo ${foo:-"substitute value if unset"}  # Outputs "substitute value if unset"
foo=bar
echo ${foo:-"substitute value if unset"}  # Outputs "bar"

• ${parameter:=word}: Results in the value of word if parameter is unset or empty and also assigns word to parameter. If parameter is not empty, the expansion results in the value of parameter.

Example:

foo=
echo ${foo:="default value if unset"}  # Outputs "default value if unset" and assigns it to foo
echo $foo                             # Outputs "default value if unset"
foo=bar
echo ${foo:="default value if unset"}  # Outputs "bar"

• ${parameter:?word}: Causes the script to exit with an error if parameter is unset or empty, sending the contents of word to standard error.

Example:

foo=
echo ${foo:?"parameter is empty"}  # Outputs an error message and exits
foo=bar
echo ${foo:?"parameter is empty"}  # Outputs "bar"

• ${parameter:+word}: Results in nothing if parameter is unset or empty, but substitutes the value of word if parameter is not empty.

Example:

foo=
echo ${foo:+"substitute value if set"}  # Outputs nothing
foo=bar
echo ${foo:+"substitute value if set"}  # Outputs "substitute value if set"

Expansions That Return Variable Names

The shell can return the names of variables in some situations. For example, ${!prefix*} and ${!prefix@} return the names of existing variables with names beginning with prefix. Both forms perform identically.

Example:

echo ${!BASH*}  # Outputs names of all variables starting with "BASH"

String Operations

There is a large set of expansions that can be used to operate on strings. Many of these expansions are particularly well suited for operations on pathnames.

• ${#parameter}: Expands into the length of the string contained by parameter.

Example:

foo="This string is long."
echo "'$foo' is ${#foo} characters long."  # Outputs "'This string is long.' is 20 characters long."

• ${parameter:offset} and ${parameter:offset:length}: Used to extract portions of the string contained in parameter, starting at offset characters from the beginning and continuing to the end of the string, unless length is specified.

Example:

foo="This string is long."
echo ${foo:5}        # Outputs "string is long."
echo ${foo:5:6}      # Outputs "string"

• ${parameter#pattern} and ${parameter##pattern}: Remove a leading portion of the string contained in parameter defined by pattern, with the former removing the shortest match and the latter removing the longest match.

Example:

foo=file.txt.zip
echo ${foo#*.}       # Outputs "txt.zip"
echo ${foo##*.}      # Outputs "zip"

• ${parameter%pattern} and ${parameter%%pattern}: Remove text from the end of the string contained in parameter.

Example:

foo=file.txt.zip
echo ${foo%.*}       # Outputs "file.txt"
echo ${foo%%.*}      # Outputs "file"

• ${parameter/pattern/string}: Performs a search-and-replace operation on the contents of parameter. In the normal form, only the first occurrence of pattern is replaced, while the // form replaces all occurrences.

Example:

foo=JPG.JPG
echo ${foo/JPG/jpg}     # Outputs "jpg.JPG"
echo ${foo//JPG/jpg}    # Outputs "jpg.jpg"
echo ${foo/#JPG/jpg}    # Outputs "jpg.JPG"
echo ${foo/%JPG/jpg}    # Outputs "JPG.jpg"

Case Conversion

Bash supports the uppercase/lowercase conversion of strings with four parameter expansions and two declare command options. These conversions are useful for normalizing user input for database lookups, ensuring consistent data entry. For example, ${parameter,,} converts all characters in parameter to lowercase, ${parameter,} changes only the first character to lowercase, ${parameter^^} converts all characters to uppercase, and ${parameter^} changes only the first character to uppercase.

Example:

foo="aBc"
echo "${foo,,}"  # Outputs "abc"
echo "${foo,}"   # Outputs "aBc"
echo "${foo^^}"  # Outputs "ABC"
echo "${foo^}"   # Outputs "ABc"

Arithmetic Evaluation and Expansion

Arithmetic expansion allows performing various arithmetic operations on integers using the basic form $((expression)). The shell supports integer constants in different bases, such as decimal, octal, and hexadecimal. Unary operators, simple arithmetic operators, and more complex assignment operators provide convenient shorthand for arithmetic tasks. Notably, the increment (++) and decrement (--) operators, derived from the C programming language, increase or decrease the value of their parameters by one, with a subtle difference between prefix and postfix usage.

Example:

echo $((2 + 3))         # Outputs "5"
echo $((5 / 2))         # Outputs "2" (integer division)
echo $((5 % 2))         # Outputs "1" (modulo operation)
foo=1
echo $((foo++))         # Outputs "1", then foo becomes 2
echo $foo               # Outputs "2"
echo $((++foo))         # Outputs "3"

Bit Operations

Bit operators manipulate numbers at the bit level, often for setting or reading bit-flags. These include bitwise negation (~), left and right bitwise shifts (<< and >>), and bitwise AND, OR, and XOR (&, |, ^). Corresponding assignment operators, such as <<=, exist for all but bitwise negation.

Example:

echo $((1 << 2))  # Outputs "4" (1 shifted left by 2 bits)
echo $((4 >> 1))  # Outputs "2" (4 shifted right by 1 bit)

Logic and Comparison Operators

The (( )) compound command supports a variety of comparison operators, including less than (<), greater than (>), equal to (==), not equal to (!=), logical AND (&&), and logical OR (||). The ternary operator (expr1?expr2:expr3) performs a stand-alone logical test, acting as a compact if/then/else statement.

Example:

a=0
((a < 1 ? ++a : --a))
echo $a  # Outputs "1"
((a < 1 ? ++a : --a))
echo $a  # Outputs "0"

Practical Example

To demonstrate the efficiency of parameter expansion, we modified the longest-word script from the previous chapter to use ${#j} in place of `$(echo -n $j

| wc -c)`, resulting in a significant performance improvement. The original script took 3.618 seconds to scan the text file, while the new version using parameter expansion took only 0.06 seconds.

#!/bin/bash
# longest-word3: find longest string in a file
for i; do
  if [[ -r "$i" ]]; then
    max_word=
    max_len=0
    for j in $(strings $i); do
      len="${#j}"
      if (( len > max_len )); then
        max_len="$len"
        max_word="$j"
      fi
    done
    echo "$i: '$max_word' ($max_len characters)"
  fi
done

In conclusion, mastering these shell features enhances your ability to manipulate strings and numbers efficiently, improving the performance and readability of your scripts.