CIS120 Linux Fundamentals by Scott Shaper

Strings and Numbers

Imagine you're organizing a massive library of books. You need tools to count them, label them, sort them by title, and even rename some. That's exactly what string and number manipulation does in the shell - it helps you organize and process your digital library!

In this chapter, we'll explore how the bash shell gives you powerful tools to work with text (strings) and values (numbers) - skills that make everyday tasks faster and more efficient. Whether you're renaming files, extracting information, or performing calculations, these techniques are your command-line superpowers.

Quick Reference: String and Number Operations

Operation Type Description Common Use
Parameter Expansion Manipulate variable contents Working with variables, setting defaults
String Operations Extract, measure and modify text Filename manipulation, text processing
Case Conversion Change text between uppercase/lowercase Normalizing user input, formatting output
Arithmetic Expansion Perform math operations Counters, calculations, logic checks
Bit Operations Manipulate individual bits in numbers Setting flags, low-level data manipulation

Parameter Expansion

Parameter expansion is like having a Swiss Army knife for your variables. It lets you retrieve, modify, and test variable values in many useful ways.

When to Use Parameter Expansion

  • When you need to use variable values in your scripts
  • When you want to provide default values for missing parameters
  • When you need to handle potentially empty variables safely
  • When working with filenames that might contain spaces or special characters

Basic Parameters

The simplest form of parameter expansion happens when you use variables with the $ sign:

# Basic variable usage
name="Alice"
echo "Hello, $name"    # Outputs "Hello, Alice"

# Using braces to clarify variable boundaries
fruit="apple"
echo "$fruit_pie"      # Tries to find variable "fruit_pie" (nothing outputs)
echo "${fruit}_pie"    # Outputs "apple_pie"

Always wrap your variables in double quotes ("$variable") to prevent problems with spaces and special characters.

Handling Empty Variables

Syntax What It Does When to Use It
${var:-word} Use word if var is empty/unset Providing fallback values
${var:=word} Use word if var is empty/unset AND set var to word Setting default values
${var:?word} Exit with error if var is empty/unset Ensuring required variables exist
${var:+word} Use word only if var is NOT empty Conditional text substitution
# Provide a default value without changing the variable
username=""
echo "Hello, ${username:-"Guest"}"  # Outputs "Hello, Guest"

# Set a default value and update the variable
config_file=""
echo "Using config: ${config_file:="default.conf"}"  # Sets and outputs "Using config: default.conf"
echo $config_file  # Now contains "default.conf"

# Error if a required variable is missing
function send_email() {
    ${recipient:?"Email recipient not specified"}
    # Rest of function...
}

# Add text only when variable exists
title="Professor"
name="Smith"
echo "Hello, ${title:+"$title "}$name"  # Outputs "Hello, Professor Smith"

String Operations

String operations are like having text editing tools built right into your command line - they help you measure, extract, and modify text with precision.

When to Use String Operations

  • When working with filenames (extracting extensions, directories)
  • When processing user input (validation, formatting)
  • When generating formatted output
  • When you need to modify text without external commands

Common String Operations

Operation What It Does When to Use It
${#var} Get string length Validating input, formatted output
${var:offset}
${var:offset:length}
Extract substring Getting parts of text, truncating
${var#pattern}
${var##pattern}
Remove from beginning (shortest/longest match) Removing file directories, prefixes
${var%pattern}
${var%%pattern}
Remove from end (shortest/longest match) Removing file extensions, suffixes
${var/pattern/replacement}
${var//pattern/replacement}
Replace text (first occurrence/all) Text substitution, formatting
# String length - Validating a password
password="abc123"
if [ ${#password} -lt 8 ]; then
    echo "Password too short - needs at least 8 characters"
fi

# Substring extraction - Getting username from email
email="student@university.edu"
username=${email%%@*}
echo "Username: $username"  # Outputs "Username: student"

# Filename manipulation - real-world examples
full_path="/home/student/documents/essay.final.docx"

# Get just the filename (remove directories)
filename=${full_path##*/}
echo "Filename: $filename"  # Outputs "Filename: essay.final.docx"

# Get just the extension
extension=${filename##*.}
echo "Extension: $extension"  # Outputs "Extension: docx"

# Remove the extension
basename=${filename%.*}
echo "Base name: $basename"  # Outputs "Base name: essay.final"

# Replace text 
text="Hello world"
echo "${text/world/friends}"  # Outputs "Hello friends"

Case Conversion

Case conversion is like having an automatic text formatter that can make your text SHOUT or whisper with a simple command.

When to Use Case Conversion

  • When normalizing user input for consistent database entries
  • When formatting output for display
  • When making case-insensitive comparisons
  • When enforcing formatting standards
Syntax What It Does Example
${var,,} Convert all to lowercase "ABCdef" → "abcdef"
${var,} Convert first character to lowercase "ABCdef" → "aBCdef"
${var^^} Convert all to uppercase "ABCdef" → "ABCDEF"
${var^} Convert first character to uppercase "aBCdef" → "ABCdef"
# Normalize user input for consistent data entry
read -p "Enter your state code: " state_code
state_code=${state_code^^}  # Convert to uppercase
echo "Normalized state code: $state_code"

# Proper capitalization for names
name="john smith"
first=${name%% *}  # Get first name
last=${name##* }   # Get last name
# Capitalize first letter of each name
proper_name="${first^} ${last^}" 
echo "Formatted name: $proper_name"  # Outputs "Formatted name: John Smith"

Arithmetic Operations

Think of arithmetic expansion as having a calculator built right into your command line. No need to open a separate app - just do math directly in your scripts!

When to Use Arithmetic Operations

  • When calculating values in scripts
  • When using counters in loops
  • When performing comparisons
  • When working with file sizes, dates, or other numeric data

Common Arithmetic Operations

Operation Syntax Example
Basic math $((expression)) echo $((2 + 3)) → 5
Increment $((var++)) or $((++var)) Adds 1 to variable
Decrement $((var--)) or $((--var)) Subtracts 1 from variable
Assignment $((var = expression)) Sets variable to result
Compound assignment $((var += expression)) Shorthand for var = var + expression
# Simple calculator
read -p "Enter first number: " num1
read -p "Enter second number: " num2
sum=$((num1 + num2))
echo "Sum: $sum"

# Loop with counter
echo "Countdown:"
for ((i=10; i>=0; i--)); do
    echo "$i..."
    # In real script, you might add: sleep 1
done
echo "Blast off!"

# Converting units - KB to MB
size_kb=2584
size_mb=$((size_kb / 1024))
echo "Size: $size_mb MB"

# Using shortcuts for counters
count=0
echo "Initial count: $count"
echo "Post-increment: $((count++))"  # Uses current value, then increments
echo "New count: $count"             # Now shows 1
echo "Pre-increment: $((++count))"   # Increments first, then uses value
echo "Final count: $count"           # Now shows 2

Bit Operations

Bit operations are like having microscopic switches you can flip on and off within a number. They're especially useful in system programming and when working with flags or binary data.

When to Use Bit Operations

  • When working with binary flags
  • When performing low-level system operations
  • When optimizing for speed or space
Operation Symbol What It Does
Bitwise AND & Sets bits that are on in both numbers
Bitwise OR | Sets bits that are on in either number
Bitwise XOR ^ Sets bits that are on in one number but not both
Left Shift << Shifts bits left (multiplies by 2^n)
Right Shift >> Shifts bits right (divides by 2^n)
# Using bit shifts for quick multiplication/division by powers of 2
echo $((1 << 3))  # 1 × 2³ = 8
echo $((16 >> 2))  # 16 ÷ 2² = 4

# Setting and checking flags
# Example: Permissions flags (read=4, write=2, execute=1)
permissions=0  # No permissions
permissions=$((permissions | 4))  # Add read permission
permissions=$((permissions | 1))  # Add execute permission
echo "Permissions value: $permissions"  # Shows 5

# Check if read permission is set (using bitwise AND)
if (( (permissions & 4) != 0 )); then
    echo "Read permission is set"
fi

Logic and Comparison Operations

Logic operations let you make decisions based on conditions, like a digital fork in the road for your script.

# Age verification with ternary operator
age=20
status=$(( age >= 21 ? "allowed" : "underage" ))
echo "Drinking status: $status"

# Checking multiple conditions
temperature=72
humidity=65
comfort_level=$(( (temperature > 65 && temperature < 75) && 
                   (humidity > 40 && humidity < 70) ? 
                   "comfortable" : "uncomfortable" ))
echo "Current conditions are $comfort_level"

# Using (( )) for conditions in if statements
count=5
if (( count > 0 && count < 10 )); then
    echo "Count is between 1 and 9"
fi

Tips for Success

  • Always quote your variables ("$var") to prevent word splitting issues with spaces
  • Use parameter expansion instead of external commands like cut or sed when possible - it's much faster
  • Use declare -i for variables you only use for integers to help catch errors
  • Test your string operations on edge cases (empty strings, special characters)
  • Remember that bash arithmetic only works with integers (no floating point)

Common Mistakes to Avoid

  • Forgetting to quote variables, especially when they might contain spaces
  • Trying to use floating-point math in bash arithmetic (use bc command instead)
  • Forgetting that ${var#pattern} and ${var##pattern} match from the beginning
  • Forgetting that ${var%pattern} and ${var%%pattern} match from the end
  • Using backticks (`` `command` ``) instead of $(command) for command substitution

Best Practices

  • Use parameter expansion over external commands when possible for better performance
  • Handle empty variables safely using default values (${var:-default})
  • Use meaningful variable names that describe their purpose
  • Comment your complex parameter expansions to explain what they do
  • Test for edge cases (empty strings, special characters, numeric limits)

Example: Performance Improvement

Look at how parameter expansion can dramatically improve performance in real scripts:

# Original slow version - took 3.618 seconds
# (using external commands)
for word in $(strings file.txt); do
  len=$(echo -n $word | wc -c)
  # ... rest of script
done

# Improved fast version - took only 0.06 seconds
# (using parameter expansion)
for word in $(strings file.txt); do
  len=${#word}
  # ... rest of script
done

That's a 60x performance improvement by using parameter expansion!

Example: Finding the Longest Word

#!/bin/bash
# longest-word: find longest string in a file

# Explain what we're doing
echo "Finding the longest word in your files..."

for filename; do
  if [[ -r "$filename" ]]; then
    max_word=
    max_len=0
    
    # Process each word in the file
    for word in $(strings "$filename"); do
      # Use parameter expansion to get length (much faster)
      len="${#word}"
      
      # Check if this word is longer than current max
      if (( len > max_len )); then
        max_len="$len"
        max_word="$word"
      fi
    done
    
    echo "$filename: '$max_word' ($max_len characters)"
  else
    echo "Error: Cannot read '$filename'" >&2
  fi
done

Example: Counting Letters in a String

This example shows how to count occurrences of a specific letter in a string using a while loop and parameter expansion.

#!/bin/bash
# count-letters: counts occurrences of a specific letter in a string

# Define a string with both uppercase and lowercase 'A'
myString="Alice was Amazed At the Apple and Ate it All. Amazing!"
echo "Original string: $myString"

# Convert to lowercase using parameter expansion
myString=${myString,,}  # Convert to lowercase
echo "Lowercase string: $myString"

# Initialize counter and index
count=0
i=0

# Loop through each character of the string
while (( i++ < ${#myString} )); do
    # Extract the current character
    char=$(expr substr "$myString" $i 1)
    
    # Check if character is 'a' and increment counter
    if [[ "$char" == "a" ]]; then
        ((count++))
    fi
done

# Display the result
echo "The letter 'a' appears $count times in the string."

This example demonstrates:

  • Using case conversion with ${string,,} to normalize input
  • Getting string length with ${#string}
  • Using a while loop with counters
  • Extracting individual characters from a string

Example: Validating Numeric Input

This example shows how to validate that user input contains only numbers, which is useful when creating scripts that perform calculations.

#!/bin/bash
# validate-numbers: ensures input values are numeric

# Function to check if input is a number
check_num() {
    # Check if parameter contains at least one digit
    num=`echo $1 | grep '[0-9]'`
    # Check if parameter contains any non-digit characters
    non_digit=`echo $1 | grep '[^0-9]'`
    
    if [ "$num" = '' ] || [ "$non_digit" != '' ]; then
        echo "Error: '$1' is not a valid number"
        return 1
    fi
    return 0
}

# Get two numbers from user
read -p "Enter first number: " num1
read -p "Enter second number: " num2

# Validate both inputs
if check_num "$num1" && check_num "$num2"; then
    # If both are valid, perform a calculation
    sum=$((num1 + num2))
    echo "The sum of $num1 and $num2 is $sum"
else
    echo "Please try again with valid whole numbers"
    exit 1
fi

This example demonstrates:

  • Creating and using shell functions
  • Using grep to validate input patterns
  • Checking for both the presence of digits and absence of non-digits
  • Using return values to communicate success/failure
  • Performing conditional arithmetic operations

With these examples and the techniques covered in this chapter, you now have powerful tools to manipulate strings and numbers in your shell scripts!