
Strings and Numbers
Imagine you're organizing a massive library of books. You need tools to count them, label them, sort them by title, and even rename some. That's exactly what string and number manipulation does in the shell - it helps you organize and process your digital library!
In this chapter, we'll explore how the bash shell gives you powerful tools to work with text (strings) and values (numbers) - skills that make everyday tasks faster and more efficient. Whether you're renaming files, extracting information, or performing calculations, these techniques are your command-line superpowers.
Quick Reference: String and Number Operations
Operation Type | Description | Common Use |
---|---|---|
Parameter Expansion | Manipulate variable contents | Working with variables, setting defaults |
String Operations | Extract, measure and modify text | Filename manipulation, text processing |
Case Conversion | Change text between uppercase/lowercase | Normalizing user input, formatting output |
Arithmetic Expansion | Perform math operations | Counters, calculations, logic checks |
Bit Operations | Manipulate individual bits in numbers | Setting flags, low-level data manipulation |
Parameter Expansion
Parameter expansion is like having a Swiss Army knife for your variables. It lets you retrieve, modify, and test variable values in many useful ways.
When to Use Parameter Expansion
- When you need to use variable values in your scripts
- When you want to provide default values for missing parameters
- When you need to handle potentially empty variables safely
- When working with filenames that might contain spaces or special characters
Basic Parameters
The simplest form of parameter expansion happens when you use variables with the $
sign:
# Basic variable usage
name="Alice"
echo "Hello, $name" # Outputs "Hello, Alice"
# Using braces to clarify variable boundaries
fruit="apple"
echo "$fruit_pie" # Tries to find variable "fruit_pie" (nothing outputs)
echo "${fruit}_pie" # Outputs "apple_pie"
Always wrap your variables in double quotes ("$variable"
) to prevent problems with spaces and special characters.
Handling Empty Variables
Syntax | What It Does | When to Use It |
---|---|---|
${var:-word} |
Use word if var is empty/unset |
Providing fallback values |
${var:=word} |
Use word if var is empty/unset AND set var to word |
Setting default values |
${var:?word} |
Exit with error if var is empty/unset |
Ensuring required variables exist |
${var:+word} |
Use word only if var is NOT empty |
Conditional text substitution |
# Provide a default value without changing the variable
username=""
echo "Hello, ${username:-"Guest"}" # Outputs "Hello, Guest"
# Set a default value and update the variable
config_file=""
echo "Using config: ${config_file:="default.conf"}" # Sets and outputs "Using config: default.conf"
echo $config_file # Now contains "default.conf"
# Error if a required variable is missing
function send_email() {
${recipient:?"Email recipient not specified"}
# Rest of function...
}
# Add text only when variable exists
title="Professor"
name="Smith"
echo "Hello, ${title:+"$title "}$name" # Outputs "Hello, Professor Smith"
String Operations
String operations are like having text editing tools built right into your command line - they help you measure, extract, and modify text with precision.
When to Use String Operations
- When working with filenames (extracting extensions, directories)
- When processing user input (validation, formatting)
- When generating formatted output
- When you need to modify text without external commands
Common String Operations
Operation | What It Does | When to Use It |
---|---|---|
${#var} |
Get string length | Validating input, formatted output |
${var:offset} ${var:offset:length} |
Extract substring | Getting parts of text, truncating |
${var#pattern} ${var##pattern} |
Remove from beginning (shortest/longest match) | Removing file directories, prefixes |
${var%pattern} ${var%%pattern} |
Remove from end (shortest/longest match) | Removing file extensions, suffixes |
${var/pattern/replacement} ${var//pattern/replacement} |
Replace text (first occurrence/all) | Text substitution, formatting |
# String length - Validating a password
password="abc123"
if [ ${#password} -lt 8 ]; then
echo "Password too short - needs at least 8 characters"
fi
# Substring extraction - Getting username from email
email="student@university.edu"
username=${email%%@*}
echo "Username: $username" # Outputs "Username: student"
# Filename manipulation - real-world examples
full_path="/home/student/documents/essay.final.docx"
# Get just the filename (remove directories)
filename=${full_path##*/}
echo "Filename: $filename" # Outputs "Filename: essay.final.docx"
# Get just the extension
extension=${filename##*.}
echo "Extension: $extension" # Outputs "Extension: docx"
# Remove the extension
basename=${filename%.*}
echo "Base name: $basename" # Outputs "Base name: essay.final"
# Replace text
text="Hello world"
echo "${text/world/friends}" # Outputs "Hello friends"
Case Conversion
Case conversion is like having an automatic text formatter that can make your text SHOUT or whisper with a simple command.
When to Use Case Conversion
- When normalizing user input for consistent database entries
- When formatting output for display
- When making case-insensitive comparisons
- When enforcing formatting standards
Syntax | What It Does | Example |
---|---|---|
${var,,} |
Convert all to lowercase | "ABCdef" → "abcdef" |
${var,} |
Convert first character to lowercase | "ABCdef" → "aBCdef" |
${var^^} |
Convert all to uppercase | "ABCdef" → "ABCDEF" |
${var^} |
Convert first character to uppercase | "aBCdef" → "ABCdef" |
# Normalize user input for consistent data entry
read -p "Enter your state code: " state_code
state_code=${state_code^^} # Convert to uppercase
echo "Normalized state code: $state_code"
# Proper capitalization for names
name="john smith"
first=${name%% *} # Get first name
last=${name##* } # Get last name
# Capitalize first letter of each name
proper_name="${first^} ${last^}"
echo "Formatted name: $proper_name" # Outputs "Formatted name: John Smith"
Arithmetic Operations
Think of arithmetic expansion as having a calculator built right into your command line. No need to open a separate app - just do math directly in your scripts!
When to Use Arithmetic Operations
- When calculating values in scripts
- When using counters in loops
- When performing comparisons
- When working with file sizes, dates, or other numeric data
Common Arithmetic Operations
Operation | Syntax | Example |
---|---|---|
Basic math | $((expression)) |
echo $((2 + 3)) → 5 |
Increment | $((var++)) or $((++var)) |
Adds 1 to variable |
Decrement | $((var--)) or $((--var)) |
Subtracts 1 from variable |
Assignment | $((var = expression)) |
Sets variable to result |
Compound assignment | $((var += expression)) |
Shorthand for var = var + expression |
# Simple calculator
read -p "Enter first number: " num1
read -p "Enter second number: " num2
sum=$((num1 + num2))
echo "Sum: $sum"
# Loop with counter
echo "Countdown:"
for ((i=10; i>=0; i--)); do
echo "$i..."
# In real script, you might add: sleep 1
done
echo "Blast off!"
# Converting units - KB to MB
size_kb=2584
size_mb=$((size_kb / 1024))
echo "Size: $size_mb MB"
# Using shortcuts for counters
count=0
echo "Initial count: $count"
echo "Post-increment: $((count++))" # Uses current value, then increments
echo "New count: $count" # Now shows 1
echo "Pre-increment: $((++count))" # Increments first, then uses value
echo "Final count: $count" # Now shows 2
Bit Operations
Bit operations are like having microscopic switches you can flip on and off within a number. They're especially useful in system programming and when working with flags or binary data.
When to Use Bit Operations
- When working with binary flags
- When performing low-level system operations
- When optimizing for speed or space
Operation | Symbol | What It Does |
---|---|---|
Bitwise AND | & |
Sets bits that are on in both numbers |
Bitwise OR | | |
Sets bits that are on in either number |
Bitwise XOR | ^ |
Sets bits that are on in one number but not both |
Left Shift | << |
Shifts bits left (multiplies by 2^n) |
Right Shift | >> |
Shifts bits right (divides by 2^n) |
# Using bit shifts for quick multiplication/division by powers of 2
echo $((1 << 3)) # 1 × 2³ = 8
echo $((16 >> 2)) # 16 ÷ 2² = 4
# Setting and checking flags
# Example: Permissions flags (read=4, write=2, execute=1)
permissions=0 # No permissions
permissions=$((permissions | 4)) # Add read permission
permissions=$((permissions | 1)) # Add execute permission
echo "Permissions value: $permissions" # Shows 5
# Check if read permission is set (using bitwise AND)
if (( (permissions & 4) != 0 )); then
echo "Read permission is set"
fi
Logic and Comparison Operations
Logic operations let you make decisions based on conditions, like a digital fork in the road for your script.
# Age verification with ternary operator
age=20
status=$(( age >= 21 ? "allowed" : "underage" ))
echo "Drinking status: $status"
# Checking multiple conditions
temperature=72
humidity=65
comfort_level=$(( (temperature > 65 && temperature < 75) &&
(humidity > 40 && humidity < 70) ?
"comfortable" : "uncomfortable" ))
echo "Current conditions are $comfort_level"
# Using (( )) for conditions in if statements
count=5
if (( count > 0 && count < 10 )); then
echo "Count is between 1 and 9"
fi
Tips for Success
- Always quote your variables (
"$var"
) to prevent word splitting issues with spaces - Use parameter expansion instead of external commands like
cut
orsed
when possible - it's much faster - Use
declare -i
for variables you only use for integers to help catch errors - Test your string operations on edge cases (empty strings, special characters)
- Remember that bash arithmetic only works with integers (no floating point)
Common Mistakes to Avoid
- Forgetting to quote variables, especially when they might contain spaces
- Trying to use floating-point math in bash arithmetic (use
bc
command instead) - Forgetting that
${var#pattern}
and${var##pattern}
match from the beginning - Forgetting that
${var%pattern}
and${var%%pattern}
match from the end - Using backticks (`` `command` ``) instead of
$(command)
for command substitution
Best Practices
- Use parameter expansion over external commands when possible for better performance
- Handle empty variables safely using default values (
${var:-default}
) - Use meaningful variable names that describe their purpose
- Comment your complex parameter expansions to explain what they do
- Test for edge cases (empty strings, special characters, numeric limits)
Example: Performance Improvement
Look at how parameter expansion can dramatically improve performance in real scripts:
# Original slow version - took 3.618 seconds
# (using external commands)
for word in $(strings file.txt); do
len=$(echo -n $word | wc -c)
# ... rest of script
done
# Improved fast version - took only 0.06 seconds
# (using parameter expansion)
for word in $(strings file.txt); do
len=${#word}
# ... rest of script
done
That's a 60x performance improvement by using parameter expansion!
Example: Finding the Longest Word
#!/bin/bash
# longest-word: find longest string in a file
# Explain what we're doing
echo "Finding the longest word in your files..."
for filename; do
if [[ -r "$filename" ]]; then
max_word=
max_len=0
# Process each word in the file
for word in $(strings "$filename"); do
# Use parameter expansion to get length (much faster)
len="${#word}"
# Check if this word is longer than current max
if (( len > max_len )); then
max_len="$len"
max_word="$word"
fi
done
echo "$filename: '$max_word' ($max_len characters)"
else
echo "Error: Cannot read '$filename'" >&2
fi
done
Example: Counting Letters in a String
This example shows how to count occurrences of a specific letter in a string using a while loop and parameter expansion.
#!/bin/bash
# count-letters: counts occurrences of a specific letter in a string
# Define a string with both uppercase and lowercase 'A'
myString="Alice was Amazed At the Apple and Ate it All. Amazing!"
echo "Original string: $myString"
# Convert to lowercase using parameter expansion
myString=${myString,,} # Convert to lowercase
echo "Lowercase string: $myString"
# Initialize counter and index
count=0
i=0
# Loop through each character of the string
while (( i++ < ${#myString} )); do
# Extract the current character
char=$(expr substr "$myString" $i 1)
# Check if character is 'a' and increment counter
if [[ "$char" == "a" ]]; then
((count++))
fi
done
# Display the result
echo "The letter 'a' appears $count times in the string."
This example demonstrates:
- Using case conversion with
${string,,}
to normalize input - Getting string length with
${#string}
- Using a while loop with counters
- Extracting individual characters from a string
Example: Validating Numeric Input
This example shows how to validate that user input contains only numbers, which is useful when creating scripts that perform calculations.
#!/bin/bash
# validate-numbers: ensures input values are numeric
# Function to check if input is a number
check_num() {
# Check if parameter contains at least one digit
num=`echo $1 | grep '[0-9]'`
# Check if parameter contains any non-digit characters
non_digit=`echo $1 | grep '[^0-9]'`
if [ "$num" = '' ] || [ "$non_digit" != '' ]; then
echo "Error: '$1' is not a valid number"
return 1
fi
return 0
}
# Get two numbers from user
read -p "Enter first number: " num1
read -p "Enter second number: " num2
# Validate both inputs
if check_num "$num1" && check_num "$num2"; then
# If both are valid, perform a calculation
sum=$((num1 + num2))
echo "The sum of $num1 and $num2 is $sum"
else
echo "Please try again with valid whole numbers"
exit 1
fi
This example demonstrates:
- Creating and using shell functions
- Using grep to validate input patterns
- Checking for both the presence of digits and absence of non-digits
- Using return values to communicate success/failure
- Performing conditional arithmetic operations
With these examples and the techniques covered in this chapter, you now have powerful tools to manipulate strings and numbers in your shell scripts!