Introduction to Zsh String Manipulation
Zsh (Z shell) is a powerful Unix shell that offers extensive built-in features for string manipulation. Unlike Bash, which often requires external tools like sed or awk for complex string operations, Zsh provides native parameter expansion flags and operators that make string processing concise and efficient. This guide covers everything from basic substring extraction to advanced pattern matching and transformation.
Why String Manipulation Matters in Zsh
String manipulation is fundamental to shell scripting. Whether you are parsing file paths, sanitizing user input, formatting output, or transforming data, the ability to manipulate strings directly within the shell reduces dependencies on external commands. This leads to faster scripts, cleaner code, and better portability across systems where Zsh is installed.
Zsh's string manipulation capabilities are particularly valuable because they:
- Eliminate the overhead of spawning external processes
- Provide consistent behavior across different operating systems
- Offer powerful pattern matching with extended globbing
- Support both simple and complex transformations with readable syntax
Basic String Operations
String Length
Getting the length of a string in Zsh is straightforward using the ${#var} syntax. This returns the number of characters in the string.
#!/usr/bin/env zsh
text="Hello, World!"
echo "Length: ${#text}"
# Output: Length: 13
# Works with empty strings too
empty=""
echo "Empty length: ${#empty}"
# Output: Empty length: 0
String Concatenation
Concatenating strings in Zsh can be done simply by placing variables and literals next to each other, or by using the += operator for appending.
#!/usr/bin/env zsh
first="Hello"
second="World"
# Direct concatenation
greeting="${first}, ${second}!"
echo "$greeting"
# Output: Hello, World!
# Append using +=
message="Welcome"
message+=" to Zsh"
message+=" scripting"
echo "$message"
# Output: Welcome to Zsh scripting
Substring Extraction
Using Offset and Length
Zsh allows you to extract substrings using the ${var:offset:length} syntax. The offset is zero-based, and the length is optional.
#!/usr/bin/env zsh
string="Zsh Scripting Guide"
# Extract from position 4 to end
echo "${string:4}"
# Output: Scripting Guide
# Extract 9 characters starting from position 4
echo "${string:4:9}"
# Output: Scripting
# Negative offset counts from the end
echo "${string: -5}"
# Output: Guide
# Negative offset with length
echo "${string: -13:9}"
# Output: Scripting
Note the space before the negative number in the offset. This is required in Zsh to distinguish it from the ${var:-default} syntax.
Substring with Flags
Zsh provides the ${(e)var} flag for various operations, but for substring extraction, the offset-length syntax is the most common approach. You can also combine flags for more complex operations.
#!/usr/bin/env zsh
path="/usr/local/bin/script.sh"
# Get the filename (last component)
filename="${path:t}"
echo "$filename"
# Output: script.sh
# Get the directory
directory="${path:h}"
echo "$directory"
# Output: /usr/local/bin
# Get the file extension
extension="${filename:e}"
echo "$extension"
# Output: sh
# Get filename without extension
basename="${filename:r}"
echo "$basename"
# Output: script
Pattern Substitution
Simple Substitution
Zsh supports inline string substitution using ${var/pattern/replacement}. This replaces the first occurrence of the pattern.
#!/usr/bin/env zsh
text="the quick brown fox jumps over the lazy dog"
# Replace first occurrence
echo "${text/fox/cat}"
# Output: the quick brown cat jumps over the lazy dog
# Replace all occurrences using double slash
echo "${text//the/THE}"
# Output: THE quick brown fox jumps over THE lazy dog
Anchored Substitution
You can anchor patterns to the beginning or end of a string using # and % respectively within the substitution syntax.
#!/usr/bin/env zsh
filename="report_2024_01_15.csv"
# Remove from the beginning (prefix)
echo "${filename#report_}"
# Output: 2024_01_15.csv
# Remove the shortest match from the beginning
echo "${filename#*_}"
# Output: 2024_01_15.csv
# Remove the longest match from the beginning
echo "${filename##*_}"
# Output: 15.csv
# Remove from the end (suffix)
echo "${filename%.csv}"
# Output: report_2024_01_15
# Remove the shortest match from the end
echo "${filename%_*}"
# Output: report_2024_01_15.csv
# Remove the longest match from the end
echo "${filename%%_*}"
# Output: report
Case Modification
Zsh provides flags for changing the case of strings, which is extremely useful for formatting output.
#!/usr/bin/env zsh
text="Hello World"
# Uppercase first character
echo "${(C)text}"
# Output: Hello World
# Uppercase entire string
echo "${(U)text}"
# Output: HELLO WORLD
# Lowercase entire string
echo "${(L)text}"
# Output: hello world
# Capitalize each word
mixed="hello world from zsh"
echo "${(C)mixed}"
# Output: Hello World From Zsh
Advanced Pattern Matching
Using Extended Globbing
Zsh's extended globbing allows for powerful pattern matching. Enable it with setopt extended_glob to use advanced patterns in your string operations.
#!/usr/bin/env zsh
setopt extended_glob
text="file1.txt file2.log file3.txt file4.dat"
# Remove all .txt files from the string
echo "${text//file[0-9]##.txt/}"
# Output: file2.log file4.dat
# Replace digits with a placeholder
echo "${text//[0-9]##/N}"
# Output: fileN.txt fileN.log fileN.txt fileN.dat
# Negation pattern - match anything that is NOT a digit
sample="abc123def456"
echo "${sample//[^0-9]#/X}"
# Output: XXX123XXX456
Regular Expression Matching
For more complex pattern matching, Zsh can use the =~ operator with POSIX extended regular expressions.
#!/usr/bin/env zsh
email="user@example.com"
if [[ "$email" =~ ^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$ ]]; then
echo "Valid email format"
else
echo "Invalid email format"
fi
# Output: Valid email format
# Access captured groups with $match array
date_string="2024-01-15"
if [[ "$date_string" =~ ^([0-9]{4})-([0-9]{2})-([0-9]{2})$ ]]; then
echo "Year: ${match[1]}"
echo "Month: ${match[2]}"
echo "Day: ${match[3]}"
fi
# Output:
# Year: 2024
# Month: 01
# Day: 15
String Splitting and Joining
Splitting Strings into Arrays
Zsh makes it easy to split strings into arrays using the ${(s:delimiter:)var} flag syntax.
#!/usr/bin/env zsh
csv="apple,banana,cherry,date"
# Split by comma
fruits=("${(s:,:)csv}")
# Iterate over the array
for fruit in "${fruits[@]}"; do
echo "Fruit: $fruit"
done
# Output:
# Fruit: apple
# Fruit: banana
# Fruit: cherry
# Fruit: date
# Split by a multi-character delimiter
path_string="/usr/local/bin:/opt/homebrew/bin:/usr/sbin"
paths=("${(s.:.)path_string}")
echo "Number of paths: ${#paths[@]}"
# Output: Number of paths: 3
Joining Arrays into Strings
To join array elements into a single string, use the ${(j:delimiter:)array} flag.
#!/usr/bin/env zsh
words=("Zsh" "is" "powerful" "and" "flexible")
# Join with spaces
sentence="${(j: :)words}"
echo "$sentence"
# Output: Zsh is powerful and flexible
# Join with commas
csv_line="${(j:,:)words}"
echo "$csv_line"
# Output: Zsh,is,powerful,and,flexible
# Join with a custom separator
custom="${(j: -> :)words}"
echo "$custom"
# Output: Zsh -> is -> powerful -> and -> flexible
Trimming and Padding
Trimming Whitespace
Removing leading and trailing whitespace is a common requirement. Zsh provides several approaches.
#!/usr/bin/env zsh
setopt extended_glob
text=" Hello, World! "
# Remove leading whitespace
trimmed_left="${text## #}"
echo "Left trimmed: '${trimmed_left}'"
# Output: Left trimmed: 'Hello, World! '
# Remove trailing whitespace
trimmed_right="${text%% #}"
echo "Right trimmed: '${trimmed_right}'"
# Output: Right trimmed: ' Hello, World!'
# Remove both leading and trailing whitespace
trimmed="${text## #}"
trimmed="${trimmed%% #}"
echo "Fully trimmed: '${trimmed}'"
# Output: Fully trimmed: 'Hello, World!'
Padding Strings
Zsh provides the ${(l:length:)var} flag for left-padding and ${(r:length:)var} for right-padding.
#!/usr/bin/env zsh
name="Zsh"
# Left pad to 10 characters (right-align)
echo "[${(l:10:)name}]"
# Output: [ Zsh]
# Left pad with a specific character
echo "[${(l:10::.)name}]"
# Output: [.......Zsh]
# Right pad to 10 characters (left-align)
echo "[${(r:10:)name}]"
# Output: [Zsh ]
# Right pad with a specific character
echo "[${(r:10::-)name}]"
# Output: [Zsh-------]
# Practical use: formatting a table
items=("Apple" "Banana" "Cherry")
prices=("1.50" "0.75" "3.20")
for i in {1..${#items[@]}}; do
echo "${(r:15:)items[$i]} \$${(l:6::0:)prices[$i]}"
done
# Output:
# Apple $0001.50
# Banana $0000.75
# Cherry $0003.20
String Comparison and Testing
Comparing Strings
Zsh provides multiple ways to compare strings, including exact matching, prefix/suffix checks, and pattern matching.
#!/usr/bin/env zsh
str1="Hello"
str2="hello"
# Exact comparison (case-sensitive)
if [[ "$str1" == "$str2" ]]; then
echo "Strings are equal"
else
echo "Strings are different"
fi
# Output: Strings are different
# Case-insensitive comparison
if [[ "${(L)str1}" == "${(L)str2}" ]]; then
echo "Strings match (case-insensitive)"
fi
# Output: Strings match (case-insensitive)
# Check if string starts with a prefix
url="https://example.com"
if [[ "$url" == https:* ]]; then
echo "Secure URL"
fi
# Output: Secure URL
# Check if string ends with a suffix
file="archive.tar.gz"
if [[ "$file" == *.gz ]]; then
echo "Gzip compressed file"
fi
# Output: Gzip compressed file
# Check if string contains a substring
sentence="The quick brown fox"
if [[ "$sentence" == *"brown"* ]]; then
echo "Contains 'brown'"
fi
# Output: Contains 'brown'
Testing String Properties
#!/usr/bin/env zsh
# Check if string is empty
var=""
if [[ -z "$var" ]]; then
echo "String is empty"
fi
# Check if string is non-empty
var="content"
if [[ -n "$var" ]]; then
echo "String has content"
fi
# Check if variable is set (even if empty)
if [[ -v var ]]; then
echo "Variable is set"
fi
# Default value if variable is unset or empty
unset missing
echo "${missing:-Default Value}"
# Output: Default Value
# Assign default value if unset or empty
config_path=""
echo "${config_path:=/etc/config}"
# Output: /etc/config
echo "$config_path"
# Output: /etc/config
Working with Special Characters
Escaping and Quoting
Handling special characters properly is crucial for robust scripts. Zsh provides several quoting mechanisms.
#!/usr/bin/env zsh
# Single quotes preserve everything literally
echo 'The value is $HOME'
# Output: The value is $HOME
# Double quotes allow variable expansion
echo "The value is $HOME"
# Output: The value is /home/user
# Using $'...' for escape sequences
echo $'Tab\there\nNew line'
# Output:
# Tab here
# New line
# Quoting individual characters with backslash
echo "The price is \$5.00"
# Output: The price is $5.00
# The (q) flag quotes a string for safe reuse
dangerous="file name with spaces"
echo "Command: echo ${(q)dangerous}"
# Output: Command: echo file\ name\ with\ spaces
Removing Quotes
#!/usr/bin/env zsh
quoted="'single quoted'"
double_quoted='"double quoted"'
# Remove surrounding quotes
unquoted_single="${quoted[2,-2]}"
echo "$unquoted_single"
# Output: single quoted
unquoted_double="${double_quoted[2,-2]}"
echo "$unquoted_double"
# Output: double quoted
# Using the (Q) flag to remove one level of quoting
escaped='hello\ world'
echo "${(Q)escaped}"
# Output: hello world
Practical Examples
Example 1: URL Parser
#!/usr/bin/env zsh
parse_url() {
local url="$1"
local protocol path host query
# Extract protocol
protocol="${url%%://*}"
# Remove protocol
url="${url#*://}"
# Extract host (everything before the first / or ?)
if [[ "$url" == */* ]]; then
host="${url%%/*}"
elif [[ "$url" == *\?* ]]; then
host="${url%%\?*}"
else
host="$url"
fi
# Extract path
if [[ "$url" == */* && "$url" != *\?* ]]; then
path="/${url#*/}"
path="${path%%\?*}"
elif [[ "$url" == */* ]]; then
path="/${url#*/}"
path="${path%%\?*}"
else
path="/"
fi
# Extract query string
if [[ "$url" == *\?* ]]; then
query="${url#*\?}"
fi
echo "Protocol: $protocol"
echo "Host: $host"
echo "Path: $path"
echo "Query: ${query:-none}"
}
parse_url "https://api.example.com/v1/users?page=1&limit=10"
# Output:
# Protocol: https
# Host: api.example.com
# Path: /v1/users
# Query: page=1&limit=10
Example 2: CSV Column Extractor
#!/usr/bin/env zsh
# Extract a specific column from CSV data
extract_column() {
local csv_line="$1"
local column_num="$2"
local delimiter="${3:-,}"
# Split the line into an array
local columns=("${(s:$delimiter:)csv_line}")
# Return the requested column
echo "${columns[$column_num]}"
}
data="John,Doe,30,New York,Engineer"
echo "First Name: $(extract_column "$data" 1)"
echo "Last Name: $(extract_column "$data" 2)"
echo "Age: $(extract_column "$data" 3)"
echo "City: $(extract_column "$data" 4)"
echo "Job: $(extract_column "$data" 5)"
# Output:
# First Name: John
# Last Name: Doe
# Age: 30
# City: New York
# Job: Engineer
# Using a different delimiter
pipe_data="apple|banana|cherry"
echo "Second item: $(extract_column "$pipe_data" 2 '|')"
# Output: Second item: banana
Example 3: String Slug Generator
#!/usr/bin/env zsh
setopt extended_glob
to_slug() {
local input="$1"
local slug
# Convert to lowercase
slug="${(L)input}"
# Replace spaces and special characters with hyphens
slug="${slug//[^a-z0-9]/-}"
# Collapse multiple hyphens into one
while [[ "$slug" == *--* ]]; do
slug="${slug//--/-}"
done
# Remove leading and trailing hyphens
slug="${slug#-}"
slug="${slug%-}"
echo "$slug"
}
echo "$(to_slug "Hello, World!")"
# Output: hello-world
echo "$(to_slug "Zsh Scripting: A Complete Guide")"
# Output: zsh-scripting-a-complete-guide
echo "$(to_slug " Multiple Spaces & Symbols!!! ")"
# Output: multiple-spaces-symbols
Best Practices
Always Quote Variables
Always wrap variable references in double quotes to prevent word splitting and glob expansion from causing unexpected behavior.
# Bad - may break with spaces or special characters
file_name="my document.txt"
cat $file_name # Will fail
# Good - properly quoted
cat "$file_name" # Works correctly
Use the Correct Flag Syntax
Zsh flags are powerful but can be confusing. Use parentheses for flags and remember that multiple flags can be combined.
# Combine multiple flags
text="hello world"
echo "${(U)text}" # Uppercase
echo "${(C)text}" # Capitalize
echo "${(U:C)text}" # Both flags (order matters)
# Use ${(flags)var} for single operations
# Use ${(flags1:flags2)var} carefully - syntax varies
Prefer Native Zsh Features Over External Tools
While sed, awk, and tr are powerful, using Zsh's built-in features avoids process spawning overhead and makes your scripts more portable within Zsh environments.
# Instead of using tr for case conversion
lower=$(echo "$var" | tr 'A-Z' 'a-z')
# Use Zsh's built-in flag
lower="${(L)var}"
Handle Empty Strings Gracefully
Always account for the possibility that a string might be empty or a variable might be unset. Use default values and length checks to prevent errors.
# Provide defaults
filename="${user_input:-default.txt}"
# Check before operating
if [[ -n "$input" ]]; then
result="${input:0:10}"
else
result=""
fi
Enable Extended Globbing for Complex Patterns
When working with complex patterns, enable extended_glob at the top of your script. This unlocks powerful pattern matching features like negation and repetition.
#!/usr/bin/env zsh
setopt extended_glob
# Now you can use patterns like:
# [^abc] - negation
# ## - one or more
# # - zero or more
Test with Edge Cases
Always test your string manipulation code with edge cases including empty strings, strings with only whitespace, strings with special characters, and very long strings.
test_string_op() {
local test_cases=(
""
" "
"a"
"hello world"
" leading spaces"
"trailing spaces "
"special!@#\$%^&*()chars"
)
for case in "${test_cases[@]}"; do
echo "Testing: '${case}' -> '$(your_function "$case")'"
done
}
Conclusion
Zsh's string manipulation capabilities are extensive and powerful, offering a rich set of built-in features that eliminate the need for many external tools. From basic operations like length checking and substring extraction to advanced pattern matching, splitting, joining, and transformation, Zsh provides a comprehensive toolkit for handling text data. By mastering parameter expansion flags, the substitution syntax, and extended globbing, you can write cleaner, faster, and more maintainable shell scripts. Remember to always quote your variables, handle edge cases gracefully, and prefer native Zsh features over external commands when performance and portability matter. With these techniques in your toolkit, you will be well-equipped to tackle any string processing challenge in your Zsh scripts.