← Back to DevBytes
Developer Tutorial July 09, 2026 16 min read

Fish Scripting: Pipes and Filters Complete Guide

Understanding Pipes and Filters in Fish Shell

Pipes and filters form the backbone of command-line data processing. In Fish shell, these concepts are not only preserved from traditional Unix shells but enhanced with a cleaner, more intuitive syntax. A pipe connects the standard output of one command to the standard input of another, while filters are commands that transform, select, or process streaming data. Together they allow you to build powerful data pipelines without intermediate files.

Fish shell treats pipes as first-class citizens. Unlike some shells where syntax quirks can trip you up, Fish provides consistent, predictable behavior. Every command in a pipeline runs concurrently, and Fish manages the data flow between them seamlessly. This makes Fish an excellent choice for both interactive use and scripting when you need to process text, logs, or system output.

What Are Pipes?

🚀 Deploy your AI agent in 10 minutes

Managed Hermes hosting. Zero DevOps. 100M tokens/mo included.

Try it free →

A pipe uses the vertical bar character | to connect two or more commands. The output of the command on the left becomes the input of the command on the right. In Fish, the pipe operator works identically to its usage in Bash or other POSIX shells, but with fewer edge cases and no unexpected subshell behavior.

Here is the simplest form of a pipe in Fish:

echo "hello world" | string upper

This sends the string hello world through a pipe to the string upper command — Fish's built-in string manipulation tool — which converts it to uppercase. The result printed to the terminal is HELLO WORLD.

Pipes can chain indefinitely. Each command reads from its standard input, processes the data, and writes to standard output, which feeds the next command in the chain:

cat /var/log/syslog | grep "error" | string split " " | count

This pipeline reads a log file, filters lines containing "error", splits each line into words, and counts the total number of words across all matching lines. No temporary files, no complex loops — just a clean linear flow of data.

What Are Filters?

Filters are commands designed to consume standard input, perform some transformation or selection, and emit the result to standard output. They are the building blocks you place between pipe symbols. Classic Unix filters like grep, sed, awk, sort, uniq, and head all work in Fish exactly as they do elsewhere. Fish also provides its own powerful built-in filters through the string command family.

Common filter operations include:

Why Pipes and Filters Matter in Fish Scripting

The pipe-and-filter model brings several critical advantages to Fish scripting:

Composability

You can build complex operations by combining simple, single-purpose commands. Each command does one thing well, and pipes let you chain them without worrying about intermediate storage or complex control flow. This matches the Unix philosophy perfectly and makes scripts easier to write, read, and debug.

# Find the five most common words in a text file
cat myfile.txt | tr '[:upper:]' '[:lower:]' | string replace -a -r '[^a-z0-9]' ' ' \
  | string split ' ' | string match -r '.' | sort | uniq -c | sort -rn | head -5

Each step in this pipeline is self-contained. You can test each filter individually, then combine them. The Fish shell handles the plumbing.

Efficiency

Pipes avoid writing intermediate results to disk. Data streams directly from one process to another through memory buffers managed by the kernel. For large datasets, this saves both time and disk I/O. Fish processes run concurrently — the entire pipeline starts at once, not sequentially — so filters can work in parallel, overlapping computation with I/O.

Memory Safety for Large Data

Streaming through pipes means you never need to load an entire dataset into memory. Commands process data line by line or block by block. A pipeline like cat hugefile.csv | grep "pattern" | string replace ... works on files of any size without exhausting RAM, because each filter consumes and emits data incrementally.

Cleaner Scripts

Fish encourages pipelines over complex loops. A traditional loop to process lines often requires temporary variables, error handling, and careful quoting. A well-designed pipeline expresses the same logic declaratively. This reduces bugs and makes intentions explicit.

Basic Pipe Syntax in Fish

The pipe operator in Fish is identical to other shells, but Fish adds a few quality-of-life improvements. Commands in a pipeline can span multiple lines without backslashes if they are inside a block or a script:

cat data.txt |
grep "important" |
string replace "old" "new" |
sort |
uniq

Fish also allows comments within pipeline chains when broken across lines, which is excellent for documentation:

# Extract error counts by date from logs
cat server.log |
  grep "ERROR" |                     # Keep only error lines
  string match -r '\d{4}-\d{2}-\d{2}'| # Extract date portion
  sort |
  uniq -c |                          # Count occurrences per date
  sort -rn                           # Sort by count descending

Fish Built-in Filters: The string Command

Fish's string command is a powerful built-in filter that replaces many external tools like sed, awk, and grep for common operations. It has several subcommands, each acting as a filter:

string match — Pattern Filtering

echo "apple banana cherry date" | string match -r '\w{5,}'
# Output: apple banana cherry

The -r flag enables regex matching. Without it, string match performs glob-style matching. Only matching words pass through; non-matching data is discarded. Use -a or --all to return all matches on a line rather than stopping at the first.

string replace — Content Transformation

echo "foo bar baz foo" | string replace "foo" "qux"
# Output: qux bar baz foo

echo "foo bar baz foo" | string replace -a "foo" "qux"
# Output: qux bar baz qux

The -a flag replaces all occurrences. Without it, only the first occurrence on each line is replaced. Regex replacement uses the -r flag:

echo "cat42 dog99 bird7" | string replace -r -a '\d+' '#'
# Output: cat# dog# bird#

string split — Tokenization

echo "one,two,three,four" | string split ","
# Output:
# one
# two
# three
# four

string split breaks input into separate lines at the delimiter. This is essential for converting structured text into one-item-per-line format for downstream filters like sort and uniq.

string collect — Joining Lines

seq 1 5 | string collect
# Output: 12345

This is the inverse of string split. It joins multiple input lines into a single output line. Use the --delimiter option to insert separators:

seq 1 5 | string collect --delimiter ","
# Output: 1,2,3,4,5

string sub — Substring Extraction

echo "hello world" | string sub -l 1 -e 5
# Output: hello

Extracts characters by byte position. Combined with pipes, you can slice and dice output precisely without resorting to cut or complex awk expressions.

string trim — Whitespace Removal

printf "  padded line  \n" | string trim
# Output: padded line

Removes leading and trailing whitespace. Use --left or --right for directional trimming, or -c to specify characters to trim.

string lower / string upper — Case Conversion

echo "MiXeD CaSe" | string lower
# Output: mixed case

These are indispensable for case-insensitive comparisons in pipelines.

External Filters in Fish Pipelines

All standard Unix filter commands work seamlessly in Fish. Here are the most commonly used ones with practical pipeline examples:

grep — Pattern-Based Line Selection

ps aux | grep "fish"
# Shows only processes with "fish" in their command line

Use grep -v to invert the match (exclude lines), grep -i for case-insensitive matching, and grep -E for extended regex.

awk — Structured Text Processing

ls -l | awk '{print $5, $9}'
# Outputs file size and name for each file

awk treats each line as a record and splits it into fields by whitespace. It excels at extracting columns and performing arithmetic on streaming data.

sed — Stream Editing

cat config.ini | sed 's/debug=true/debug=false/' | sed '/^#/d'
# Changes a setting value and removes comment lines

While Fish's string replace handles many sed use cases, sed remains useful for in-place file editing and more complex line-addressable operations.

sort and uniq — Ordering and Deduplication

cat access.log | awk '{print $1}' | sort | uniq -c | sort -rn | head -10
# Top 10 IP addresses by request count

sort orders lines alphabetically or numerically. uniq removes duplicate adjacent lines; the -c flag counts occurrences. Always sort before using uniq, as it only detects consecutive duplicates.

head and tail — Windowing

cat largefile.csv | tail -n +2 | head -100
# Skip the header line, then take the first 100 data rows

head outputs the first N lines; tail outputs the last N lines or starts from line N with the -n +N syntax. These are crucial for sampling and pagination in pipelines.

wc and count — Line/Word/Character Counting

echo "one two three" | wc -w
# Output: 3

echo "one two three" | string split " " | count
# Output: 3 (using Fish's built-in count)

wc (word count) is the traditional tool. Fish's count command counts lines of input and is often more convenient when working with line-oriented data.

tee — Pipeline Branching

cat data.txt | grep "error" | tee errors.txt | wc -l
# Saves error lines to a file while also counting them

tee duplicates the pipeline stream — one copy goes to a file, the other continues down the pipe. This is invaluable for logging or debugging without breaking the flow.

Redirecting Standard Error in Fish Pipelines

By default, pipes only connect standard output (file descriptor 1). Standard error (file descriptor 2) from the left-hand command still goes to the terminal. Fish provides a clean syntax for redirecting stderr through the pipe as well:

some_command 2>&1 | grep "error"

This merges stderr into stdout before the pipe, so grep can filter both streams. Alternatively, you can pipe only stderr while leaving stdout untouched using a more advanced construct:

some_command 2>| grep "error"

The 2>| syntax in Fish redirects only stderr into the pipe. This is a Fish-specific improvement over the POSIX syntax, which requires subshell tricks to achieve the same effect.

You can also pipe stdout and stderr to different destinations using a combination of redirects:

some_command > >(grep "INFO" >> info.log) 2> >(grep "ERROR" >> error.log)

Fish supports process substitution with () similarly to Bash, allowing you to treat command outputs as files for redirection purposes.

Advanced Pipe Patterns in Fish

Using Variables with Pipes

Fish allows you to capture pipeline output into variables directly:

set total_lines (cat data.txt | wc -l)
echo "The file has $total_lines lines"

The () command substitution syntax captures the entire pipeline output. You can then use the variable in subsequent commands. This is cleaner than storing intermediate files.

For multi-line output, Fish preserves newlines properly:

set error_dates (cat server.log | grep "ERROR" | string match -r '\d{4}-\d{2}-\d{2}')
# $error_dates is now a list, one date per element
count $error_dates
# Use count on the list variable, not on piped input

Piping into while Loops

Fish supports reading piped input line by line in loops. This is useful when you need to perform complex per-line operations that are awkward in a pure pipeline:

cat urls.txt | while read -l url
    set response (curl -s -o /dev/null -w "%{http_code}" $url)
    echo "$url returned HTTP $response"
end

The read -l command reads one line at a time from standard input. The -l flag puts the line into a local variable. This pattern combines the streaming efficiency of pipes with the flexibility of imperative logic when needed.

Named Pipes for Complex Topologies

For pipelines that need to feed multiple consumers or create feedback loops, Fish supports named pipes (FIFOs) just like any Unix shell:

mkfifo mypipe
# In one terminal or background process:
cat largefile.txt | grep "pattern" > mypipe
# In another:
cat mypipe | sort | uniq -c

Named pipes allow you to split a pipeline across different processes, terminals, or even machines when combined with network tools like ssh or nc.

Piping Through Functions

Fish functions can act as filters. Any function that reads from standard input and writes to standard output can participate in a pipeline:

function to_json_lines
    while read -l line
        echo "{\"value\": \"$line\"}"
    end
end

seq 1 5 | to_json_lines
# Outputs JSON objects, one per line

This makes Fish scripting extremely composable — you can build your own filter functions and combine them with built-in and external commands seamlessly.

Error Handling in Pipelines

Fish tracks the exit status of all commands in a pipeline, not just the last one. The special variable $pipestatus contains a list of exit codes:

false | true | false
echo $pipestatus
# Output: 1 0 1

You can check individual pipeline stages for failure. The overall pipeline exit status (stored in $status) follows the rightmost command by default, but you can use $pipestatus to implement more nuanced error handling:

cat data.txt | grep "rare_pattern" | sort
if contains 1 $pipestatus
    echo "Warning: grep found no matches, but pipeline continued"
end

This is invaluable for robust scripts where intermediate steps might legitimately produce empty output without being errors.

Fish vs. Bash: Pipeline Differences

Developers coming from Bash will find Fish pipelines mostly familiar, but several differences are worth noting:

Best Practices for Fish Pipelines

Prefer Fish Built-ins When Possible

Use string commands instead of sed and awk for simple text operations. Built-ins are faster, more predictable, and don't vary between systems. Reserve external tools for complex operations that Fish can't handle natively.

# Good: Fish-native pipeline
cat file.txt | string replace -a "old" "new" | string lower | string trim

# Unnecessary external dependency
cat file.txt | sed 's/old/new/g' | tr '[:upper:]' '[:lower:]' | sed 's/^[ \t]*//;s/[ \t]*$//'

Minimize Pipeline Depth

Deep pipelines become hard to debug. If you have more than 5-6 pipe stages, consider breaking the logic into functions or using intermediate variables. Each additional stage adds cognitive overhead and potential failure points.

# Hard to debug at a glance
cat data.txt | grep -v "^#" | string replace -a '\t' ',' | string split ',' \
  | string match -r '^\d+' | sort -n | uniq -c | sort -rn | head

# Better: extract logical steps into functions
function parse_csv
    cat $argv[1] | grep -v "^#" | string replace -a '\t' ',' | string split ','
end

function extract_numbers
    string match -r '^\d+'
end

parse_csv data.txt | extract_numbers | sort -n | uniq -c | sort -rn | head

Always Check Pipeline Exit Status

Use $pipestatus for scripts that must be robust. A pipeline can succeed overall even if an intermediate step fails. Explicitly check critical stages:

cat important.log | grep "FATAL" | wc -l
if test $pipestatus[2] -ne 0
    echo "Error: grep failed or returned no matches" >&2
end

Use tee for Debugging

When developing complex pipelines, insert tee at intermediate points to inspect the data flowing through. Remove tee stages when the pipeline is verified:

# During development
cat data.txt | string replace "X" "Y" | tee /dev/tty | grep "pattern" | sort

# After verification
cat data.txt | string replace "X" "Y" | grep "pattern" | sort

The /dev/tty argument writes to the terminal without breaking the pipe.

Quote Carefully Around Pipes

The pipe character itself doesn't require quoting, but arguments to commands in the pipeline do. Fish's quoting rules are simpler than Bash's, but still require attention with special characters:

echo "data with spaces and | symbol" | grep "spaces and |"
# The pipe inside quotes is literal, not an operator

Leverage Process Substitution for Complex Redirects

When a pipeline needs to feed two different commands, process substitution is cleaner than temporary files:

diff (cat file1.txt | sort | uniq) (cat file2.txt | sort | uniq)
# Compare sorted, deduplicated versions of two files

Handle Empty Input Gracefully

Many filters behave differently when input is empty. Test your pipelines with empty input to ensure they handle the edge case:

# This might hang waiting for input if data.txt is empty
cat data.txt | grep "pattern" | sort

# Safer: explicitly handle empty input
if test -s data.txt
    cat data.txt | grep "pattern" | sort
else
    echo "No data to process" >&2
end

Real-World Pipeline Examples

Example 1: Log Analysis Dashboard

# Generate a summary of HTTP status codes from an access log
cat /var/log/nginx/access.log |
  awk '{print $9}' |                      # Extract status code field
  string match -r '^[2345]\d{2}$' |        # Keep only valid status codes
  sort |
  uniq -c |
  sort -rn |
  string replace -r '^\s+' '' |            # Trim leading spaces from uniq -c
  awk '{printf "%-5s %s\n", $2, $1}'       # Format as "code count"

# Output example:
# 200   1523
# 404   89
# 500   12
# 302   8

Example 2: Batch File Renaming Preview

# Preview renaming all .jpg files to include their modification date
ls *.jpg | while read -l filename
    set timestamp (stat -c %Y $filename | date -r - +%Y%m%d)
    echo "$filename -> $timestamp-$filename"
end

Example 3: Multi-Source Data Merge

# Combine and deduplicate user lists from multiple servers
function get_users
    ssh $argv[1] "cat /etc/passwd | cut -d: -f1"
end

get_users server1 | string collect --delimiter '\n' > /tmp/users.txt
get_users server2 >> /tmp/users.txt
get_users server3 >> /tmp/users.txt

cat /tmp/users.txt | sort | uniq | string split '\n' | count
# Reports total unique users across all servers

Example 4: Real-Time Monitoring Pipeline

# Monitor systemd journal for failed services in real time
journalctl -f |
  grep "FAILED" |
  while read -l line
      set service_name (echo $line | string match -r '(\w+\.service)' | string sub -s 2)
      echo "[ALERT] Service $service_name failed at "(date)
  end

Conclusion

Pipes and filters represent one of the most powerful paradigms in command-line computing, and Fish shell honors this tradition while modernizing it with cleaner syntax and powerful built-in tools. The pipe operator connects commands into flowing data pipelines; filters transform, select, and process that data without intermediate storage. Fish's string command replaces many external dependencies, $pipestatus enables robust error handling, and the consistent behavior across pipeline stages reduces the debugging burden that plagues other shells. By mastering pipes and filters in Fish, you gain the ability to express complex data transformations declaratively — writing scripts that are concise, efficient, and maintainable. Whether you are analyzing logs, processing configuration files, or building ETL pipelines, the techniques covered in this guide provide a complete foundation for productive Fish scripting.

🚀 Need a reliable AI agent for your project?

Deploy Hermes Agent in 10 minutes. Managed hosting, zero DevOps.

Get Started — $23.99/mo
← Back to all articles