PowerShell Pipeline Optimization: Speed Up Slow Scripts
A slow PowerShell script is rarely slow because of PowerShell itself — it is usually slow because data flows through too many pipeline stages before it gets filtered, or because an expensive cmdlet gets called inside a loop that should have been called once. Mastering PowerShell pipeline performance means applying the filter-early principle, choosing the right collection method for the job, and measuring before and after to confirm real improvement.
Filter at the Source Not the End
The single biggest performance win in most scripts is moving filtering to the cmdlet that generates the data, not the end of the pipeline. Cmdlets like Get-ChildItem, Get-ADUser, and Get-WinEvent all support server-side or native filtering that returns far fewer objects before PowerShell processes them. Where-Object at the end processes every object first, then discards the ones that don’t match.
# Slow: loads entire tree, then filters in PowerShell
Get-ChildItem C:\Logs -Recurse | Where-Object Name -like '*.log'
# Fast: native -Filter parameter, OS-level filtering before objects are created
Get-ChildItem C:\Logs -Recurse -Filter '*.log'
# AD: filter at server side
Get-ADUser -Filter "Department -eq 'IT' -and Enabled -eq `$true" -Properties Department
.Where() and .ForEach() Method vs Cmdlet
For in-memory collections (arrays already in a variable), the array method syntax .Where()` and `.ForEach() is significantly faster than piping to Where-Object and ForEach-Object. The pipeline has per-object overhead for binding and process blocks; method calls bypass that overhead entirely. The trade-off is that methods load the entire collection into memory, so pipeline streaming is still better for very large datasets.
$users = Import-Csv C:\Data\users.csv # already in memory
# Method syntax: faster for in-memory collections
$itUsers = $users.Where({ $_.Department -eq 'IT' })
$names = $users.ForEach({ $_.DisplayName.ToUpper() })
# Pipeline: better for streaming large files
Get-Content C:\Logs\huge.log |
Where-Object { $_ -match 'ERROR' } |
Out-File C:\Logs\errors_only.log
Avoid Repeated Get-ADUser Calls
One of the most common performance killers is calling an expensive cmdlet like Get-ADUser or Get-CimInstance inside a loop. Each call opens a network connection or queries a service. Retrieve all the data you need once, store it in a hashtable keyed by the lookup value, then reference the hashtable inside the loop — O(1) lookups instead of O(n) calls.
# Slow: one AD query per record
foreach ($row in Import-Csv C:\Data\mailboxes.csv) {
$user = Get-ADUser -Identity $row.SAM -Properties Manager # N calls!
}
# Fast: one AD query, hashtable lookup
$allUsers = Get-ADUser -Filter * -Properties Manager |
Group-Object SamAccountName -AsHashTable -AsString
foreach ($row in Import-Csv C:\Data\mailboxes.csv) {
$user = $allUsers[$row.SAM] # instant hashtable lookup
}
ArrayList vs Fixed Array for Large Collections
Appending to a fixed PowerShell array with += creates a new array on every iteration — O(n²) time for large collections. Use [System.Collections.Generic.List[object]] or [System.Collections.ArrayList] instead. They grow dynamically without copying the entire array each time.
# Slow: += on array reallocates every iteration
$results = @()
foreach ($i in 1..10000) { $results += $i }
# Fast: Generic List — add is O(1) amortized
$results = [System.Collections.Generic.List[int]]::new()
foreach ($i in 1..10000) { $results.Add($i) }
# Or let the pipeline collect results (also fast)
$results = foreach ($i in 1..10000) { $i }
Streaming vs Batch Processing
For very large files or datasets that exceed available RAM, streaming beats batch. Get-Content with no switches streams one line at a time through the pipeline. Loading with -Raw or assigning to a variable pulls everything into memory. Use streaming when you transform and output one record at a time without needing the whole dataset.
# Streaming: low memory, processes line by line
Get-Content C:\Logs\10gb.log |
Where-Object { $_ -match '\[ERROR\]' } |
Set-Content C:\Logs\errors.log
# Batch: fast but requires memory proportional to file size
$content = Get-Content C:\Logs\small.log -Raw
$matches = [regex]::Matches($content, '\[ERROR\].*')
Measure Before and After Optimizing
Never assume an optimization helped — measure with Measure-Command before and after every change. Use a realistic dataset size; optimizations that shine on 100 rows may not matter on 1,000 but are critical on 100,000.
$before = Measure-Command {
Get-ChildItem C:\Logs -Recurse | Where-Object Name -like '*.log'
}
$after = Measure-Command {
Get-ChildItem C:\Logs -Recurse -Filter '*.log'
}
Write-Host "Before: $($before.TotalMilliseconds) ms"
Write-Host "After: $($after.TotalMilliseconds) ms"
Write-Host "Speedup: $([math]::Round($before.TotalMilliseconds / $after.TotalMilliseconds, 1))x"
Common Errors and Fixes
- Where-Object after Get-ChildItem -Recurse loads entire tree first — use -Filter.
Get-ChildItem -Recurse | Where-Object Name -like '*.log'enumerates every file in the tree before filtering.-Filter '*.log'passes the pattern to the file system provider, which filters at the OS level and is dramatically faster for large directory trees. - Select-Object -First N does not stop enumeration in all cases.
Select-Object -First 10does stop the pipeline once 10 objects are received, but only if the upstream cmdlet supports early termination.Get-Content | Select-Object -First 10works correctly;Get-ADUser -Filter * | Select-Object -First 10may still retrieve all users before stopping, depending on the provider implementation.
Related Cmdlets / See Also
Wrapping Up
Pipeline performance comes down to three habits: filter at the source, avoid repeated expensive calls inside loops, and choose the right collection type. Measure before and after every change with Measure-Command to confirm actual improvement rather than assumed improvement. These changes typically yield 5–50x speedups with minimal code changes.
