PipeAssign
PipeAssign provides a macro for capturing intermediate values in Elixir pipe chains without breaking the flow or requiring separate assignment statements.
⚠️ Warning
This project was created for research purposes. To understand how assign_to/2
will affect codebase:
- What performance overhead would be?
- What readability would be, worse or better?
The Problem
Traditional Elixir code often forces you to choose between clean pipe flow and intermediate value access:
# Clean pipes, but no intermediate access
final_result = data |> transform() |> process() |> finalize()
# Intermediate access, but broken flow
step1 = data |> transform()
step2 = step1 |> process()
final_result = step2 |> finalize()
The Solution
PipeAssign bridges this gap by allowing you to capture values while maintaining the elegance of pipe operators:
import PipeAssign
data
|> transform()
|> assign_to(step1) # Capture without breaking flow
|> process()
|> assign_to(step2)
|> finalize()
|> assign_to(result) # Assign to result variable
# Now you have both clean pipe flow, access to step1, step2, result variables and no
# assignment before pipes.
Installation
Add pipe_assign
to your list of dependencies in mix.exs
:
def deps do
[
{:pipe_assign, "~> 1.0"}
]
end
Usage
Basic Usage
Import the module and use assign_to/2
in your pipes:
import PipeAssign
[1, 2, 3]
|> Enum.map(&(&1 * 2))
|> Enum.sum()
|> assign_to(result)
# result == 12
Multiple Assignments
Chain multiple assignments in a single pipe:
import PipeAssign
%{name: "John", age: 30}
|> Map.put(:email, "john@example.com")
|> assign_to(with_email)
|> Map.put(:active, true)
|> assign_to(complete)
|> Map.keys()
|> length()
|> assign_to(result)
# result == 4
# with_email == %{name: "John", age: 30, email: "john@example.com"}
# complete == %{name: "John", age: 30, email: "john@example.com", active: true}
Existing Variables
Works seamlessly with existing variables (no compiler warnings):
import PipeAssign
temp = nil
"hello world"
|> String.upcase()
|> assign_to(temp)
|> String.length()
|> assign_to(length)
# length == 11
# temp == "HELLO WORLD"
Without Import
Use the fully qualified name for occasional use:
[1, 2, 3, 4, 5]
|> Enum.filter(&rem(&1, 2) == 0)
|> PipeAssign.assign_to(evens)
# evens == [2, 4]
Use Cases
Recommended for:
- Debugging: Capture intermediate states for inspection during development
- Logging: Store values for audit trails or monitoring in non-critical paths
- Conditional Logic: Make decisions based on intermediate results
- Testing: Verify intermediate transformations in complex pipelines
- Development: Avoid redundant computations by caching intermediate results during prototyping
⚠️ Performance Note: This macro introduces minimal overhead (~1% or less) and can be used freely in most scenarios. See Performance Considerations for details.
Examples
Data Processing Pipeline
import PipeAssign
def process_user_data(raw_data) do
raw_data
|> Jason.decode!()
|> assign_to(parsed_json)
|> normalize_keys()
|> assign_to(normalized)
|> validate_required_fields()
|> assign_to(validated)
|> save_to_database()
|> assign_to(result)
Logger.info("Processed user data", %{
raw_size: byte_size(raw_data),
parsed_keys: Map.keys(parsed_json),
normalized_count: map_size(normalized),
validation_status: validated.status
})
result
end
API Response Processing
import PipeAssign
def fetch_and_process_posts(user_id) do
user_id
|> fetch_user_posts()
|> assign_to(raw_posts)
|> Enum.filter(&(&1.published))
|> assign_to(published_posts)
|> Enum.sort_by(&(&1.created_at), :desc)
|> assign_to(sorted_posts)
|> Enum.take(10)
|> format_for_api()
|> tap(fn _ ->
Analytics.track("posts_fetched", %{
user_id: user_id,
total_posts: length(raw_posts),
published_posts: length(published_posts),
returned_posts: length(sorted_posts)
})
end)
end
Smart Variable Handling
The macro automatically detects whether the variable already exists in the current scope:
- If the variable doesn't exist, it creates a new one using
var!()
- If the variable already exists, it reassigns it using regular assignment and references the old value to avoid "unused variable" warnings
This means you can use the same macro whether the variable is new or existing.
Performance Considerations
While assign_to/2
provides significant convenience for debugging and intermediate value capture, benchmarking shows it introduces minimal performance overhead:
- Hot path operations: ~1% slower (1% slower)
- String processing: Essentially no impact
- Complex pipelines: Negligible impact (~0.3% slower)
- List operations: Essentially no impact (~0.001% slower)
- Map operations: Essentially no impact (~0.3% slower)
- Memory usage: Slight increase due to intermediate variable storage
When to Use
✅ Good use cases:
- Development and debugging workflows
- Complex data processing pipelines
- Non-critical application paths
- Code where readability and debugging outweigh small performance costs
❌ Consider alternatives for:
- Extremely performance-critical hot paths
- Memory-constrained environments with strict limitations
- High-frequency loops
Performance Impact
Based on benchmarking with MacBook Air M1 16GB, the overhead is minimal:
# Hot path - 1% slower, acceptable for most use cases:
data |> transform() |> assign_to(intermediate) |> process()
# Complex pipelines - negligible impact, use freely:
data
|> complex_transform()
|> assign_to(step1) # <-- Minimal overhead (~0.1 μs)
|> another_complex_operation()
# Traditional assignment only needed for extreme performance requirements:
intermediate = data |> transform()
intermediate |> process()
Benchmarking
PipeAssign includes a comprehensive benchmarking suite to help you understand the performance implications in your specific use cases.
Running Benchmarks
# Quick comparison (recommended for most users)
mix benchmark
# Comprehensive benchmark suite (takes longer)
mix benchmark --full
# Specific benchmark types
mix benchmark --type=hotpath # Performance-critical scenarios
mix benchmark --type=complex # Multi-step pipelines
mix benchmark --type=string # String processing
mix benchmark --type=list # List operations
mix benchmark --type=map # Map manipulations
Understanding Results
The benchmarks compare assign_to/2
against traditional assignment patterns across various scenarios. All performance measurements were conducted on MacBook Air M1 16GB running macOS.
- Hot path scenarios: Simple operations where overhead is most visible
- Complex pipelines: Multi-step transformations where overhead is proportionally smaller
- Different data sizes: Small, medium, and large datasets
- Various data types: Lists, strings, maps, and mixed operations
Sample Results
Latest benchmark results (MacBook Air M1 16GB, macOS) show:
Name ips average deviation median 99th %
Hot Path Traditional 7.69 M 130.07 ns ±5190.73% 125 ns 167 ns
Hot Path assign_to/2 7.62 M 131.15 ns ±5253.34% 125 ns 167 ns
String assign_to/2 21.12 K 47.35 μs ±11.25% 47.50 μs 61.54 μs
String Traditional 21.01 K 47.60 μs ±29.76% 47.50 μs 62.67 μs
Complex Traditional 24.70 K 40.48 μs ±14.02% 40.92 μs 64.90 μs
Complex assign_to/2 24.64 K 40.59 μs ±14.10% 41.04 μs 64.24 μs
List Traditional 52.73 K 18.96 μs ±21.24% 18.67 μs 24.00 μs
List assign_to/2 52.72 K 18.97 μs ±21.30% 18.67 μs 23.18 μs
Map Traditional 236.04 K 4.24 μs ±176.91% 4.13 μs 6.42 μs
Map assign_to/2 235.40 K 4.25 μs ±177.41% 4.13 μs 6.29 μs
Comparison:
Hot Path Traditional 7.69 M
Hot Path assign_to/2 7.62 M - 1.01x slower +1.08 ns
String assign_to/2 21.12 K
String Traditional 21.01 K - 1.01x slower +0.25 μs
Complex Traditional 24.70 K
Complex assign_to/2 24.64 K - 1.00x slower +0.106 μs
List Traditional 52.73 K
List assign_to/2 52.72 K - 1.00x slower +0.00176 μs
Map Traditional 236.04 K
Map assign_to/2 235.40 K - 1.00x slower +0.0114 μs
Key Insights
- Hot paths: ~1% overhead (1% slower)
- String operations: Essentially no impact
- Complex pipelines: Negligible difference (~0.3% slower)
- List operations: Essentially no impact (~0.001% slower)
- Map operations: Essentially no impact (~0.3% slower)
- Real-world impact: Minimal performance cost
Test Environment: All benchmarks performed on MacBook Air M1 16GB, macOS, Elixir 1.18.3, Erlang/OTP 27.3.3.
Use these benchmarks to make informed decisions about where to use assign_to/2
in your codebase.
Testing
The library includes comprehensive test coverage. Run the tests with:
mix test
To check test coverage:
mix test --cover
Support Matrix
Tests automatically run against a matrix of OTP and Elixir Versions, see the ci.yml for details.
OTP \ Elixir | 1.15 | 1.16 | 1.17 | 1.18 |
---|---|---|---|---|
25 | ✅ | ✅ | ✅ | ✅ |
26 | ✅ | ✅ | ✅ | ✅ |
27 | N/A | N/A | ✅ | ✅ |
Documentation
Full documentation is available at https://hexdocs.pm/pipe_assign.
License
This project is licensed under the Apache License 2.0 - see the LICENSE file for details.