Enumerator

Class Enumerator supports:

• External iteration.

• Internal iteration.

An Enumerator may be created by the following methods:

• Object#to_enum.

• Object#enum_for.

• Enumerator.new.

In addition, certain Ruby methods return Enumerator objects: a Ruby iterator method that accepts a block may return an Enumerator if no block is given. There are many such methods, for example, in classes Array and Hash. (In the documentation for those classes, search for new_enumerator.)

Internal Iteration

In _internal iteration_, an iterator method drives the iteration and the caller’s block handles the processing; this example uses method each_with_index:

words = %w[foo bar baz] # => ["foo", "bar", "baz"]
enumerator = words.each # => #<Enumerator: ...>
enumerator.each_with_index {|word, i| puts "#{i}: #{word}" }
0: foo
1: bar
2: baz

Iterator methods in class Enumerator include:

• each: passes each item to the block.

• each_with_index: passes each item and its index to the block.

• each_with_object (aliased as with_object): passes each item and a given object to the block.

• with_index: like each_with_index, but starting at a given offset (instead of zero).

Class Enumerator includes module Enumerable, which provides many more iterator methods.

External Iteration

In _external iteration_, the user’s program both drives the iteration and handles the processing in stream-like fashion; this example uses method next:

words = %w[foo bar baz]
enumerator = words.each
enumerator.next # => "foo"
enumerator.next # => "bar"
enumerator.next # => "baz"
enumerator.next # Raises StopIteration: iteration reached an end

External iteration methods in class Enumerator include:

• feed: sets the value that is next to be returned.

• next: returns the next value and increments the position.

• next_values: returns the next value in a 1-element array and increments the position.

• peek: returns the next value but does not increment the position.

• peek_values: returns the next value in a 1-element array but does not increment the position.

• rewind: sets the position to zero.

Each of these methods raises FrozenError if called from a frozen Enumerator.

External Iteration and Fiber

External iteration that uses Fiber differs significantly from internal iteration:

• Using Fiber adds some overhead compared to internal enumeration.

• The stacktrace will only include the stack from the Enumerator, not above.

• Fiber-local variables are not inherited inside the Enumerator Fiber, which instead starts with no Fiber-local variables.

• Fiber storage variables are inherited and are designed to handle Enumerator Fibers. Assigning to a Fiber storage variable only affects the current Fiber, so if you want to change state in the caller Fiber of the Enumerator Fiber, you need to use an extra indirection (e.g., use some object in the Fiber storage variable and mutate some ivar of it).

Concretely:

Thread.current[:fiber_local] = 1
Fiber[:storage_var] = 1
e = Enumerator.new do |y|
  p Thread.current[:fiber_local] # for external iteration: nil, for internal iteration: 1
  p Fiber[:storage_var] # => 1, inherited
  Fiber[:storage_var] += 1
  y << 42
end

p e.next # => 42
p Fiber[:storage_var] # => 1 (it ran in a different Fiber)

e.each { p _1 }
p Fiber[:storage_var] # => 2 (it ran in the same Fiber/"stack" as the current Fiber)

Converting External Iteration to Internal Iteration

You can use an external iterator to implement an internal iterator as follows:

def ext_each(e)
  while true
    begin
      vs = e.next_values
    rescue StopIteration
      return $!.result
    end
    y = yield(*vs)
    e.feed y
  end
end

o = Object.new

def o.each
  puts yield
  puts yield(1)
  puts yield(1, 2)
  3
end

# use o.each as an internal iterator directly.
puts o.each {|*x| puts x; [:b, *x] }
# => [], [:b], [1], [:b, 1], [1, 2], [:b, 1, 2], 3

# convert o.each to an external iterator for
# implementing an internal iterator.
puts ext_each(o.to_enum) {|*x| puts x; [:b, *x] }
# => [], [:b], [1], [:b, 1], [1, 2], [:b, 1, 2], 3