generator (yield)

generator (yield)

What is the generator?

‣
Generator function created using yield. Generator doesn't restart on each next(), it resumes where it left off. You mostly use loop inside generator func.

Return vs yield?

  • The difference is that while a return statement terminates a function entirely, yieldstatement pauses the function saving all its states and later continues from there on successive calls.

Examples

def my_range(start, end):
		current = start
		while current < end:
				yield current
				current += 1

for num in my_range(1, 10): # range(1, 10)
		print(num)  

# OR
nums = my_range(1, 10)
next(nums)
next(nums)
‣
Simple examples
‣
Fibonacci example
‣
Reading text file example

yield & yield from

yield from g is equivalent to for v in g: yield v

Useful for creating coroutines (handles StopIteration automatically.

yield from is a transparent two way channelbetween the caller and the sub-generator.

In addition to yield, generator objects can make use of the following methods:

  • .send() - sends data to a generator
  • .throw() - to raise generator exceptions
  • .close() - to stop a generator’s iteration
‣
Sending data with send() to a generator (coroutine)
‣
Coroutine using yield from

Generator expression

# Initialize the list
my_list = [1, 3, 6, 10]

# square each term using list comprehension
list_ = [x**2 for x in my_list]

# same thing can be done using a generator expression
# generator expressions are surrounded by parenthesis ()
generator = (x**2 for x in my_list)

print(list_)
print(generator)

[1, 9, 36, 100]
<generator object <genexpr> at 0x7f5d4eb4bf50>

Generator expression vs list comprehension

  • Generator expressions generate values “just in time”, while LC executed everything at once.
‣
Examples

Generator chains

  • Used for data processing pipelines.
‣
Example

*All the ways to call generator

Here's the full set, split into how you create one and how you consume it.

Creating a generator

def gen():                    # generator function
    yield 1
    yield 2

g = gen()                     # calling it returns a generator object, runs nothing yet

squares = (x*x for x in range(5))   # generator expression

Consuming it

One item at a time

next(g)          # preferred
next(g, None)    # with default instead of StopIteration
g.__next__()     # what next() calls under the hood

Iterating

for x in g: ...

while True:
    try:
        x = next(g)
    except StopIteration:
        break

Draining into a container

list(g), tuple(g), set(g), dict(g), frozenset(g)

Unpacking

a, b, c = g          # must match length exactly
first, *rest = g
[*g], {*g}, (*g,)    # star-unpack into a literal
f(*g)                # spread as function arguments

Functions that eat an iterable

sum(g), max(g), min(g), any(g), all(g), sorted(g), len(list(g))
"".join(g)
zip(g, other), map(f, g), enumerate(g), filter(f, g)
itertools.islice(g, 3)   # partial consumption, stays lazy

Two-way communication (coroutine style)

g.send(None)        # equivalent to next(g); required to prime
g.send(value)       # value becomes the result of the paused `yield`
g.throw(ValueError) # raise an exception at the paused yield
g.close()           # raise GeneratorExit inside, run cleanup

Delegation from another generator

def outer():
    result = yield from gen()   # forwards next/send/throw, captures return value

Discarding everything fast

collections.deque(g, maxlen=0)   # C-speed exhaust with no memory

The one gotcha across all of these: a generator is single-pass. Once exhausted, every further call gives StopIteration — you have to build a new one.

References

SuperMade with Super