06 Mar Python Generators
Python generators are special functions that let you iterate over data lazily, producing values one at a time using the yield keyword instead of storing everything in memory. They are memory-efficient, ideal for large datasets or streams, and maintain state between iterations.
The yield keyword
In Python, the yield keyword is used inside a function to turn it into a generator, allowing values to be returned one at a time while preserving the function’s state for the next call. Unlike return, which ends the function immediately, yield pauses execution and resumes from the same point when the next value is requested.
How yield Works
- Creates a generator: A function with yield returns a generator object (an iterator).
- Lazy evaluation: Values are produced only when needed, saving memory.
- State preservation: The function remembers where it left off after each yield.
- Multiple outputs: A single function can yield multiple values across iterations.
Why is yield more memory-efficient
When you use yield inside a function, Python turns that function into a generator. A generator doesn’t build the entire list of values in memory at once. Instead, it produces values one at a time as you iterate over it.
Generators are frequently used for streaming data in programming (especially Python, JavaScript, and PHP) to process large datasets or real-time feeds efficiently without loading everything into memory. They allow for “lazy evaluation,” enabling the processing of data in chunks
Benefits of Generators
Instead of loading the entire file into memory, this generator yields one line at a time, making it efficient for huge files:
- Memory Efficient: No need to store large datasets in memory
- Lazy Evaluation: Values are generated only when needed
- Readable Code: Cleaner than manual iterator classes
- Infinite Sequences: Can generate endless streams safely
Example 1
Let us see an example:
def myfunc():
yield "Hi, John"
yield 5
yield "Bye"
for val in myfunc():
print(val)
Output
Hi, John 5 Bye
Each yield pauses the function and returns a value. The next iteration resumes from the paused point.
Code Explanation
- Defining a Generator Function
- def myfunc(): defines a function.
- Inside, instead of return, you use yield.
- This makes myfunc a generator function. When called, it doesn’t run immediately—it returns a generator object.
- Yield Statements
- yield “Hi, John” → pauses the function and outputs “Hi, John”.
- Next time the generator is resumed, it continues from where it left off.
- yield 5 → outputs 5.
- yield “Bye” → outputs “Bye”.
- Iteration with for Loop
- for val in myfunc(): automatically calls next() on the generator until it’s exhausted.
- Each yield provides one value to val.
- The loop prints each value in sequence.
Above, yield pauses execution and remembers the state.
- Each loop iteration resumes from the last yield.
- Once all yields are done, the generator ends.
Example 2
Let us see another example. Here, we have a reverse countdown:
def countdown(start):
while start > 0:
yield start
start -= 1
for val in countdown(5):
print(val)
Output
5 4 3 2 1
The generator only keeps track of the current state (start) and where it left off in the function.
When you loop with for val in countdown(5):, Python asks the generator for the next value, and it computes it on the fly.
This means no big list is stored in memory; just the current number and the function’s state.
Explanation
- Defining the Generator
- def countdown(start): creates a generator function.
- Inside, we use a while loop that runs until start becomes 0.
- Yielding Values
- yield start → pauses the function and outputs the current value of start.
- After yielding, the function doesn’t end—it remembers where it left off.
- Updating State
- start -= 1 → decreases the value by 1.
- When the generator resumes, it continues from this updated state.
- Iteration
- The for loop calls next() on the generator repeatedly.
- Each time, the generator yields the next countdown number until the loop ends.
In the above code:
- The generator pauses at each yield and resumes from the same point.
- It’s memory-efficient because it doesn’t build a list like [5,4,3,2,1]; instead, it produces values one at a time.
- Perfect for scenarios where you want to generate sequences step by step.
If you liked the tutorial, spread the word and share the link and our website, Studyopedia, with others.
For Videos, Join Our YouTube Channel: Join Now
Read More:
No Comments