Rodrigo Girão Serrão: Thread-safe iterators

Wait 5 sec.

Learn how to write and use thread-safe iterators in Python 3.15.This article shows how to write and use iterators in a thread-safe manner using the new tools from Python 3.15:threading.serialize_iteratorthreading.synchronized_iterator15 days of Python 3.15To celebrate the release of Python 3.15, over the next 15 business days I'll be writing about a new 3.15 feature every day.Explained clearly and with examples so you don't have to sift through the changelog.Subscribe to receive this free email series: Enroll for free 🐍🚀 If you want, you can also check the schedule of the upcoming emails.Iterators are not thread-safeMost iterators are not thread-safe by default.For example, generator iterators can raise a ValueError if accessed simultaneously from two different threads, as you'll see now.First, you're going to define a slow generator:import timedef slow_counter(): idx = 0 while True: time.sleep(1) idx += 1 yield idxcounter = slow_counter()The generator counter takes a second to produce each successive number.Then, you're going to write a function that is going to try to consume a value out of the counter in a dedicated thread:def count_consumer(name, counter): try: idx = next(counter) except ValueError as e: print(f"ValueError in {name}: {e}") else: print(f"{name} got {idx = }")The function count_consumer will run in two different threads at the same time.The idea is that the second thread will try to fetch a value out of counter while the first thread is still waiting for the first slow result.By passing count_consumer to two different threads, you'll see that the second thread triggers an exception:t1 = threading.Thread(target=count_consumer, args=("t1", counter))t2 = threading.Thread(target=count_consumer, args=("t2", counter))t1.start()t2.start()t1.join()...