Python Bytes

#499 So many questions??

Listen

Description

Topics covered in this episode:

Watch on YouTube

About the show

Sponsored by us! Support our work through:

Connect with the hosts

Join us on YouTube at pythonbytes.fm/live to be part of the audience. Usually Tuesday at 7am PT. Older video versions available there too.

Finally, if you want an artisanal, hand-crafted digest of every week of the show notes in email form? Add your name and email to our friends of the show list, we'll never share it.

Michael #1: PEP 824 brings ?? and ??= to Python for None handling

PEP 824, a new draft from Marc Mueller sponsored by Guido van Rossum, proposes None-coalescing operators for Python 3.16. The ?? operator returns its left-hand side unless that value is None, and ??= assigns a fallback only when the target is None. It revives the long-deferred PEP 505 from over a decade ago, trimmed to just these two operators, with None-aware access split off into PEP 823. The goal is replacing verbose is None checks with something closer to or, but keyed on None instead of truthiness.

# OG way

user = db.get_user()

city = None

if user is not None:

    if user.address is not None:

        if user.address.city is not None:

            city = user.address.city
# Michael's preferred way

user = db.get_user()

city = user?.address?.city

Calvin #2: Python 3.15 gets a surprise RC3, and Python 3.10 reaches end of life

Michael #3: asyncio.shield

Where async code can be cancelled

In async Python, every await is a point where your coroutine can be stopped. If someone cancels the task, asyncio raises CancelledError at whichever await the coroutine is paused on, and nothing after that line runs.
Quart does this on purpose. When the client disconnects, it cancels the request's task. From quart/asgi.py:

elif message["type"] == "http.disconnect":

    self._disconnected = True

    request.body.disconnect()

    request_task.cancel()

That's reasonable for a slow page nobody is waiting on anymore. It's a problem for a handler that writes to the database in two steps:

await downloads.insert_one(event)            # 1. log the download

await totals.update_one(..., {'$inc': ...})  # 2. bump the episode's total

A podcast app requests the MP3, gets what it needs, and hangs up. If it hangs up while step 2 is in flight, step 1 has already landed and step 2 never runs. Nothing is logged and nothing reaches Sentry, because a cancelled request isn't an error. The two collections just drift apart.
Under WSGI this couldn't happen. A disconnect didn't stop your view; the view finished both writes and the server only failed later, when it tried to send the response.

The demo

I ran this on Python 3.14. The "client" hangs up 15 ms into a pair of 10 ms writes:

async def record_download():

    await db_write('events')

    await db_write('total')

_pending: set[asyncio.Task] = set()

async def record_download_shielded():

    task = asyncio.create_task(record_download())

    _pending.add(task)

    task.add_done_callback(_pending.discard)

    await asyncio.shield(task)

record_download            events=1 total=0

record_download_shielded   events=1 total=1

The unshielded version loses the increment. The shielded version finishes it, even though the request was cancelled. The full script is in the scratchpad as shield_demo.py if you want it for the show.

What asyncio.shield does, and its traps

shield(task) protects the inner task from a cancellation aimed at the outer one. The handler still gets CancelledError immediately, so nothing after that line runs, but the inner task keeps going until both writes finish. The traps:

  1. Keep a reference to the task. The event loop only holds weak references to tasks, so a task with no other reference can be garbage-collected mid-run. The asyncio docs warn about this. That's what _pending is for.
  2. Its errors go nowhere by default. If a shielded write fails after the handler has already been cancelled, nobody awaits the result. You only get a "Task exception was never retrieved" warning. Log errors from inside the task, or in the done callback.
  3. It doesn't survive a process shutdown. If the event loop is closing, as during a Granian worker recycle or a deploy, pending tasks are still cancelled. The window is tiny, but it isn't zero.
  4. It hides hangs. A shielded write to a stuck database lives forever. Put a timeout inside the shielded task (asyncio.timeout(...)), not outside it.
  5. It makes the pair finish, not atomic. A crash between the two writes still splits them. The real fix is one operation that can't half-happen. That's what the SQLite move gets: an insert and an upsert in one synchronous transaction, with no await in the middle. Shield is the right patch for the Mongo code until then.
    The general lesson: in async code, related writes made one after another are only as reliable as the client's patience. Before shipping something like that, check whether your framework cancels on disconnect. Quart does, and I believe Django 5's async views do too. I don't think Starlette cancels a plain endpoint, but I haven't checked any of these other frameworks' source, so verify them before you say it on air.

Calvin #4: Pyxel: the retro game engine for Python

Extras

Calvin:

Joke: I was there.