Español

sum() Streams Lazily, sum([]) Builds First

Two square brackets are quietly eating your RAM in Python. sum([x * x for x in data]) and sum(x * x for x in data) return the exact same number — but the first one materializes an entire list in memory before adding a single value, and the second streams items one at a time. On a million items, that's the difference between holding a million intermediate values at once and holding roughly one.

What the brackets actually do

A list comprehension — [x * x for x in data] — builds a full list object. Every element is computed, boxed, and stored before sum() sees any of them. That list exists only to be consumed once and thrown away.

A generator expression — (x * x for x in data) — returns an iterator instead. Per the Python language reference, its values are "evaluated lazily … when the iterator is asked to yield a value." sum() pulls one value, adds it to the running total, and the value becomes garbage immediately. The full list never exists.

Python's syntax makes the fix a one-character-ish edit: PEP 289 (the generator expressions PEP, final since Python 2.4) specifies that "if a function call has a single positional argument, it can be a generator expression without extra parentheses." So inside sum(...), just delete the brackets:

# Builds a 1,000,000-element list first, then folds it
total = sum([x * x for x in range(1_000_000)])

# Streams one value at a time — same total, near-zero extra memory
total = sum(x * x for x in range(1_000_000))

When each form wins

PEP 289 also notes that "as data volumes grow larger, generator expressions tend to perform better because they do not exhaust cache memory." Lazy isn't just leaner — at scale it's often faster.

The same trick everywhere

sum() is just the poster child. Every built-in that folds an iterable to a single value takes a bare generator expression the same way:

any(user.is_admin for user in users)      # short-circuits on first True
all(len(row) == 8 for row in rows)        # short-circuits on first False
max(len(line) for line in open("app.log"))

any() and all() add a second win: they short-circuit. With a generator they stop pulling values at the first decisive element; with a list comprehension you've already paid to compute all of them.

Power-user notes

• Generators are single-use. After sum(g), the generator g is exhausted; iterating again yields nothing. If two consumers need the data, build the list once on purpose.
• The leftmost iterable is evaluated eagerly. The language reference is explicit: the iterable in the leftmost for clause runs immediately when the expression is defined — so a NameError or a bad filename fails at definition time, not at first pull. Everything else is lazy.
• Floats deserve math.fsum(). For extended-precision addition of floating-point values, the sum() docs point you to math.fsum() — it too takes a bare generator expression.
• Chaining iterables? Don't sum(list_of_lists, []) — the docs recommend itertools.chain() instead, which is also lazy.
• Watch for infinite streams. max()/min() over an unbounded generator will never return (Functional HOWTO). Laziness moves the cost; it doesn't cap it.

Resources

• Built-in sum() — Python docs
• PEP 289 — Generator Expressions
• Generator expressions — Python language reference
• Functional Programming HOWTO — generator expressions vs list comprehensions
• math.fsum() — precise float summation