CodeOath
← All posts
Python70 min total · 18 parts

Django Fundamentals: MVT Architecture, the ORM, and Middleware

Contents — Part 7 of 18: The N+1 Query Problem — Django's Most Common Performance Bug
Part 7 of 18 · ~1 min

The N+1 Query Problem — Django's Most Common Performance Bug

orders = Order.objects.all()
for order in orders:
    print(order.customer.name)  # one extra query PER order

If Order has a foreign key to Customer, accessing order.customer inside the loop fires a separate query for every single order — one query to get the orders, then N more to get each one's customer. This is called the N+1 problem because the total query count is N (one per row) plus the original 1. select_related (for foreign keys and one-to-one relationships) fixes it by fetching both in a single SQL JOIN:

orders = Order.objects.select_related("customer").all()
for order in orders:
    print(order.customer.name)  # no extra queries — already fetched in the JOIN

For many-to-many or reverse foreign-key relationships, prefetch_related does the equivalent job, but with a second separate query rather than a join (a join doesn't work well for "many" relationships, since it would multiply and duplicate the parent rows):

# Order has a ManyToManyField to Tag
orders = Order.objects.prefetch_related("tags").all()
for order in orders:
    print([t.name for t in order.tags.all()])  # no extra query per order — 2 queries total
select_relatedprefetch_related
Relationship typesForeignKey, OneToOneFieldManyToManyField, reverse FK, anything
SQL mechanismA single JOINThe main query, plus one additional query per prefetched relation
Result cachingJoined data is attached to each objectA Prefetch cache attached to each object

The django-debug-toolbar package (or simply logging connection.queries in a test) is the practical way to actually catch N+1 problems — they're easy to introduce without noticing, since each individual query, viewed in isolation, looks completely fine.