A good rule of thumb: if a request makes the user wait for work they don't need to see the result of, move that work out of the request. In Django projects, I use Celery with Redis for this.

What belongs in the background

  • Sending emails and push notifications
  • Generating PDFs and reports
  • Resizing uploaded images
  • Calling slow third-party APIs
  • Scheduled work: reminders, clean-ups, nightly syncs

The user gets an immediate response, and the work happens a moment later.

A minimal task

from celery import shared_task

@shared_task(bind=True, max_retries=3, default_retry_delay=30)
def send_booking_confirmation(self, booking_id):
    booking = Booking.objects.select_related("customer").get(pk=booking_id)
    try:
        email_service.send_confirmation(booking)
    except TemporaryEmailError as exc:
        raise self.retry(exc=exc)

Two habits in that snippet matter:

  1. Pass ids, not objects. The task loads fresh data when it runs, rather than working from a stale copy serialised minutes earlier.
  2. Retry what's temporary. A mail server hiccup shouldn't lose the confirmation.

Queue after the transaction commits

If you queue a task inside a database transaction, the worker can start before the data is saved and fail to find the record. Django has a clean fix:

transaction.on_commit(lambda: send_booking_confirmation.delay(booking.id))

Keep tasks small and safe to repeat

Tasks can run more than once: after a retry, or after a worker restarts mid-job. Write them so a second run does no harm, for example by checking whether the email was already sent.

Watch the queues

A tool like Flower, or simply logging task durations and failures, tells you when a queue is backing up. A growing queue is often the first sign that something downstream is struggling.