How does garbage collection work in Python?

Python employs a multi-faceted approach to automatic memory management, primarily relying on reference counting, supplemented by a generational garbage collector to handle reference cycles.

1. Reference Counting

Reference counting is Python's primary garbage collection mechanism. Every object in Python maintains a ref_count (reference count), which tracks the number of references pointing to it. When an object is created, its ref_count is 1. The count increments when a new reference points to it (e.g., assignment, passing as argument), and decrements when a reference is removed (e.g., del statement, function scope exit).

When an object's ref_count drops to zero, it means there are no longer any active references to that object, making it inaccessible. At this point, Python automatically deallocates the object's memory and calls its __del__ method (if defined). This mechanism is efficient because memory is reclaimed immediately as soon as an object is no longer needed.

While efficient for most cases, reference counting has a significant limitation: it cannot detect and reclaim memory involved in reference cycles. A reference cycle occurs when two or more objects refer to each other, forming a closed loop, even if no external references point to the cycle. In such a scenario, each object's ref_count remains at least 1, preventing them from being deallocated by reference counting alone.

2. Generational Cyclic Garbage Collector

To address the problem of reference cycles, Python includes a separate, optional generational garbage collector. This collector is specifically designed to find and reclaim objects that are part of a reference cycle but are otherwise unreachable from the root set of objects (e.g., global variables, stack frames).

The cyclic garbage collector works by dividing objects into three 'generations' (0, 1, and 2) based on their age. New objects start in Generation 0. If an object survives a garbage collection pass in its current generation, it is promoted to the next older generation. Older generations are scanned less frequently, based on the assumption that older objects are less likely to be part of a newly formed cycle and are more likely to be long-lived.

During a collection cycle, the garbage collector identifies container objects (like lists, dictionaries, custom class instances) that might participate in cycles. It temporarily decrements the reference count of objects reachable from the candidate set of objects and then identifies objects whose reference count drops to zero after this 'trial deletion'. These are the objects that are part of cycles and are otherwise unreachable. The collector then breaks these cycles and deallocates the memory.

3. The `gc` Module

Python provides the gc module, which offers an interface to the cyclic garbage collector. Developers can use this module to inspect, enable, disable, or manually trigger garbage collection runs. For example, gc.collect() forces an immediate collection of all generations. gc.isenabled() checks if the collector is active, and gc.disable() can turn it off. The collection thresholds (when a collection is triggered for each generation) can also be adjusted using gc.set_threshold().

Summary

In summary, Python's memory management combines the simplicity and efficiency of reference counting for immediate deallocation of most objects, with a sophisticated generational cyclic garbage collector to tackle the more complex issue of unreachable reference cycles. This hybrid approach ensures effective memory reclamation while minimizing performance overhead for typical Python programs.