What is the GIL in Python, and when should you use threads versus multiprocessing?

Picture a college library with one shared computer for issuing books, and four librarians. Only one librarian can use the computer at a time. If a librarian walks to the shelves to fetch a book, she leaves the computer free and another librarian can use it. So when most of the work is walking around, four librarians help a lot. But if all the work is typing on that one computer, adding more librarians does not make it faster.

The GIL (Global Interpreter Lock) is that one computer. It is a lock inside CPython, the standard Python interpreter. A thread must hold it to run Python code.

• I/O-bound work (waiting for a network reply, a database or a file) is like walking to the shelves. Threads release the GIL while they wait, so threads help.
• CPU-bound work (heavy calculations in pure Python) is like typing. Only one thread can do it at a time, so threads do not help.

Multiprocessing is like opening a second library branch with its own computer. Each process has its own interpreter and its own GIL, so they can truly run on different CPU cores at once.

from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor
import urllib.request

def fetch(url):                       # I/O-bound: mostly waiting
    with urllib.request.urlopen(url) as r:
        return len(r.read())

def count_primes(limit):              # CPU-bound: pure computation
    count = 0
    for n in range(2, limit):
        if all(n % d for d in range(2, int(n ** 0.5) + 1)):
            count += 1
    return count

if __name__ == "__main__":
    urls = ["https://example.com"] * 10
    with ThreadPoolExecutor(max_workers=10) as pool:      # threads for I/O
        sizes = list(pool.map(fetch, urls))

    limits = [200_000] * 4
    with ProcessPoolExecutor() as pool:                   # processes for CPU
        results = list(pool.map(count_primes, limits))

The if __name__ == "__main__": guard matters for multiprocessing on Windows and macOS, where child processes re-import the main module.

• CPython manages memory with reference counting. The GIL keeps those counts and other interpreter internals safe without putting a lock on every object, which keeps single-threaded code simple and fast.
• A running thread is asked to give up the GIL at regular intervals so others get a turn, and it releases the GIL during blocking I/O.
• Many C extensions, such as NumPy for large array operations, release the GIL while doing heavy work in C, so threads can help there too.
• Processes do not share memory by default. Data passed between them is pickled and copied, which has a cost. Starting a process is also slower than starting a thread.
• The GIL does not make your own code thread-safe. Operations like count += 1 are several steps, and threads can still interleave between them. Use threading.Lock for shared state.

• Threads or asyncio: a scraper fetching 500 product pages, a service calling several payment or SMS APIs, downloading files.
• Multiprocessing: resizing thousands of images, parsing large CSVs, running a simulation, CPU-heavy feature engineering.
• Neither: for heavy number crunching, libraries like NumPy and pandas already run the loops in optimised C code.

• Saying "Python cannot do multithreading". It can; threads just do not run Python bytecode in parallel in standard CPython.
• Using threads for CPU-heavy pure Python and expecting a speed-up.
• Assuming the GIL removes the need for locks.
• Starting hundreds of processes. Use a pool sized near the number of CPU cores.
• Saying the GIL is part of the Python language. It is a detail of CPython; the language specification does not require it.

• Why do threads help when downloading files but not when computing primes?
• How is asyncio different from threading?
• Does the GIL make a list.append from two threads safe? Does it make count += 1 safe?
• How do processes share data with each other?