Profile, Optimize, and Scale Python Code with NumPy, Numba, and Smart Algorithms

Read before you buy: the PDF here is a free sample — the first 20 pages of the 482-page book (payments, sales and reviews on this demo site are sample data). Get the full e-book at thenexbooks on Gumroad.

Measure first, fix the real bottleneck, and prove every speedup.

Every Python developer eventually meets code that is simply too slow: a nightly report that now takes an hour, a script that freezes on the real file, an endpoint that times out as data grows. Guessing at fixes wastes days. The developers who make Python fast follow a discipline instead — measure, profile, apply the cheapest fix that works, check that the answers are still correct, and measure again — and they know which tool fits which bottleneck.

This book teaches that discipline and those tools for intermediate Python developers. We take one realistic, slow codebase and make it fast step by step, explaining why each change helps and showing the measured effect and a correctness check every time.

What's inside

• The performance mindset: why CPython is slow for tight loops, when it is fast enough, latency vs throughput, and Amdahl's law
• Timing done right: perf_counter, process_time, timeit, warm-ups, garbage collection, noise, and why the minimum beats the mean
• CPU profiling with cProfile, pstats, and pyinstrument; line-level thinking; memory profiling with tracemalloc
• Big-O for working developers and choosing the right data structure: set, dict, deque, heapq, and bisect
• Fast built-ins and idioms — and which popular "tricks" make no difference, with measurements
• Caching and memoization with lru_cache and cache, precomputation, and generators that save time and memory
• NumPy for performance: dtypes and memory layout, vectorization, broadcasting, fancy indexing, reductions, views, and temporaries
• pandas and Polars performance basics: vectorized columns vs apply, categories, lazy queries, and DuckDB
• Numba: JIT-compiling hot loops, type specialization, parallel=True and prange, and common pitfalls
• Multiprocessing for CPU-bound work (with the real costs of pickling and start-up), threads, the GIL, and asyncio for I/O
• C extensions, Cython, mypyc, and Rust with PyO3 explained, plus PyPy and the experimental free-threaded and JIT CPython builds
• Memory optimization with slots, array, and columnar layouts; fast file I/O with binary formats, Parquet, and memory mapping; out-of-core processing
• Benchmarking discipline, performance regression tests with pytest, a decision playbook, and a complete end-to-end case study

One running example, start to finish. You work on the code of Stonebridge Logistics, a fictional delivery company with 12 depots, 200,000 parcels a day, and 50 vans sending a GPS ping every four seconds. Its clear, correct, pure-Python code finds the nearest depot for every parcel, prices deliveries, measures distance driven, checks scans against the manifest, plans routes, and builds a daily report that has become painfully slow. All data is generated from fixed seeds, so you get exactly the same datasets and checksums we do, and every faster version is checked against the original for identical answers.

Honest about numbers. Performance depends on your hardware and software, so every timing in the book is labeled as measured on our test machine and the focus is on ratios and method, never on universal benchmark claims. Tools that need a compiler or are experimental are clearly labeled, and no results are invented. Every chapter includes runnable code with real output, Tips, Notes, Warnings, exercises, and Key Takeaways, and the appendices give you a cheat sheet, copy-ready templates, an optimization checklist, and a glossary.

Format: PDF, 482 pages (A4), linked table of contents and bookmarks. English. First Edition, 2026. Published by thenexbooks. Examples target Python 3.12 or later. This book is independent and is not affiliated with, sponsored by, or endorsed by the Python Software Foundation or any project mentioned.

Table of Contents

Part I — Measure Before You Optimize

• Ch 1 · The Performance Mindset: Why Python Is Slow and When It Matters
• Ch 2 · The Stonebridge Workbench: Datasets, Baselines, and Correctness Checks
• Ch 3 · Timing Code Correctly: perf_counter, timeit, and Benchmark Hygiene
• Ch 4 · Profiling CPU Time: cProfile, pstats, and pyinstrument
• Ch 5 · Profiling Lines and Memory: Hot Lines, tracemalloc, and Allocation Hunting

Part II — Faster Pure Python

• Ch 6 · Algorithms First: Big-O for Working Developers
• Ch 7 · Choosing the Right Data Structure: list, dict, set, deque, heapq, and bisect
• Ch 8 · Fast Built-ins and Idioms: Loops, Comprehensions, Strings, and Lookups
• Ch 9 · Caching and Memoization: lru_cache, cache, and Precomputation
• Ch 10 · Generators and Lazy Pipelines: Saving Time and Memory

Part III — Numerical Speed with NumPy and DataFrames

• Ch 11 · NumPy for Performance: Arrays, dtypes, and Memory Layout
• Ch 12 · Vectorization: Replacing Python Loops with Array Operations
• Ch 13 · Broadcasting, Fancy Indexing, and Fast Reductions
• Ch 14 · Advanced NumPy: Views, Copies, In-Place Operations, and Temporaries
• Ch 15 · DataFrames at Speed: pandas and Polars Performance Basics

Part IV — Compiling and Parallel Execution

• Ch 16 · Numba Fundamentals: JIT-Compiling Hot Loops
• Ch 17 · Numba in Depth: Types, parallel=True, prange, and Pitfalls
• Ch 18 · Multiprocessing for CPU-Bound Work: Pools, Chunks, and Shared Memory
• Ch 19 · Threads, the GIL, and asyncio for I/O-Bound Work
• Ch 20 · Compiled Extensions: C, Cython, mypyc, and Rust with PyO3

Part V — Memory, I/O, and the Runtime

• Ch 21 · Memory Optimization: Object Overhead, slots, array, and Compact Types
• Ch 22 · Fast File I/O: CSV, Binary Formats, Parquet, and Memory Mapping
• Ch 23 · Out-of-Core Data: Chunking, Streaming, and Lazy Queries
• Ch 24 · The Runtime Itself: Startup, Imports, Garbage Collection, PyPy, and New CPython Builds

Part VI — Performance Engineering in Practice

• Ch 25 · Benchmarking Discipline: Noise, Statistics, and Fair Comparisons
• Ch 26 · Performance Regression Testing: Budgets, pytest, and CI
• Ch 27 · The Optimization Playbook: Matching Techniques to Bottlenecks
• Ch 28 · Case Study: Optimizing the Stonebridge Daily Report End to End

Appendices
A · Performance Cheat Sheet
B · Templates and Reference
C · Optimization Checklist
D · Glossary

Book details