Image Classification, Object Detection, Segmentation, and Deployment with PyTorch

Read before you buy: the PDF here is a free sample — the first 20 pages of the 456-page book (payments, sales and reviews on this demo site are sample data). Get the full e-book at thenexbooks on Gumroad.

Build a real vision system, from pixels to production — and understand every step.

Deep learning made it possible for ordinary software to look at an image and decide what is in it. But a vision system is a pipeline with traps at every stage: images normalized twice, augmentation that creates pictures your camera could never take, a model that scores well on the test set and fails when a new lamp is installed, an exported model that gives slightly different answers. This book teaches the whole pipeline, with the traps, by building one complete system step by step in PyTorch and torchvision.

Every example runs on a plain CPU, and every output in the book is the real output we got.

What's inside

• Images as tensors: pixels, channels, color spaces, file formats, resizing, and the (H, W, C) vs (C, H, W) traps
• A seeded synthetic image generator that builds a complete dataset — classification images, bounding boxes, and pixel masks — so every number is reproducible
• Classic image processing (filters, thresholds, morphology) and a rule-based baseline every model must beat
• Datasets, DataLoaders, and torchvision transforms v2; convolutions from first principles; your first CNN
• Data augmentation that matches how your images are really made — and an experiment showing how the "textbook" augmentation can hurt
• Honest evaluation: confusion matrices, per-class precision and recall, business costs of errors, and decision thresholds
• Batch normalization, residual connections, a tour of ResNet, MobileNet, EfficientNet, and ConvNeXt, and transfer learning done right (including the frozen-BatchNorm trap)
• Vision Transformers from scratch and pretrained; foundation-model embeddings for similarity search; zero-shot classification and where it fails
• Object detection: IoU, anchors, non-maximum suppression, fine-tuning Faster R-CNN, and mean average precision computed by hand
• Semantic segmentation with a U-Net and a pretrained LR-ASPP model, IoU per class, and measuring defect area
• Explainability with Grad-CAM, distribution shift from a new camera, calibration, confidence thresholds, finding wrong labels, class imbalance, and reproducible experiments
• Deployment: torch.export, ONNX and ONNX Runtime, quantization, honest benchmarking, a FastAPI prediction service, and monitoring in production

One running example, start to finish. You build the apple-grading system for Greenvale Orchards, a fictional family-run orchard and packhouse. A camera above the conveyor belt must grade every apple as good, bruised, or scab and send it to fresh pack, juice, processing, or a manual check when the model is unsure. You start with a hand-written rule-based grader and finish with an exported, served, and monitored model — plus a detector that finds every apple on a tray and a segmentation model that outlines each defect.

Current and honest. Examples were tested with PyTorch 2.x and torchvision on CPU, with recent versions of Hugging Face Transformers, ONNX, and ONNX Runtime. The dataset is synthetic and the book says clearly where real photos would be harder. GPU-only and not-installed tools are labeled as such. There are no invented benchmarks or statistics. Every chapter includes runnable code, real output, Tips, Notes, Warnings, exercises, and Key Takeaways, and the appendices give you a cheat sheet, the complete dataset generator and templates, a production checklist, and a glossary.

Format: PDF, 456 pages (A4), linked table of contents and bookmarks. English. First Edition, 2026. Published by thenexbooks. This book is independent and is not affiliated with, sponsored by, or endorsed by the PyTorch project, The Linux Foundation, or any other organization mentioned.

Table of Contents

Part I — Foundations

• Ch 1 · What Computer Vision Is and How Deep Learning Changed It
• Ch 2 · Setting Up a Vision Workbench: Python, PyTorch, torchvision, and the Greenvale Project
• Ch 3 · Images as Tensors: Pixels, Channels, Color, and File Formats
• Ch 4 · Building the Greenvale Apple Dataset: A Seeded Synthetic Image Generator
• Ch 5 · Classic Image Processing: Filters, Thresholds, and a Rule-Based Baseline

Part II — Image Classification

• Ch 6 · Datasets, DataLoaders, and Transforms for Images
• Ch 7 · Convolutions from First Principles
• Ch 8 · Your First CNN: Training AppleNet to Grade Apples
• Ch 9 · Data Augmentation That Matches the Packhouse
• Ch 10 · Evaluating Image Classifiers: Confusion Matrices, Per-Class Metrics, and Thresholds

Part III — Modern Architectures and Pretrained Models

• Ch 11 · Making CNNs Train Well: Batch Norm, Residual Connections, and Regularization
• Ch 12 · A Tour of Modern Architectures: ResNet, MobileNet, EfficientNet, and ConvNeXt
• Ch 13 · Transfer Learning: Feature Extraction and Fine-Tuning
• Ch 14 · Vision Transformers: Patches, Attention, and When to Use Them
• Ch 15 · Vision Foundation Models: Embeddings, Similarity Search, and Zero-Shot Classification

Part IV — Detection and Segmentation

• Ch 16 · Object Detection Concepts: Boxes, IoU, Anchors, and Non-Maximum Suppression
• Ch 17 · Training a Detector: Fine-Tuning Faster R-CNN on Greenvale Trays
• Ch 18 · Evaluating Detectors: Precision, Recall, and Mean Average Precision
• Ch 19 · Semantic Segmentation: A U-Net for Defect Masks
• Ch 20 · Pretrained Segmentation, Instance Masks, and Measuring Defect Area

Part V — Trust and Data Quality

• Ch 21 · Explainability: Saliency Maps, Grad-CAM, and Checking What the Model Sees
• Ch 22 · Robustness: Distribution Shift, Calibration, and Confidence Thresholds
• Ch 23 · Data-Centric Vision: Labeling, Label Noise, Class Imbalance, and Active Learning
• Ch 24 · Experiments and Reproducibility: Seeds, Checkpoints, Tracking, and Model Cards

Part VI — Deployment and Production

• Ch 25 · Exporting Models: torch.export, ONNX, and ONNX Runtime
• Ch 26 · Fast Inference on CPU and Edge Devices: Quantization, Batching, and Honest Benchmarks
• Ch 27 · Serving the Grader: A Prediction API and the Packing Line
• Ch 28 · Monitoring and Capstone: The Greenvale Grading System End to End

Appendices
A · Computer Vision Cheat Sheet
B · Templates and Reference
C · Production Checklist
D · Glossary

Book details