This book is licensed for the personal use of the buyer. Please do not share or resell the file. Short quotations with attribution are welcome.
The information in this book is provided for education. It is not legal, financial or other professional advice. Product and company names are used only to describe them and belong to their owners.
This is a free sample: the introduction and first chapter.
IntroductionIntroduction
For years, AI meant sending data to the cloud and waiting for an answer. That is changing. Phones, laptops, cars, cameras, watches and industrial sensors now contain dedicated AI accelerators, and models have become small and efficient enough to run on them. Speech recognition, photo enhancement, text prediction, translation, and even compact language models increasingly run directly on the device.
On-device AI offers privacy (data stays local), speed (no network round trip), offline operation and lower cloud costs. It also brings constraints: limited memory, compute, battery and storage, a huge variety of hardware, and the challenge of updating models across millions of devices.
This book is a practical guide for product managers, engineers and technical leaders deciding when and how to run AI on devices. It covers hardware, model compression, runtimes, application design, privacy, security, testing and operations, with worked examples.
How the book is organised
Chapters 1 to 10: what on-device AI is and when to use it.
Chapters 11 to 22: hardware and making models small.
Chapters 23 to 34: runtimes, formats and deployment.
Chapters 35 to 46: designing applications and common use cases.
Chapters 47 to 58: privacy, security, updates, testing and monitoring.
Chapters 59 to 66: worked examples and case studies.
Chapters 67 to 72: checklists, templates and a first-project plan.
Chapter 1What Runs Where
AI workloads can run:
in the cloud: large models, central data, constant connectivity;
on the edge: local servers or gateways near devices, such as in a factory or store;
on the device: phones, laptops, wearables, cameras, vehicles and sensors.
Many products combine them.
You have reached the end of the free sample. The full book continues with 71 more chapters, plus the appendices.