Images as data

An image is a grid of numbers. A colour photograph is three grids, one each for red, green and blue, with each number giving the intensity at that point. Everything in computer vision operates on those numbers.

The tasks, which are distinct and worth separating.

Classification: what is this an image of. One label for the whole image.

Detection: what objects are present and where, returning boxes.

Segmentation: which pixels belong to which object, which is more precise and more expensive.

Recognition: which specific individual is this, as opposed to what kind of thing it is. Face recognition is this, and it is the task with the heaviest consequences.

Optical character recognition: reading text in an image, which is a distinct and mature area.

Where it is used in practical business. Reading documents and forms. Counting things. Quality inspection on a production line. Reading number plates. Checking that a photograph contains what it should. Sorting and tagging large image collections.

What is difficult. Poor light, unusual angles, occlusion, things the model has not seen before, and anything requiring an understanding of the scene rather than recognition of objects in it.

And a practical note for this market. Almost all of this runs on photographs taken by people on phones in ordinary conditions, which is far harder than the clean datasets on which models are usually demonstrated.