On this page
Phones that recognise faces and scanners that read printed text both run on computer vision. If you are an engineering student, here is how the field works, what to learn and what to build first.
What does a computer vision engineer actually do?
An IIT Madras course outline defines computer vision as developing algorithms and techniques to analyse and interpret the visible world. It lists applications from biometrics and medical diagnosis to document processing and surveillance. Day to day, the job involves four things:
collecting and labelling images
training a model
finding where the model fails, such as in poor light
making it fast enough for a phone, camera or robot
How do machines learn from images?
Stanford's CS231n notes describe images as arrays of integers from 0 to 255, of size width × height × 3 (one layer each for red, green and blue). Their 248 × 400 photo of a cat comes to 297,600 numbers.
A convolutional network reads those numbers with small filters. Each filter slides across the image and computes a dot product at every position, as the CS231n notes on convolutional networks explain. In the first layer, the network learns filters that respond to features such as edges.
Worked example: find an edge by hand
Take a 5 × 5 greyscale image in which every row reads 10, 10, 10, 200, 200: dark on the left, bright on the right. Use a 3 × 3 filter in which every row reads −1, 0, 1.
Place the filter on columns 1 to 3. Each row gives −10 + 0 + 10 = 0, so the total is 0.
Slide it to columns 2 to 4. Each row gives −10 + 0 + 200 = 190, so the total is 570.
Slide it to columns 3 to 5. Again each row gives 190, so the total is 570.
The output is 0, 570, 570: the large numbers mark where dark turns to bright. Training means the network adjusts the numbers inside many such filters until its answers on labelled images improve.
Krizhevsky, Sutskever and Hinton's network, with 60 million parameters, was trained on 1.2 million ImageNet images in 1,000 classes. A variant of it won the ILSVRC-2012 ImageNet competition with a top-5 test error of 15.3%, against 26.2% for the second-best entry.
What maths and coding do you need for computer vision?
The IIT Madras course outline expects basic linear algebra, 2D and 3D geometry, probability and statistics, signal processing and programming (Python, C/C++ or OpenCV). Its syllabus covers the Fourier transform, convolution, epipolar geometry, homography and PCA. For a student, that means:
Linear algebra: images are matrices, and filters are dot products.
Geometry: this is how a flat photo maps to a 3D world.
Probability and statistics: models output confidences, not certainties.
Calculus: not on that course's list, but derivatives tell training which way to nudge each number.
Python: you will also use libraries such as OpenCV, the open-source computer vision library.
If matrices or derivatives feel shaky, the Fermi Concept Library has every concept in mathematics and computing, each with its own lesson. For a structured course, NPTEL lists Deep Learning for Computer Vision from IIT Hyderabad.
Which beginner computer vision projects should you build first?
Follow this ten-week ladder.
Weeks 1-2, pixels: load a photo in Python, print its array shape, convert it to greyscale and code the edge filter above with plain loops, then compare with OpenCV.
Weeks 3-4, digits: train a classifier on MNIST, which has 60,000 training and 10,000 test images of handwritten digits, each 28 × 28 pixels.
Weeks 5-6, colour: work through the PyTorch CIFAR-10 tutorial, which trains a convolutional network on 32 × 32 colour images in 10 classes. Then list which classes your model confuses.
Weeks 7-10, your own data: photograph something local, such as leaves from three trees on your street. Label the photos, train a model and write up where it fails.
New to Python? Start with these first Python project ideas for school students. Put every project on GitHub with a short note on what went wrong.
Is computer vision a good career path for engineering students?
Your branch matters less than what you have built. You can reach computer vision from CSE, from the newer AI branches (see whether a B.Tech in AI and ML is worth it) or from ECE, through signal processing. Before you commit, ask yourself three questions:
Did the worked example feel like a puzzle or a chore?
Does your college offer an image processing elective, or a lab with cameras or robots?
Can you stay with a wrong model for a week while you find out why?
When a derivative problem keeps going wrong at the same step, the Fermi Problem Library has a step-by-step solver that explains the reasoning. Start this week: load one photo and find your first edge.
