Face value

Machines are learning to identify individual fish out of thousands.

If you believe all salmon have the same facial expression, then it could be time to think again, according to new research.
A recent study by SINTEF, one of Europe’s largest independent research organisations, and the Norwegian University of Science and Technology (NTNU), suggests it is possible to distinguish one face from another.
They are basing their results on artificial intelligence (AI) technology which has helped them carry out much of the work. By adapting the type of existing AI technology that recognises human faces, the researchers believe they can learn more about farmed fish and how they feel at a particular time.
Picking out 100 random salmon from a cage that contained more than 100,000 individuals, the identification success rate was just over 96, which is extremely high.
NTNU research fellow and SINTEF researcher Bjørn Magnus Mathisen says it is difficult to explain why the technology has been so accurate, but he has been working with machine learning and artificial intelligence for 10 years and it seems to work.
He said: “We are not sure how they actually recognise the salmon, but we have a theory it is through the pigment spots on the face. They have a distinctive pigment, in the same way as with the cheetah or giraffe.”
One of the things Mathisen found particularly interesting about the research is whether they can identify the same salmon throughout their life cycle. He said: “I am really looking forward to testing if this works on smolts and see if the machine is able to recognize the fish as it gets bigger.
What adds a little to the mystery is that machines can also see things that humans do not, he suggests.
In order to study the images of salmon taken by underwater cameras Mathisen uses a type of machine called a deep neural network which is modelled on the way cells in the brain are organised.
A SINTEF report on his work says these neural networks are able to identify animals, people and objects through sound and images in a way that was previously difficult to do mathematically.
SINTEF says: “You cannot tell a machine how to see the difference in each fish. Like us humans, it must learn by itself.
Mathisen adds: “Methodologically, machines learn a bit in the same way as humans. We learn by seeing differences.”
The research is also being supported by SINTEF’s aquaculture innovation centre, SFI Exposed, and was started as a master’s project by Espen Meidell and Edvard Schreiner Sjøblom, supervised by Kerstin Bach, Håkon Måløy and Mathisen. To train the software to identify fish, the group had to carry out significant manual work first.
The researchers were sent a video file with thousands of pictures of the salmon in a cage.
Then the task of marking the fish heads by hand began, with 500 salmon heads manually identified and stored in a database. This collection taught the neural network to cut salmon heads from the images themselves, and in a short time the network had done the same job thousands of times. The work also gave the team a new and larger database which was used to train new neural networks to recognise each individual salmon throughout the cage.
Facial recognition is now becoming increasingly common in human commerce, but its use for applications like law enforcement has led to intense ethical debates. Anders Bryhni, a business developer at SINTEF says: “By using facial recognition on fish we of course avoid such privacy-related issues. We are like everyone else, concerned with making good ethical assessment when we build a system based on artificial intelligence.
“At the same time, we want the business community in Norway to take advantage of the great opportunities that lie in technology as quickly as possible.”
Mathisen says the system has a number of benefits for aquaculture: “By learning more about each individual salmon, we get to know more about what makes them sick or why they are healthy and why they are happy or sad. The technology makes it possible to know with confidence about how an individual salmon feels at a particular time.
He adds: “I’m pretty sure this is going to be a goldmine for biologists. By following individuals through life we can find even more about the eating habits of fish, their social hierarchies, general welfare and their tendency to attract lice.
“Also we do not have to take random samples by manual methods which are not only expensive and inaccurate, but are also harmful to the fish.
“It can also create a new business model because if we know the life cycle of the fish it may be possible to differentiate the price of a fillet based on how a salmon lived.”

Bjorn Mathisen