Scientists at MIT’s Computer Science and Artificial Intelligence Laboratory, (CSAIL) say that a Photo App will soon make pics more memorable by predicting how easily forgotten or likely to be and making suggestions for how to improve them. Their algorithm could be used to improve advert content, among other uses. MenNet has yet to be created. Aditya Khosla, a graduate student and her colleagues created the algorithm called “MemNet” that can predict how memorable a photo will be. The algorithm will be transformed into an app which subtly alters the photo to increase its recall. The algorithm creates heat maps for each image that identify the parts that are most memorable. Upload your photos online to see the results. You can view the memorability maps in the jet color scheme. It ranges from blue to red (lowest up to highest). Independently normalized, the memorability maps lie between 0 and 1. Each image’s memorability score is indicated by the numbers shown in white. (Image: people.csail.mit.edu) Mr. Khosla said: “Understanding memorability can help us make systems to capture the most important information, or, conversely, to store information that humans will most likely forget. This is like having an immediate focus group that will tell you the likelihood that someone will recall a visual message.” There are many uses of algorithm. The researchers believe they have a variety of possible applications. They can improve ad content and social media posts as well as develop more effective teaching materials. The largest global image-memorability database – LaMem – was also released by the researchers as part of this project. The database contains 60,000 images annotated in detail with metadata about qualities like popularity and emotional impact. LaMem will trigger more studies According to the team, LaMem represents their attempt to stimulate further research in an area they believe has been understudied. Akhil Raju (also a graduate student in CSAIL), Prof. Antonio Torralba (CSAIL), and Aude Oliva (principal research scientist at MIT), co-authored the paper. The paper was presented by Mr. Khosla at the International Conference on Computer Vision at Santiago’s Convention Center. Aditya Khosla, a fifth-year computer science PhD student at MIT. He is a member of CSAIL’s Computer Vision Group, where he receives advice from Professor Antonio Torralba and collaborates frequently with Professor Aude Oliva. What is the working principle of this algorithm? A similar algorithm was previously developed by the team for facial memorability. The new algorithm is different because it employs techniques from “deep learning”, an AI field. These systems are called “neural networks” and teach computers how to sort through large amounts of data without human assistance. Apple’s Siri is an integrated ‘intelligent advisor’ which allows users to speak natural language commands to control their mobile phones and apps. Google’s autocomplete and Facebook’s photo tag use similar techniques. Google, Apple and Facebook have invested hundreds of millions in deep-learning startups. Professor Oliva stated that deep-learning can predict human memories. “While it has made great progress in scene recognition and object recognition, prediction of human memory is often viewed as an advanced cognitive task that computer scientists won’t be able tackle. We can and did! Neural networks are able to link information without human supervision. Each layer performs random computations of the data. The network adjusts as more data is collected to make more precise predictions. Researchers fed the algorithm a variety of datasets that included LaMem and scene-oriented SUN, as well as Places and Places. Every image was assigned a “memorability score” based on how well humans can remember it in online tests. Algorithm versus human subjects Professor Oliva and his colleagues set the algorithm up against human volunteers. The model was asked to determine how easily a group would remember a newly-discovered photograph. The algorithm was 30% better than other algorithms and performed just a fraction below human average performance. The algorithm generates heat maps for each image that highlight the most interesting parts. They can enhance an image’s recallability by highlighting certain regions. Alexei Efros is an associate professor of computer sciences at University of California at Berkeley. He said that although researchers at CSAIL (MIT’s Computer Science and Artificial Intelligence Laboratory), have performed such manipulations on faces, it was impressive to see how they were able extend the technique to general images. Although you can alter the look of a face, such as making it smiley or more expressive, it’s much harder to make it generalize for all images. The unexpected results of the study provide insight into human memory. According to Mr. Hosla, he was curious about whether volunteers could remember all the pictures if only they were presented with the most significant. According to Mr. Khosla, “You would expect people to forget the same things they used to remember,” but research shows otherwise. It means we can potentially increase people’s memories if they present them with memorable pictures.” Scientists said that the new system will be updated to predict the individual’s recall and target the system for specific industries such as retail clothing design or logo. Professor Efros stated that visual data is something we are interested in. “This type of research allows us to better understand the visual information people pay attention too.” Marketers, movie-makers, and other content creators will find it exciting to be able to predict your mental state while you are looking at something. This project received grants from the National Science Foundation and the McGovern Institute Brain Technology Program. Nvidia donated hardware. Citation: “Understanding Image Memorability and Predicting It at Large Scale,” Aditya Kohla, Akhil Raju, Antonio Torralba and Aude Oliva. International Conference on Computer Vision, 2015. Related article: Native vs. “Native vs. Hybrid App Development: What is the Difference?”
