ArtEmis: Affective language for visual art
2021-03-25
(Press-News.org) March 7, 2021, KAUST, Saudi Arabia - KAUST Assistant Professor of Computer Science Mohamed Elhoseiny has developed, in collaboration with Stanford University, CA, and École Polytechnique (LIX), France, a large-scale dataset to train AI to reproduce human emotions when presented with artwork.
The resulting paper, "ArtEmis: Affective Language for Visual Art," will be presented at the Conference on Computer Vision and Pattern Recognition (CVPR), the premier annual computer science conference, which will be held June 19-25, 2021.
Described as the "Affective Language for Visual Art," ArtEmis's user interface has seven emotional descriptions on average for each image, bringing the total count to over 439K emotional-explained attributions from humans on 81K pieces of art from WikiArt.
"Before this project, most machine learning models were based on factual description datasets," Elhoseiny explains. "For example, with 'a bird is perched on the chair,' Artemis expanded on the image description by requesting that people also add the emotions they felt when observing the artwork, which incorporated complex metaphoric language and abstract ideas," he adds.
The initial design was inspired by Northeastern University's, U.S., Distinguished Professor of Psychology Lisa Feldman Barrett, and is explained in her book "How Emotions Are Made: The Secret Life of the Brain." In her book, Barrett showed how stereotypical faces helped improve people's identification of constructed emotions. "We intentionally used emojis in our interface because Barrett's experiments proved that recognizing emotions is a challenging problem, even for human beings.", Elhoseiny adds. Data generated by ArtEmis enable the building of AI systems beyond the classical view of emotions that are currently adopted in affective AI industrial products based on facial expression recognition. Affective image description models based on ArtEmis-like data may help people to have a more positive experience by connecting better to artworks and appreciating them. In line with Barret's view, this may also open the door to using affective AI to alleviate mental health problems.
The researchers then carried out human studies to show the unique features of the ArtEmis dataset. For example, ArtEmis requires more emotional and cognitive maturity compared with well-established vision and language datasets. The research was also validated via a user study where participants were asked whether the descriptions were pertinent to the associated artwork.
"But we did not stop there. To show the potential of affective neural speakers, we also trained image captioning models in both grounded and nongrounded versions on our ArtEmis dataset. The Turing Test showed that generated descriptions closely resemble human ones," says Elhoseiny.
ArtEmis started while Dr. Elhoseiny was a visiting professor at Stanford University with Prof. Guibas. In collaboration with Stanford's Paul Pigott, professor of computer science and one of the leading authorities in Computer vision and Graphics, Elhoseiny co-build a large-scale art and language dataset as a partnership project with Panos Achlioptas, a Stanford Ph.D. student of Prof. Guibas, who adopted the proposal and made significant efforts in making this project a solid reality. The project implementation was also supported by Kilich Hydarov, an M.S./Ph.D. candidate from the KAUST Vision-CAIR group. The collaboration also benefited from the expertise of LIX Ecole Polytechnique's Maks Ovsjanikov, professor of computer Science and one of the leading graphics and vision researchers.
"Our dataset is novel as it concerns an underexplored problem in computer vision: the formation of emo-linguistic explanations grounded on visuals. Specifically, ArtEmis exposes moods, feelings, personal attitudes and abstract concepts, such as freedom or love, induced by a wide range of complex visual stimuli," concludes Elhoseiny.
The dataset can be accessed at https://www.artemisdataset.org/.
King Abdullah University of Science and Technology (KAUST) advances science and technology through distinctive and collaborative research integrated with graduate education. Located on the Red Sea coast in Saudi Arabia, KAUST conducts curiosity-driven and goal-oriented research to address global challenges related to food, water, energy, and the environment.
Established in 2009, KAUST is a catalyst for innovation, economic development and social prosperity in Saudi Arabia and the world. The University currently educates and trains master's and doctoral students, supported by an academic community of faculty members, postdoctoral fellows and research scientists.
With over 100 nationalities working and living at KAUST, the University brings together people and ideas from all over the world.
Visit kaust.edu.sa
INFORMATION:
ELSE PRESS RELEASES FROM THIS DATE:
2021-03-25
While urban agriculture can play a role in supporting food supply chains for many major American cities -- contributing to food diversity, sustainability and localizing food systems -- it is unrealistic to expect rooftop gardens, community plots and the like to provide the majority of nutrition for the population of a metropolis.
That's the conclusion of a team of researchers who analyzed the nutritional needs of the population of Chicago and calculated how much food could be produced in the city by maximizing urban agriculture, and how much crop land would be needed adjacent to the city to grow the rest. The study was the first to evaluate land required to meet ...
2021-03-25
Scientists have shown that SARS-CoV-2, the virus that causes COVID-19, can infect specific cells in the salivary gland in the mouth. The study by researchers from the Wellcome Sanger Institute, National Institutes of Health and the University of North Carolina at Chapel Hill, and their collaborators within the Human Cell Atlas Oral & Craniofacial Network, also discovered that live cells from the mouth were found in saliva, and that the virus was able to reproduce within these infected cells.
The study revealed that salivary gland cells could play a role in transmission ...
2021-03-25
Göttingen, March 25, 2021. Testing and vaccination - these are the pillars on which humanity is trying to get a grip on the Coronavirus pandemic. Although it is taking longer than many had expected, it is believed that it is only a matter of time before we are all vaccinated and thus protected. However, time is also working for the virus, which has now mutated several times, with variants B.1.1.7 from the United Kingdom, B.1.351 from South Africa and P.1 from Brazil spreading rapidly. These viruses have mutations in the so-called spike protein, the structure on the surface of the virus that is responsible for attachment to host cells. At the same time, the spike protein is also the major target of the immune response. Antibodies generated in response to SARS-CoV-2 infection or vaccination ...
2021-03-25
Supersized alcopops are ready-to-drink flavored alcoholic beverages with high alcohol content that are disproportionately consumed by underage drinkers. There can be up to 5.5 standard alcoholic drinks in a single 24 ounce can, so consuming only one can of supersized alcopop is considered binge drinking, and consuming two cans can cause alcohol poisoning. Still, these products remain under-regulated and are available inexpensively at gas stations and convenience stores, where they are more readily accessible by underage youth.
New research led by George Mason University's College of Health and Human Services found that nearly ...
2021-03-25
The burial field in Valsgärde outside Uppsala in central Sweden contains more than 90 graves from the Iron Age.
"On a light note, we could say that Valsgärde is Scandinavia's answer to Sutton Hoo in England as portrayed in the film The Dig on Netflix," says Birgitta Berglund, professor emeritus of archaeology at the Norwegian University of Science and Technology's NTNU University Museum.
Valsgärde is especially known for its spectacular boat graves from the 600s and 700s CE. This timeframe is in the middle of what Norway calls the Merovingian period, the era just before the Viking Age.
Two of these spectacular boat graves are at the centre of this story -- or more specifically, the story is really about the down bedding that was found ...
2021-03-25
Oncotarget published "Phase 1 study of Z-Endoxifen in patients with advanced gynecologic, desmoid, and hormone receptor-positive solid tumors" which reported that Z-endoxifen administration was anticipated to bypass these variations, increasing active drug levels, and potentially benefiting patients responding sub-optimally to tamoxifen.
Patients with treatment-refractory gynecologic malignancies, desmoid tumors, or hormone receptor-positive solid tumors took oral Z-endoxifen daily with a 3 3 phase 1 dose escalation format over 8 dose levels.
Three patients had partial responses and 8 had prolonged stable disease; 44.4% of patients at dose levels 6–8 achieved one of these ...
2021-03-25
UPTON, NY — Inspired by the mastery of artificial intelligence (AI) over games like Go and Super Mario, scientists at the National Synchrotron Light Source II (NSLS-II) trained an AI agent – an autonomous computational program that observes and acts – how to conduct research experiments at superhuman levels by using the same approach. The Brookhaven team published their findings in the journal Machine Learning: Science and Technology and implemented the AI agent as part of the research capabilities at NSLS-II.
As a U.S. Department of Energy (DOE) Office of Science User Facility located at DOE’s Brookhaven National Laboratory, NSLS-II enables scientific studies by more than 2000 researchers each year, ...
2021-03-25
Akito Kawahara was snapping pictures at a scenic outlook in Hawaii when he spotted the moth equivalent of a dodo.
An entomologist, Kawahara recognized the squiggly patterns on nearby plants as trails carved by leaf-mining caterpillars and lowered his camera to take a closer look. To his astonishment, he saw a tiny moth most experts assumed was extinct. It belonged to a genus known as Philodoria, a type of moth found only in Hawaii and one that hadn’t been documented in the wild since 1976.
“I thought, ‘Oh my God, there’s a Philodoria ...
2021-03-25
New research shows that consumers judge 'activist brands' based on how morally competent they are perceived to be when challenging free speech.
The report, co-authored by experts at the Business School (formerly Cass), Birkbeck, University of London and the University of Sussex Business School explains that stakeholders draw their conclusions on the biggest brands by measuring three moral skills: sensitivity, vision, and integration.
Lacking these traits, a brand raising controversy is judged as transgressing, reproducing and manipulating the boundaries of free speech. Displaying these traits proves the brand is not merely 'woke-washing' ...
2021-03-25
HOUSTON, TX - March 25, 2021 - Humans are born with tens of thousands of hematopoietic stem cells (HSCs) that collectively ensure lifelong production of blood and immune cells that protect us from infections. HSCs can either duplicate to produce more stem cell progeny or differentiate to produce distinct immune cell lineages, an extremely critical decision that ensures that the body achieves the fine balance between having enough immune cells to fight invaders while still retaining enough HSCs to maintain future blood production. As we age, HSCs accumulate mutations that lead to the ...
LAST 30 PRESS RELEASES:
[Press-News.org] ArtEmis: Affective language for visual art
