(Press-News.org) Researchers from North Carolina State University have developed a new state-of-the-art method for controlling how artificial intelligence (AI) systems create images. The work has applications for fields from autonomous robotics to AI training.
At issue is a type of AI task called conditional image generation, in which AI systems create images that meet a specific set of conditions. For example, a system could be trained to create original images of cats or dogs, depending on which animal the user requested. More recent techniques have built on this to incorporate conditions regarding an image layout. This allows users to specify which types of objects they want to appear in particular places on the screen. For example, the sky might go in one box, a tree might be in another box, a stream might be in a separate box, and so on.
The new work builds on those techniques to give users more control over the resulting images, and to retain certain characteristics across a series of images.
"Our approach is highly reconfigurable," says Tianfu Wu, co-author of a paper on the work and an assistant professor of computer engineering at NC State. "Like previous approaches, ours allows users to have the system generate an image based on a specific set of conditions. But ours also allows you to retain that image and add to it. For example, users could have the AI create a mountain scene. The users could then have the system add skiers to that scene."
In addition, the new approach allows users to have the AI manipulate specific elements so that they are identifiably the same, but have moved or changed in some way. For example, the AI might create a series of images showing skiers turn toward the viewer as they move across the landscape.
"One application for this would be to help autonomous robots 'imagine' what the end result might look like before they begin a given task," Wu says. "You could also use the system to generate images for AI training. So, instead of compiling images from external sources, you could use this system to create images for training other AI systems."
The researchers tested their new approach using the COCO-Stuff dataset and the Visual Genome dataset. Based on standard measures of image quality, the new approach outperformed the previous state-of-the-art image creation techniques.
"Our next step is to see if we can extend this work to video and three-dimensional images," Wu says.
Training for the new approach requires a fair amount of computational power; the researchers used a 4-GPU workstation. However, deploying the system is less computationally expensive.
"We found that one GPU gives you almost real-time speed," Wu says.
"In addition to our paper, we've made our source code for this approach available on GitHub. That said, we're always open to collaborating with industry partners."
INFORMATION:
The paper, "Learning Layout and Style Reconfigurable GANs for Controllable Image Synthesis," is published in the journal IEEE Transactions on Pattern Analysis and Machine Intelligence. First author of the paper is Wei Sun, a recent Ph.D. graduate from NC State.
The work was supported by the National Science Foundation, under grants 1909644, 1822477, 2024688 and 2013451; by the U.S. Army Research Office, under grant W911NF1810295; and by the Administration for Community Living, under grant 90IFDV0017-01-00.
Amsterdam, June 1, 2021 - The Treatabolome project is a research initiative to develop a freely available, interoperable online platform dedicated to disseminating rare disease and gene-specific treatment information to healthcare professionals regardless of their level of specialized expertise. Developed under the Solve-RD European Research Project, it is intended to reduce treatment delays for patients with rare diseases by directly linking diagnosis and treatment information. This initiative is highly relevant to neuromuscular disorders as they are rare diseases by definition. In this special issue of the Journal of Neuromuscular Diseases, experts contribute Treatabolome-feeding systematic literature reviews on rare neurological ...
Neutrons - Space ice, un-Earthly cold
Researchers from NASA's Jet Propulsion Laboratory and Oak Ridge National Laboratory successfully created amorphous ice, similar to ice in interstellar space and on icy worlds in our solar system. They documented that its disordered atomic behavior is unlike any ice on Earth.
The findings could help interpret data from future NASA missions such as Europa Clipper, which will assess the habitability of Jupiter's moon, Europa.
Using the Spallation Neutron Source's SNAP instrument, the scientists replicated the cold vacuum of space and added a few molecules ...
Single atomic defect is the smallest structural unit of solid material. The construction of devices based on single defect can reach the limit of miniaturization of semiconductor devices. In the past decades, the creation and manipulation of single defects in semiconductors opened a new research field, and could be used to physically realize "qubits" of solid-state quantum computation through spin or electron charge. Most interest have focused on the studies of spin quantum computing. However, the spin manipulation need an optical and magnetic field. On the contrary, multiple ...
Leipzig. The smoke from the extreme forest fires on the US West Coast in September 2020 travelled over many thousands of kilometres to Central Europe, where it continued to affect the atmosphere for days afterwards. A comparison of ground and satellite measurements now shows: The forest fire aerosol disturbed the free troposphere over Leipzig in Germany as never before. An evaluation by an international research team led by the Leibniz Institute for Tropospheric Research (TROPOS) revealed an extraordinary optical thickness on 11 September 2020, which attenuated ...
Oncotarget published "Development of a ghrelin receptor inverse agonist for positron emission tomograph" which reported that imaging of Ghrelin receptors in vivo provides unique potential to gain deeper understanding on Ghrelin and its receptors in health and disease, in particular, in cancer.
Ghrelin, an octanoylated 28-mer peptide hormone, activates the constitutively active growth hormone secretagogue receptor type 1a with nanomolar activity.
The authors developed novel compounds, derived from the potent inverse agonist K- -FwLL-NH2 but structurally varied by lysine conjugation ...
Oncotarget published "Global DNA hypermethylation pattern and unique gene expression signature in liver cancer from patients with Indigenous American ancestry" which reported that contrasting with this pattern, the age structure of HCC in Andean people displays a bimodal distribution with half of the patients developing HCC in adolescence and early adulthood.
To deepen the understanding of the molecular determinants of the disease in this population, the authors conducted an integrative analysis of gene expression and DNA methylation in HCC developed by 74 Peruvian patients, including 39 adolescents and young adults.
While genome-wide hypomethylation ...
New research from the Prevention Research Center of the Pacific Institute for Research and Evaluation, The Ohio State University, and San Jose State University finds that during COVID-19 stay-at-home orders, parental stress was higher during the workday compared to after the workday and lower during weekends than during weekdays.
Previous research compares parental stress before and during the pandemic, but none has measured it during stay-at-home orders. In this study, scientists assessed how time-varying and day-varying factors are related to parents' level of stress. In specific, stress was examined 3 times a day for 14 days for survey participants in Ohio from April to May 2020.
Specific findings include:
Parents ...
University of Queensland scientists working to unlock the mysteries Australia's deadly stonefish have made a discovery which could change how sting victims are treated in the future.
Stonefish are the most venomous fish in world and are found throughout shallow coastal waters of the northern half of Australia.
Study co-author Associate Professor Bryan Fry said previous studies have not been able to uncover all of the mechanisms at play in stonefish venom because of the way the venom was tested.
"There's a couple of reasons previous studies haven't been able to thoroughly decipher the toxicological mysteries of stonefish venom," Dr Fry said.
"But ...
Oncotarget published "Multi-modal effects of 1B3, a novel synthetic miR-193a-3p mimic, support strong potential for therapeutic intervention in oncology" which reported that the authors comprehensively investigated miRNA-193a-3p's mode of action in a panel of human cancer cell lines, with a variety of genetic backgrounds, using 1B3, a synthetic microRNA mimic.
Interestingly, the exact mechanism through which 1B3 reduced cell proliferation varied between cell lines.
1B3 efficiently reduced target gene expression, leading to reduced cell proliferation/survival, cell cycle arrest, induction of apoptosis, increased cell senescence, DNA damage, and inhibition of migration.
SiRNA ...
A local research team, comprised of members of the Korea Institute of Machinery and Materials(KIMM) under the Ministry of Science and ICT and UNIST, developed a metamaterial absorber that significantly enhances the detection of harmful substances or biomolecules, and published their results in Small Methods.
The joint research team led by Principal Researcher Dr. Joo-Yun Jung of the Nano-Convergence Mechanical Systems Research Division at KIMM and Professor Jongwon Lee of UNIST developed a metamaterial that enhances infrared absorption spectroscopy through 100-fold amplification of detection signals. The proposed metamaterial is a special functional material with vertical nanogaps of a smaller size than infrared wavelength.
Infrared spectroscopy is a technique that identifies ...
