Lmod: The 'secret sauce' behind module management at TACC

Environmental module system lets scientists mix and match software safely and with ease

2012-11-14

(Press-News.org) It's likely that fellow users on your favorite supercomputer have widely varying needs. The applications, compilers, and libraries that you need are probably different from the ones other users need. That is where modern environment module systems come in. A good module system "sets the table " for users by loading the packages each user needs.

The Texas Advanced Computing Center (TACC) has developed an innovative module system that addresses some unique challenges facing modern computing centers.

Life in the world of computing has become much more complicated since John Furlani first introduced the concept of modules in 1991. A few years ago, system administrators at TACC noticed a recurring problem. The existing module system was not preventing researchers from making mistakes. These mistakes were leading to extra work and frustrations for both the scientists and the TACC staff that supported them.

Incompatible packages, the work of manually disabling or re-enabling modules, and having to choose from multiple versions of tools was the norm for users of TACC's Ranger or Lonestar supercomputers. In addition, a parallel application or library usually depends on both the compiler and MPI (Message Passing Interface) stack. At TACC, with three major compiler families and three MPI families, this means that as many as nine versions of a given package might be available. Only one version will run properly and give the correct answer when paired with complementary tools.

In 2009, Robert McLay, research associate at TACC, decided to address some of the recurrent challenges and remedy this situation by creating Lmod, a complete rewrite of the Environment Module system.

"Lmod protects our users so they can't load mismatched compilers, libraries, and other parts of the software stack," McLay said.

McLay presented the new tool at Supercomputing '11 in Seattle, as well as the 2011 IEEE Conference in Austin. He has worked with staff at many leading universities and computing centers to install the tool broadly. Lmod is freely available via Sourceforge and is already being used at computing centers across the country, including those at the National Center for Atmospheric Research, the Ohio Supercomputing Center, the University of Florida and the University of TromØ in Norway.

"I advocated the use of Lmod over the alternative implementations because of its flexibility and full access to the power provided by the module files," said Oleksandr Moskalenko, Bioinformatics Specialist at the University of Florida and an early adopter of Lmod. "As we upgraded our cluster from CentOS 5 to RHEL 6 last Summer we made Lmod the default and the only supported way for making the software we provide available to the users"

While Lmod helps prevent users from making mistakes in building their programs and setting up their computations, it also simplifies the work of systems administrators, who previously received several requests for assistance each week from users unsure of which version of a tool to use.

Lmod allows administrators to encode dependencies into the modules themselves.

Only valid options are available to users. Lmod disables the "wrong" choices, and informs users about precisely which versions of software are the right ones to use. But it goes even further: suppose you have gcc (a popular compiler system that supports various programming languages) loaded, along with a compatible MPI stack and a parallel solver. If you change the compiler, Lmod will automatically replace the MPI stack and other parallel libraries with compatible versions. The user cannot load a module that is not present and the system ensures loaded modules remain compatible with each other.

"This capability is part of our secret sauce," McLay said. "It's part of what makes TACC systems easy to use. It has reduced software selection errors to nearly zero."

The tool is helpful for beginners and expert users. Every aspect of the tool can be optimized by experienced command-line users to suit their needs. It also provides some behind-the-scenes advantages to administrators. For one, it allows them to track the usage of modules and mine this data to determine which packages are being used, by whom, and how.

This information is useful to centers like TACC as they determine which software to install, upgrade or optimize.

"The metrics built into Lmod have allowed us to track adoption of new biocomputing-oriented software modules and to monitor the effectiveness of our outreach and training strategy," said Matthew Vaughn, director of TACC's Life Sciences group. "As we begin to transition to the next generation of supercomputers powered by extremely parallel microprocessors, this has allowed us to focus our efforts in software optimization for those chips on the most popular pieces of software."

In January 2013, TACC will deploy Stampede, its newest supercomputer and the first large-scale deployment of the Intel Xeon Phi coprocessors. McLay is already developing modules for Lmod that are Xeon Phi-aware, with color coded properties that show a user which packages can be run on the coprocessor.

As high performance computing systems grow larger and more heterogeneous, the need for simplifying tools grows in importance.

Said Moskalenko, "We hope to see TACC continue to develop and improve Lmod for years to come as the computing environments become more complex on the back-end and the need for making it easier for our clients to access and use the provided software grows."

For McLay, the reason to use the tool is obvious: "If you're not using Lmod, you're working too hard."

INFORMATION:

[Attachments] See images for this press release:

ELSE PRESS RELEASES FROM THIS DATE:

Rare parasitic fungi could have anti-flammatory benefits

2012-11-14

Caterpillar fungi (Cordyceps) are rare parasites found on hibernating caterpillars in the mountains of Tibet. For centuries they have been highly prized as a traditional Chinese medicine - just a small amount can fetch hundreds of pounds. Scientists at The University of Nottingham have been studying how this fungus could work by studying cordycepin, one of the drugs found in these mushrooms. They have already discovered that cordycepin has potential as a cancer drug. Their new work indicates that it could also have anti-inflammatory characteristics with the potential ...

In financial ecosystems, big banks trample economic habitats and spread fiscal disease

2012-11-14

Like the impact of an elephant herd grazing on grassland, multinational banks shape the financial environment to an extent that far outweighs their small number. And like a contagious person on a transnational flight, when these giant, interconnected banks succumb to financial ills, they are uniquely positioned to infect wide swaths of the financial system. Researchers from Princeton University, the Bank of England and the University of Oxford applied methods inspired by ecosystem stability and contagion models to banking meltdowns and found that large national and international ...

Tolerance to malaria by means of iron control

2012-11-14

Malaria is a life-threatening condition that exposes approximately half of the world's population to the risk of developing a severe and often lethal form of disease. In a study published in the latest issue of the journal Cell Host & Microbe*, Miguel Soares and his team at Instituto Gulbenkian de Ciência (IGC), Portugal, discovered that the development of severe forms of malaria can be prevented by a simple mechanism that controls the accumulation of iron in tissues of the infected host. They found that expression of a gene that neutralizes iron inside cells, named H Ferritin, ...

Want better employees? Get somebody else to rate their personalities, suggests new study

2012-11-14

Toronto – Businesses will get more accurate assessments of potential and current employees if they do away with self-rated personality tests and ask those being assessed to find someone else to rate them, suggest results from a new study. Previous job performance studies have shown that outsiders are best at rating an individual's personality in terms of how they work on the job. But observers in these studies have always been co-workers. The recent paper by Prof. Brian Connelly, who is cross-appointed to the University of Toronto's Rotman School of Management and ...

Brain waves make waves

2012-11-14

This press release is available in German. Naturally, our brain activity waxes and wanes. When listening, this oscillation synchronizes to the sounds we are hearing. Researchers at the Max Planck Institute for Human Cognitive and Brain Sciences have found that this influences the way we listen. Hearing abilities also oscillate and depend on the exact timing of one's brain rhythms. This discovery that sound, brain, and behaviour are so intimately coupled will help us to learn more about listening abilities in hearing loss. Our world is full of cyclic phenomena: For ...

USC scientists 'clone' carbon nanotubes to unlock their potential for use in electronics

2012-11-14

The heart of the computer industry is known as "Silicon Valley" for a reason. Integrated circuit computer chips have been made from silicon since computing's infancy in the 1960s. Now, thanks to a team of USC researchers, carbon nanotubes may emerge as a contender to silicon's throne. Scientists and industry experts have long speculated that carbon nanotube transistors would one day replace their silicon predecessors. In 1998, Delft University built the world's first carbon nanotube transistors – carbon nanotubes have the potential to be far smaller, faster, and consume ...

Teenagers' brains affected by preterm birth

2012-11-14

New research at the University of Adelaide has demonstrated that teenagers born prematurely may suffer brain development problems that directly affect their memory and learning abilities. The research, conducted by Dr Julia Pitcher and Dr Michael Ridding from the University of Adelaide's Robinson Institute, shows reduced 'plasticity' in the brains of teenagers who were born preterm (at or before 37 weeks gestation). The results of the research are published today in the Journal of Neuroscience. "Plasticity in the brain is vital for learning and memory throughout life," ...

Life-saving role of heart attack centers confirmed in new study

2012-11-14

Recent studies questioning the role of specialist heart attack centres produced misleading results because doctors tend to send the sickest patients to have the best care, according to new research. Many heart attack patients in the UK are sent to a specialist centre for primary angioplasty - a surgical procedure to reopen the blocked artery. Randomised trials have found that angioplasty is much more successful than drug treatment alone, but research based on "real-world" data suggest that patients given an angioplasty don't tend to do better. Now researchers at Imperial ...

Hormone affects distance men keep from unknown women they find attractive

2012-11-14

Washington, DC — Men in committed relationships choose to keep a greater distance between themselves and an unknown woman they find attractive when given the hormone oxytocin, according to new research in the November 14 issue of The Journal of Neuroscience. The findings suggest oxytocin may help promote fidelity within monogamous relationships. Oxytocin plays a vital role in triggering childbirth and facilitating nursing. The hormone, which is produced in a region of the brain called the hypothalamus, is also involved in the formation of social bonds. In humans and other ...

Western media coverage of female genital surgeries in Africa called 'hyperbolic' and 'one sided'

2012-11-14

Despite widespread condemnation of female genital surgeries as a form of mutilation and a violation of human rights, an international advisory group argues that the practice is poorly understood and unfairly characterized. In a public policy statement in the Hastings Center Report, the Public Policy Advisory Network on Female Genital Surgeries in Africa, a group that includes doctors, anthropologists, legal scholars, and feminists, argues that media coverage of the practice is hyperbolic and one sided, "painting the now familiar portrait of African female genital surgeries ...

Lmod: The 'secret sauce' behind module management at TACC

ELSE PRESS RELEASES FROM THIS DATE:

LAST 30 PRESS RELEASES: