Compact molecular orbitals spotted in a kagome metal — confirming a key quantum theory

Rice University

What happens when electrons in a material effectively stop moving? Not because of temperature or pressure, but because the material's atomic geometry creates destructive interference that traps them in place?

This is the central puzzle of flat band quantum materials — solids where the electronic energy bands are perfectly level, meaning electrons have no incentive to flow in any particular direction. The bands are flat because the geometry of the crystal lattice causes electron waves to cancel each other out, pinning the electrons in localized patterns. These materials have captivated condensed matter physicists because when electrons can't move freely, even tiny interactions between them get amplified, producing exotic states of matter including superconductivity, magnetism, and quantum criticality.

But a basic question has lingered: what are the actual building blocks — the physical units — that give rise to these exotic states? A collaboration between Rice University and the Weizmann Institute of Science now has a direct answer, published in Nature Physics.

A theory built on traffic jams

Qimiao Si, the Harry C. and Olga K. Wiess Professor of Physics and Astronomy at Rice, had developed a theoretical framework — published earlier in Science Advances — predicting that flat band physics could be understood through compact molecular orbitals: tightly localized electron configurations that act as the fundamental agents of the flat bands.

Si's theory centered on the quantum critical point, a transition zone where a material hovers between two competing states. He uses a highway analogy: imagine two lanes of traffic, one jammed solid, the other flowing freely. Cars switching between lanes shift the balance. At a critical point, either lane could tip into either state. By studying the stuck lane — the compact molecular orbitals — Si believed he could learn about the free-flowing quantum states on the other side of the transition.

The theory was elegant. But as Si himself put it: it remained a hypothesis until proven by experiment.

An atomic-resolution look inside Ni3In

The experiment came together through a chance meeting. Si and Haim Beidenkopf, a professor at the Weizmann Institute, crossed paths at the Kavli Institute of Theoretical Physics at UC Santa Barbara. Beidenkopf specializes in imaging quantum materials at atomic resolution, and he was already running experiments on a flat band material. Their conversation revealed that his setup was ideally suited to test Si's predictions.

The material they chose was Ni3In, a kagome metal. Kagome refers to a specific lattice geometry — a pattern of interlocking triangles and hexagons, named after a traditional Japanese basket weave. This structure naturally produces flat bands because of how electron waves interfere across its repeating triangular units.

Ni3In was selected for practical reasons beyond its flat band structure. It's a strongly correlated metal, meaning its electrons interact intensely with one another, and understanding its electronic properties could offer insights into high-temperature superconductivity — one of the most sought-after goals in materials science.

Using an atomic-resolution spectrometer, Beidenkopf's team probed the spatial profile of electrical current flowing into and out of the Ni3In lattice. What they found matched Si's theoretical predictions: compact molecular orbitals, sitting right where the theory said they should be, with the spatial profile the theory had predicted.

From hypothesis to confirmed mechanism

The experimental confirmation has two layers. First, the team directly observed the compact molecular orbitals — confirming they exist as physical objects, not just mathematical constructs. Second, by applying Si's theoretical framework to the data, they identified the specific kagome structure responsible for the material's quantum critical state.

"This collaboration showed, experimentally, that compact molecular orbitals serve as the agents that underlie the highly agitated quantum critical state of matter," Si said.

Beidenkopf described the approach as combining atomic-scale spectroscopy with material-specific analytical modeling to reveal the kagome flat-band origin of the unusual quantum behavior. The spatial profile of the orbitals matched what the compact molecular orbital theory had predicted.

Topology enters the picture

There's a deeper layer to this work involving topology — a branch of mathematics concerned with properties that remain unchanged when a shape is continuously deformed. In flat band materials, topology imposes a global constraint on electron behavior. As Mounica Mahankali, a graduate student and co-first author, explained: when you trace through the space of electron states and return to the starting point, you acquire a nonzero winding number — a topological signature indicating the electrons' configuration has a twist that can't be smoothed away.

Si's earlier theoretical work asked how this topology affects correlation physics — how electrons interact and organize. The new experiment confirms that the compact molecular orbitals sit at the intersection of topology and strong electron correlations, making them a key ingredient for understanding both phenomena simultaneously.

What remains uncertain

The study demonstrates the mechanism in one material — Ni3In. Whether compact molecular orbitals play the same role in other kagome metals, or in flat band materials with different lattice geometries, is an open question. The kagome structure is one of several that can produce flat bands; whether this framework generalizes to pyrochlore lattices, Lieb lattices, or moire systems (like twisted bilayer graphene) requires separate investigation.

The connection to high-temperature superconductivity, while motivating, remains indirect. Ni3In itself is not a superconductor under ambient conditions. The insight is that understanding how compact molecular orbitals drive quantum criticality could inform the design of materials that are — but that step involves significant additional research.

The imaging technique, while powerful, provides a snapshot of the surface electronic structure. Bulk properties may differ, and extending these measurements to three-dimensional electronic behavior adds experimental complexity.

Still, the work closes a loop that began with pure theory: a predicted building block of flat band physics has been seen for the first time. The next question — one Si is clearly eager to pursue — is what can be built with it.