Compact camera uses 25 color channels for high-speed, high-definition hyperspectral video

A traditional digital camera splits an image into three channels—red, green and blue—mirroring how the human eye perceives color. But those are just three discrete points along a continuous spectrum of wavelengths. Specialized “spectral” cameras go further by sequentially capturing dozens, or even hundreds, of these divisions across the spectrum.

This process is slow, however, meaning that hyperspectral cameras can only take still images, or videos with very low frame rates, or frames per second (fps). But what if a high-fps video camera could capture dozens of wavelengths at once, revealing details invisible to the naked eye?

Now, researchers at the University of Utah’s John and Marcia Price College of Engineering have developed a new way of taking a high-definition snapshot that encodes spectral data into images, much like a traditional camera encodes color. Instead of a filter that divides light into three color channels, their specialized filter divides it into 25. Each pixel stores compressed spectral information along with its spatial information, which computer algorithms can later reconstruct into a “cube” of 25 separate images—each representing a distinct slice of the visible spectrum.

This instantaneous encoding enables the researchers’ camera system—small enough to fit into a cellphone—to take high-definition video, and the compressed nature of the component images opens up new real-world applications.

A study demonstrating the camera was led by Research Assistant Professor Apratim Majumder and Professor Rajesh Menon, both in the Department of Electrical & Computer Engineering. The results are reported in the journal Optica.

The camera’s design represents a leap forward in how spectral data can be captured.

“We introduce a compact camera that captures both color and fine spectral details in a single snapshot, producing a ‘spectral fingerprint’ for every pixel,” Menon said.

Hyperspectral cameras have long been used in agriculture, astronomy and medicine, where subtle differences in color can make a big difference. But these cameras have historically been bulky, expensive and limited to still images.

“When we started out on this research, our intention was to demonstrate a compact, fast, megapixel resolution hyperspectral camera, able to record highly compressed spatial-spectral information from scenes at video-rates, which did not exist,” Majumder said.

The Utah team’s breakthrough lies in how it captures and processes the data. The key component is a diffractive element that is placed directly over the camera’s sensor. It’s the element’s repeating nanoscale patterns that diffract incoming light and encodes both spatial and spectral information for each pixel on the sensor. By encoding the scene into a single, compact two-dimensional image rather than a massive three-dimensional data cube, the camera makes hyperspectral imaging faster and more efficient.

“One of the primary advantages of our camera is its ability to capture the spatial-spectral information in a highly compressed two-dimensional image instead of a three-dimensional data cube and use sophisticated computer algorithms to extract the full data cube at a later point,” Majumder explained. “This allows for fast, highly compressed data capture.”

The streamlined approach also cuts costs dramatically.

“Our camera costs many times less, is very compact and captures data much faster than most available commercial hyperspectral cameras,” Majumder said. “We have also shown the ability to post-process the data as per the need of the application and implement different classifiers suited to different fields such as agriculture, astronomy and bio-imaging.”

Data storage is another advantage.

“Satellites would have trouble beaming down full image cubes, but since we extract the cubes in post-processing, the original files are much smaller,” Majumder added.

To demonstrate the camera’s capabilities, the researchers tried three real-world applications: telling different types of tissue apart in a surgical scene; predicting the age of strawberries as they decayed over time; and mimicking a series of spectral filters that are used in astronomy.

The current prototype takes images at just over one megapixel in size and can break them down into 25 separate wavelengths across the spectrum. But the team is already working on improvements.

“This work demonstrates a first snapshot megapixel hyperspectral camera,” Majumder said. “Next, we are developing a more improved version of the camera that will allow us to capture images at a larger image size and increased number of wavelength channels, while also making the nano-structured diffractive element much simpler in design.”

By making hyperspectral imaging cheaper, faster, and more compact, the U engineers have opened the door for technologies that could change the way we see the world and uncover details hidden across the spectrum.