New APIC method eliminates blurriness and distortion in microscopy

California Institute of TechnologyJun 28 2024

For hundreds of years, the clarity and magnification of microscopes were ultimately limited by the physical properties of their optical lenses. Microscope makers pushed those boundaries by making increasingly complicated and expensive stacks of lens elements. Still, scientists had to decide between high resolution and a small field of view on the one hand or low resolution and a large field of view on the other.

In 2013, a team of Caltech engineers introduced a microscopy technique called FPM (for Fourier ptychographic microscopy). This technology marked the advent of computational microscopy, the use of techniques that wed the sensing of conventional microscopes with computer algorithms that process detected information in new ways to create deeper, sharper images covering larger areas. FPM has since been widely adopted for its ability to acquire high-resolution images of samples while maintaining a large field of view using relatively inexpensive equipment.

Now the same lab has developed a new method that can outperform FPM in its ability to obtain images free of blurriness or distortion, even while taking fewer measurements. The new technique, described in a paper that appeared in the journal Nature Communications, could lead to advances in such areas as biomedical imaging, digital pathology, and drug screening.

The new method, dubbed APIC (for Angular Ptychographic Imaging with Closed-form method), has all the advantages of FPM without what could be described as its biggest weakness-;namely, that to arrive at a final image, the FPM algorithm relies on starting at one or several best guesses and then adjusting a bit at a time to arrive at its "optimal" solution, which may not always be true to the original image.

Under the leadership of Changhuei Yang, the Thomas G. Myers Professor of Electrical Engineering, Bioengineering, and Medical Engineering and an investigator with the Heritage Medical Research Institute, the Caltech team realized that it was possible to eliminate this iterative nature of the algorithm.

Rather than relying on trial and error to try to home in on a solution, APIC solves a linear equation, yielding details of the aberrations, or distortions introduced by a microscope's optical system. Once the aberrations are known, the system can correct for them, basically performing as though it is ideal, and yielding clear images covering large fields of view.

"We arrive at a solution of the high-resolution complex field in a closed-form fashion, as we now have a deeper understanding in what a microscope captures, what we already know, and what we need to truly figure out, so we don't need any iteration," says Ruizhi Cao (PhD '24), co-lead author on the paper, a former graduate student in Yang's lab, and now a postdoctoral scholar at UC Berkeley. "In this way, we can basically guarantee that we are seeing the true final details of a sample."

As with FPM, the new method measures not only the intensity of the light seen through the microscope but also an important property of light called "phase," which is related to the distance that light travels. This property goes undetected by human eyes but contains information that is very useful in terms of correcting aberrations. It was in solving for this phase information that FPM relied on a trial-and-error method, explains Cheng Shen (PhD '23), co-lead author on the APIC paper, who also completed the work while in Yang's lab and is now a computer vision algorithm engineer at Apple. "We have proven that our method gives you an analytical solution and in a much more straightforward way. It is faster, more accurate, and leverages some deep insights about the optical system."

Beyond eliminating the iterative nature of the phase-solving algorithm, the new technique also allows researchers to gather clear images over a large field of view without repeatedly refocusing the microscope. With FPM, if the height of the sample varied even a few tens of microns from one section to another, the person using the microscope would have to refocus in order to make the algorithm work. Since these computational microscopy techniques frequently involve stitching together more than 100 lower-resolution images to piece together the larger field of view, that means APIC can make the process much faster and prevent the possible introduction of human error at many steps.

"We have developed a framework to correct for the aberrations and also to improve resolution," says Cao. "Those two capabilities can be potentially fruitful for a broader range of imaging systems."

Yang says the development of APIC is vital to the broader scope of work his lab is currently working on to optimize image data input for artificial intelligence (AI) applications. "Recently, my lab showed that AI can outperform expert pathologists at predicting metastatic progression from simple histopathology slides from lung cancer patients," says Yang. "That prediction ability is exquisitely dependent on obtaining uniformly in-focus and high-quality microscopy images, something that APIC is highly suited for."

The paper, titled, "High-resolution, large field-of-view label-free imaging via aberration-corrected, closed-form complex field reconstruction" appeared online in Nature Communications on June 3. The work was supported by the Heritage Medical Research Institute.

For hundreds of years, the clarity and magnification of microscopes were ultimately limited by the physical properties of their optical lenses. Microscope makers pushed those boundaries by making increasingly complicated and expensive stacks of lens elements. Still, scientists had to decide between high resolution and a small field of view on the one hand or low resolution and a large field of view on the other.

In 2013, a team of Caltech engineers introduced a microscopy technique called FPM (for Fourier ptychographic microscopy). This technology marked the advent of computational microscopy, the use of techniques that wed the sensing of conventional microscopes with computer algorithms that process detected information in new ways to create deeper, sharper images covering larger areas. FPM has since been widely adopted for its ability to acquire high-resolution images of samples while maintaining a large field of view using relatively inexpensive equipment.

Now the same lab has developed a new method that can outperform FPM in its ability to obtain images free of blurriness or distortion, even while taking fewer measurements. The new technique, described in a paper that appeared in the journal Nature Communications, could lead to advances in such areas as biomedical imaging, digital pathology, and drug screening.

The new method, dubbed APIC (for Angular Ptychographic Imaging with Closed-form method), has all the advantages of FPM without what could be described as its biggest weakness-;namely, that to arrive at a final image, the FPM algorithm relies on starting at one or several best guesses and then adjusting a bit at a time to arrive at its "optimal" solution, which may not always be true to the original image.

Under the leadership of Changhuei Yang, the Thomas G. Myers Professor of Electrical Engineering, Bioengineering, and Medical Engineering and an investigator with the Heritage Medical Research Institute, the Caltech team realized that it was possible to eliminate this iterative nature of the algorithm.

Rather than relying on trial and error to try to home in on a solution, APIC solves a linear equation, yielding details of the aberrations, or distortions introduced by a microscope's optical system. Once the aberrations are known, the system can correct for them, basically performing as though it is ideal, and yielding clear images covering large fields of view.

"We arrive at a solution of the high-resolution complex field in a closed-form fashion, as we now have a deeper understanding in what a microscope captures, what we already know, and what we need to truly figure out, so we don't need any iteration," says Ruizhi Cao (PhD '24), co-lead author on the paper, a former graduate student in Yang's lab, and now a postdoctoral scholar at UC Berkeley. "In this way, we can basically guarantee that we are seeing the true final details of a sample."

As with FPM, the new method measures not only the intensity of the light seen through the microscope but also an important property of light called "phase," which is related to the distance that light travels. This property goes undetected by human eyes but contains information that is very useful in terms of correcting aberrations. It was in solving for this phase information that FPM relied on a trial-and-error method, explains Cheng Shen (PhD '23), co-lead author on the APIC paper, who also completed the work while in Yang's lab and is now a computer vision algorithm engineer at Apple. "We have proven that our method gives you an analytical solution and in a much more straightforward way. It is faster, more accurate, and leverages some deep insights about the optical system."

Beyond eliminating the iterative nature of the phase-solving algorithm, the new technique also allows researchers to gather clear images over a large field of view without repeatedly refocusing the microscope. With FPM, if the height of the sample varied even a few tens of microns from one section to another, the person using the microscope would have to refocus in order to make the algorithm work. Since these computational microscopy techniques frequently involve stitching together more than 100 lower-resolution images to piece together the larger field of view, that means APIC can make the process much faster and prevent the possible introduction of human error at many steps.

"We have developed a framework to correct for the aberrations and also to improve resolution," says Cao. "Those two capabilities can be potentially fruitful for a broader range of imaging systems."

Yang says the development of APIC is vital to the broader scope of work his lab is currently working on to optimize image data input for artificial intelligence (AI) applications. "Recently, my lab showed that AI can outperform expert pathologists at predicting metastatic progression from simple histopathology slides from lung cancer patients," says Yang. "That prediction ability is exquisitely dependent on obtaining uniformly in-focus and high-quality microscopy images, something that APIC is highly suited for."

The paper, titled, "High-resolution, large field-of-view label-free imaging via aberration-corrected, closed-form complex field reconstruction" appeared online in Nature Communications on June 3. The work was supported by the Heritage Medical Research Institute.