How your phone learned to see in the dark

Apr 5, 2023, 2:30 PM

Smart phone night photography...

Lights illuminate the Atlanta Botanical Gardens, in this photo taken using Google Pixel 5 Night Sight setting. Brook Joyner/CNN

 (CNN) — Open up Instagram at any given moment and it probably won’t take long to find crisp pictures of the night sky, a skyline after dark or a dimly lit restaurant. While shots like these used to require advanced cameras, they’re now often possible from the phone you already carry around in your pocket.

Tech companies such as Apple, Samsung and Google are investing resources to improve their night photography options at a time when camera features have increasingly become a key selling point for smartphones that otherwise largely all look and feel the same from one year to the next.

Earlier this month, Google brought a faster version of its Night Sight mode, which uses AI algorithms to lighten or brighten images in dark environments, to more of its Pixel models. Apple’s Night mode, which is available on models as far back as the iPhone 11, was touted as a premier feature on its iPhone 14 lineup last year thanks to its improved camera system.

These tools have come a long way in just the past few years, thanks to significant advancements in artificial intelligence technology as well as image processing that has become sharper, quicker, and more resilient to challenging photography situations. And smartphone makers aren’t done yet.

“People increasingly rely on their smartphones to take photos, record videos, and create content,” said Lian Jye Su, an artificial intelligence analyst at ABI Research. “[This] will only fuel the smartphone companies to up their games in AI-enhanced image and video processing.”

While there has been much focus lately on Silicon Valley’s renewed AI arms race over chatbots, the push to develop more sophisticated AI tools could also help further improve night photography and bring our smartphones closer to being able to see in the dark.

How it works

Samsung’s Night mode feature, which is available on various Galaxy models but optimized for its premium S23 Ultra smartphone, promises to do what would have seemed unthinkable just five to 10 years ago: enable phones to take clearer pictures with little light.

The feature is designed to minimize what’s called “noise,” a term in photography that typically refers to poor lighting conditions, long exposure times, and other elements that can take away from the quality of an image.

The secret to reducing noise, according to the company, is a combination of the S23 Ultra’s adaptive 200M pixel sensor. After the shutter button is pressed, Samsung uses advanced multi-frame processing to combine multiple images into a single picture and AI to automatically adjust the photo as necessary.

“When a user takes a photo in low or dark lighting conditions, the processor helps remove noise through multi-frame processing,” said Joshua Cho, executive vice president of Samsung’s Visual Solution Team. “Instantaneously, the Galaxy S23 Ultra detects the detail that should be kept, and the noise that should be removed.”

For Samsung and other tech companies, AI algorithms are crucial to delivering photos taken in the dark. “The AI training process is based on a large number of images tuned and annotated by experts, and AI learns the parameters to adjust for every photo taken in low-light situations,” Su explained.

For example, algorithms identify the right level of exposure, determine the correct color pallet and gradient under certain lighting conditions, sharpen blurred faces or objects artificially, and then makes those changes. The final result, however, can look quite different from what the person taking the picture saw in real time, in what some might argue is a technical sleight-of-hand trick.

Google is also focused on reducing noise in photography. Its AI-powered Night Sight feature captures a burst of longer-exposure frames. It then uses something called HDR+ Bracketing, which creates several photos with different settings. After a picture is taken, the images are combined together to create “sharper photos” even in dark environments “that are still incredibly bright and detailed,” said Alex Schiffhauer, a group product manager at Google.

While effective, there can be a slight but noticeable delay before the image is ready. But Schiffhauer said Google intends to speed up this process more on future Pixel iterations. “We’d love a world in which customers can get the quality of Night Sight without needing to hold still for a few seconds,” Schiffhauer said.

Google also has an astrophotography feature which allows people to take shots of the night sky without needing to tweak the exposure or other settings. The algorithms detect details in the sky and enhances them to stand out, according to the company.

Apple has long been rumored to be working on an astrophotography feature, but some iPhone 14 Pro Max users have successfully been able to capture pictures of the sky through its existing Night Mode tool. When a device detects a low-light environment, Night mode turns on to capture details and brighten shots. (The company did not respond to a request to elaborate on how the algorithms work.)

AI can make a difference in the image, but the end results for each of these features also depend on the phone’s lenses, said Gartner analyst Bill Ray. A traditional camera will have the lens several centimeters from the sensor, but the limited space on a phone often requires squeezing things together, which can result in a more shallow depth of field and reduced image quality, especially in darker environments.

“The quality of the lens is still a big deal, and how the phone addresses the lack of depth,” Ray said.

The next big thing

While night photography on phones has come a long way, a buzzy new technology could push it ahead even more.

Generative AI, the technology that powers the viral chatbot ChatGPT, has earned plenty of attention for its ability to create compelling essays and images in response to user prompts. But these AI systems, which are trained on vast troves of online data, also have potential to edit and process images.

“In recent years, generative AI models have also been used in photo-editing functions like background removal or replacement,” Su said. If this technology is added to smartphone photo systems, it could eventually make night modes even more powerful, Su said.

Big Tech companies, including Google, are already fully embracing this technology in other parts of their business. Meanwhile, smartphone chipset vendors like Qualcomm and MediaTek are looking to support more generative AI applications natively on consumer devices, Su said. These include image and video augmentation.

“But this is still about two to three years away from limited versions of this showing up on smartphones,” he said.

Related reading


We want to hear from you.

Have a story idea or tip? Send it to the KSL NewsRadio team here.

Today’s Top Stories

Science + Technology

Khalil Watson uses an exoskeleton to stand and walk at his college graduation ceremony years after ...

Nicki Brown and Ashley R. Williams, CNN

Weeks before graduating high school, he was shot and paralyzed. 7 years later, he walked across the stage to accept his college degree

Originally Published: 20 MAY 23 14:10 ET Updated: 21 MAY 23 12:41 ET (CNN) — Seven years ago, the chance to walk at graduation was stripped from Khalil Watson, a Virginia man who was paralyzed from the neck down after being shot weeks before finishing high school. But with help from physical therapists and a […]

11 days ago

FILE - The Twitter splash page is seen on a digital device, April 25, 2022, in San Diego. (AP Photo...

Brian Fung, CNN

Supreme Court shields Twitter from liability for terror-related content

The rulings are wins for Twitter that have faced a barrage of lawsuits from critics who allege they should be held legally responsible for societal harms, including terrorist attacks.

14 days ago

AI healthcare could be the next big thing....

Amanda Dickson

Here’s how AI could help Utah healthcare workers

AI healthcare could be the next future movement to help you receive good care and make it easier on healthcare professionals.

14 days ago

learning about fault lines...

Alexandrea Bonilla

Utah Geological Survey learning more on connection between Wasatch and West Valley fault lines

Scientists with the Utah Geological Survey are learning just how connected the Wasatch Fault line is to the West Valley fault line.

16 days ago

Linda Yaccarino, here in 2022, is leaving NBCUniversal amid reports that Elon Musk has selected her...

Chris Isidore

Elon Musk names NBCU ad chief Linda Yaccarino as Twitter CEO

 (CNN) — Elon Musk on Friday named longtime media executive Linda Yaccarino as the new CEO of Twitter, months after he promised to step back from the role. “I am excited to welcome Linda Yaccarino as the new CEO of Twitter!” Musk wrote in a tweet on Friday. He said she “will focus primarily on business operations, […]

19 days ago

twitter building shown, a new ceo will replace elon musk...

Chris Isidore, CNN

Twitter’s new CEO is an NBCUniversal executive with deep ad industry ties

Elon Musk confirmed that the new CEO for Twitter will be NBCUniversal’s Linda Yaccarino, an executive with deep ties to the advertising industry. “I am excited to welcome Linda Yaccarino as the new CEO of Twitter!” Musk wrote in a Friday tweet. He added that Yaccarino “will focus primarily on business operations” while Musk will […]

20 days ago

Sponsored Articles

close up of rose marvel saliva blooms in purple...

Shannon Cavalero

Drought Tolerant Perennials for Utah

The best drought tolerant plants for Utah can handle high elevations, alkaline soils, excessive exposure to wind, and use of secondary water.

Group of cheerful team members high fiving each other...

Visit Bear Lake

How To Plan a Business Retreat in Bear Lake This Spring

Are you wondering how to plan a business retreat this spring? Read our sample itinerary to plan a team getaway to Bear Lake.

Cheerful young woman writing an assignment while sitting at desk between two classmates during clas...

BYU EMBA at the Marriott School of Business

Hear it Firsthand: 6 Students Share Their Executive MBA Experience at BYU’s Marriott School of Business

The Executive MBA program at BYU offers great opportunities. Hear experiences straight from students enrolled in the program.

Skier being towed by a rider on a horse. Skijoring....

Bear Lake Convention and Visitors Bureau

Looking for a New Winter Activity? Try Skijoring in Bear Lake

Skijoring is when someone on skis is pulled by a horse, dog, animal, or motor vehicle. The driver leads the skiers through an obstacle course over jumps, hoops, and gates.

Banner with Cervical Cancer Awareness Realistic Ribbon...

Intermountain Health

Five Common Causes of Cervical Cancer – and What You Can Do to Lower Your Risk

January is National Cervical Cancer Awareness month and cancer experts at Intermountain Health are working to educate women about cervical cancer.

Kid holding a cisco fish at winterfest...

Bear Lake Convention and Visitors Bureau

Get Ready for Fun at the 2023 Bear Lake Monster Winterfest

The Bear Lake Monster Winterfest is an annual weekend event jam-packed full of fun activities the whole family can enjoy.

How your phone learned to see in the dark