Google Camera’s Portrait Mode technology from the Pixel 2 is now open source

The Pixel 2, which launched in October 2017, is continuing to be recognized as the best smartphone camera. This is thanks to the various software enhancements, including a dedicated processor. The processor makes it possible for Google's Portrait Mode to do some magical things.

Google's Portrait Mode

For the unaware, Portrait Mode identifies the subject in the foreground and then blurs the background. This creates a unique and interesting image and has been coming to more devices.What is interesting is the technology behind Portrait Mode. Back in March, Google announced that 'Semantic Image Segmentation' is now open source.By open-sourcing this technology, it opens the doors for developers to jump on board. The specific code is DeepLab-v3+ and is what Google based its own Portrait Mode on.DeepLab-V3+ works as an image segmentation tool, using "convolutional neural networks" (CNN). This is a type of machine learning which analyzes objects, then splits them apart.From there the technology then divides the foreground elements from the background elements. After the software works its magic, then you see the "bokeh" effect found in Portrait Mode.In addition to providing the image segmentation, DeepLab-V3+ also makes real-time video segmentation possible. With all of this, Google's technology will improve just about any camera app that uses it.Finally, the semantic image segmentation is smart enough to recognize what you're taking a picture of. Not only does it recognize the object, but it also labels the item without any other input.

What makes the Pixel 2 so great?

What makes this so great is the fact that the Pixel 2 only features a single rear camera. Instead of joining the brigade of dual-camera systems, Google opted to stick to its guns.The singular camera sensor is combined with great software to produce phenomenal images. When the Pixel 2 was announced, it was awarded the best smartphone camera from DxOMark.Other devices are available and are ranked higher, but in real-world use, the Pixel 2 continues to impress. There are some that still believe that Google is the industry standard for cameras on smartphones.When it comes to the software side of things, we have seen developers port the stock Google Camera app. You can download the app, with all of the best software features from Google, and improves image quality on non-Pixel handsets.Unfortunately, the app is not available for the Galaxy S9, "hackers" are making strides. The biggest benefit to using Google's own camera app is due to the various software enhancements.Not only do we see improvements in Portrait Mode, but there are also benefits when using HDR mode. It's only a matter of time before this happens, so we'll be keeping our fingers crossed.

SOURCE: Google AI Blog