Visual 1st Perspectives
May 26, 2021
Industry giants and academics tackle
the harmful parts of imaging AI


[Scroll down for The imaging announcement highlights from Google I/O and the Snap Partner Summit]
A few weeks ago I described the broad adoption of AI among imaging companies in my Where Gartner’s Hype Cycle goes wrong: today’s surge in real-life AI imaging solutions piece, while noting that besides broad adoption of AI the imaging industry is also stuck in what Gartner calls the Trough of Disillusionment – specifically the realization that AI-based imaging solutions can also cause harm by tricking and misinforming people (deepfakes) or by displaying racial or gender biases.

The good news: awareness of these negative effects is mounting and major AI imaging players and academic researchers are developing solutions to eliminate – or at least reduce – the most negative aspects.

Just in the last two weeks alone, we’ve seen noteworthy announcements of initiatives to combat undesired AI imaging effects, including those from Google and Snap.


Oct. 19-20: Visual 1st 2021
Virtual interactive conference

Oct. 21: DIY Video Ecosystem Summit
Virtual interactive conference

Oct. 28: Meet & Greet,
in-person, San Francisco

Stay tuned for updates!
Check out our newly updated DIY Video Creation Apps report
Google:
More equitable camera initiative
After having fired two prominent AI ethics researchers earlier this year, at Google I/O last week Google put its best face forward by highlighting its “More equitable camera” initiative, aimed at improving its own camera and imaging products with a focus on making images of people of color “more beautiful and more accurate.”

These changes will come to Google’s own Pixel smartphone cameras this fall, and the company says it will also share what it learns across the broader Android ecosystem.
Image: Google
Specifically, Google is making changes to its auto-white balance and exposure algorithms to improve accuracy for dark skin tones based on a broader data set of images featuring black and brown faces. With these tweaks, Google aims to avoid over-brightening and de-saturating people of color in photos for more accurate representation. Google has also made improvements for portrait mode selfies, creating a more accurate depth map for curly and wavy hair types — rather than simply cutting around the subject’s hair.
Snap:
Inclusive camera initiative
At the Snap Partner Summit last week, Snap disclosed how it is making its camera app more inclusive, including by improving Snapchat’s selfie camera to better capture low light by making adjustments to the front flash.

As Snapchat’s camera functionality also includes image creation and AR effects, non-biased image capture is extra crucial. Hence, Snapchat’s "Inclusive camera" initiative is broader than just capturing dark skin as well as light skin. It also includes identifying and removing a range of biased assumptions (for instance, that smaller, thinner noses are better) when automatically adjusting people's appearance.
Image: Snap
Twitter:
Removing AI auto-cropping
Twitter shared the results of an internal study of the workings of its AI-based auto-cropping features, which showed racial and gender biases. Its underlaying AI algorithms were trained on human eye-tracking data and significantly favored white individuals over black people, and women over men.
Image: Twitter
In testing, compared to a 50-50 chance of "demographic parity", Twitter found:
  • An 8% difference in favor of women over men
  • A 4% difference favoring white people over black people of both sexes
  • A 7% difference favoring white women over black women
  • A 2% difference in favor of white men over black men

As a result of these findings, Twitter is now moving away from AI-powered photo cropping and is putting the decision-making back in the hands of its users.

“We considered the tradeoffs between the speed and consistency of automated cropping with the potential risks we saw in this research,” Rumman Chowdhury, software engineering director, says. “One of our conclusions is that not everything on Twitter is a good candidate for an algorithm, and in this case, how to crop an image is a decision best made by people.”

In the interim, Twitter’s latest update includes a realistic preview of photos during Tweet creation, so that users will know exactly how the images will look prior to publishing.
Fawkes and LowKey:
Cloaking to spoof AI imaging tools
Still don’t trust the social media networks’ initiatives to eliminate biases in their AI imaging tools? Or don’t trust face recognition software in the hands of certain government agencies or enterprises?

On the academic side of the spectrum, last week two methods were presented at the International Conference of Learning Representations (ICLR) that leverage adversarial attacks to spoof AI or prevent detection by facial recognition software.

Tools based on these methods make tiny changes to an image that are not visible to the human eye but can confuse an imaging AI tool (they "cloak"), forcing the software to make a mistake in identifying the person or the object in the image, or, even stopping it from realizing the image is a selfie. The Fawkes tool can be downloaded here. The LowKey solution can be tested here.
Image: Fawkes
Snap Partner Summit takeaways
Snapchat now has 500M monthly active users, 40% of Snap's users these days residing outside of North America and Europe – apparently finally having an up to par Android version of their apps is paying off!

Each day 5 billion pictures are taken by using Snapchat's camera app.
250,000 developers are registered to use Snap's developer tools, Snap Kit, which helps developers build and distribute augmented reality lenses.

Nearly 2M augmented reality lenses have been created by Snapchat users using Snap's developer tools. Those lenses have been viewed by Snapchat users more than 2 trillion times.

More than 135M people use Snapchat's TikTok rival feature, Spotlight. Over 5,400 creators have collectively earned more than $130M dollars from payouts Snapchat has awarded to viral video makers. Snapchat will add "tipping" for creators later this year.

Snap announced new Spectacles, 3D glasses with AR features. The new glasses have 4 built in microphones, 2 stereo speakers and built-in touchpad controls that make it easy to send pictures and videos using AR to friends. Note that for now these Spectacles are only available to creators who want to make AR filters with the glasses using Snapchat's Lens Studio developer tools. Snap hasn’t announced if or when they will become available to the general market.

Snap announced Story Studio, an iOS video editing app for creating vertical videos for Spotlight. The Story Studio app not only does the usual (trimming, clipping, adding effects) but can also look through Snapchat Insights to determine what’s trending on the app across sounds, hashtags, and lenses. In addition, Story Studio can leverage the wealth of AR lenses available in the overall Snapchat platform.
Google I/O takeaways
Google Photos. Creating phodeos. Google announces the ability to create animated photos from just 2 similar still shots by using its AI smartness to add transitional frames. (More about the trends behind hybrid photo/video formats in our Videos and Phodeos study).
Google Photos. Locked folder and hidden memories. Google announced it will let you hide your most sensitive images behind a folder protected by a password or thumbprint. The app will also soon let you hide specific people or periods of your life from its “memories” to avoid unwanted reminders of the past.

Project Starline. 1-on-1 life-size video chat booth. Imagine looking through a sort of magic window, and through that window, you see another person, life-size and in three dimensions. Featuring high-end cameras, custom depth sensors and a new type of light field display this 1-on-1 video chat booth is not going to replace Zoom anytime soon – or ever. Google is still in the early internal testing phase and hopes to expand Starline to partners in media and health care in the future. Check out the video.

Google Lens. Part of the camera. This camera-based search engine will be part of the Android camera app, with 11 smartphone models so far signed up. Lens’ latest update has more text-reading and additional language features and, of course, shopping suggestions – not just for the objects you point your camera at but whatever the camera can scan around you. Could easily become creepy if not annoying, so we’ll keep our fingers crossed. The race is on with other visual search providers, including Pinterest Lens (which powers Samsung’s Bixby Vision visual search tool).
And a few more things...
Vimeo. IPO. Congratulations to video solution provider Vimeo, which went public yesterday, while spinning off from IAC. Vimeo previously acquired Visual 1st presenter Magisto and is looking to acquire more video startups in the future, according to CEO Anjali Sud. Vimeo has more than 200M users and as of the end of March had 1.6M paying subscribers, up 25% year over year. Its first day of trading was disappointing: down 12.8% from its price when it was trading on when-issued basis last week but still …. Giving it a market capitalization of about $7.2 billion.

Zenfolio. BookMe. Zenfolio launches BookMe, a comprehensive client management suite to automate bookings and payments, which can also be accessed through Zenfolio’s website builder solutions, Portfolio and PortfolioPlus. BookMe syncs with Google calendar to show the photographer’s availability in real-time. BookMe is the latest step in the modernizing Zenfolio’s original technology platform.

Promo.com & Getty Images. A sign of the times: Getty Images leads $16M investment in DIY video tool provider, Promo.com (née Slidely). Promo.com is also covered in our DIY Video Creation Apps report. Note that Getty recently also acquired Unsplash, a free photo stock platform, another sign that Getty is further diversifying from its roots as a high-end photo content licensing platform.

Snap. Biggest acquisition to date. Snap acquired AR startup WaveOptics for over $500M. The company, which represents Snap’s biggest acquisition to date, provides the waveguides and projectors used in Snap’s AR glasses, Spectacles. Who said that Spectacles are just a side project for Snap?

Pinterest. Pins going video. Pinterest is upgrading its Story Pins feature (and renaming it as Idea Pins) by centering Pins around the ability to share product or “how to” ideas though short-form videos, alongside other media, such as photos, music, text and stickers. Creators can edit their up to 60 second videos by adding their own voiceover or using a “ghost mode” transition tool to better showcase their before-and-afters by overlaying one part of a video on another. Like companies such as Snap, YouTube, TikTok and Instagram (for its Reels app), Pinterest also provides a fund to attract Creators. However, the $500K fund is only a fraction of what the other video-promoting platforms offer.

Poparazzi. Photo app bans selfies. No more “Look at this photo I took of what I’m doing.” The idea behind the new Poparazzi app: you propagate your friend’s social profile with your photos, and they do so with yours. You are your friends' “poparazzi,” and they are yours. You'll receive a notification when a friend adds a photo of you, while the app gives you control to remove this photo from your profile if you don't like it.

Reface. Face-swapping beyond selfies. Reface now lets users face-swap uploaded photos, or humanoid imagery (such as monuments, memes, or fine art portraits) inside photos, GIFs or videos. As of March 2021, Reface said it had 100M installs some 14 months since going live.

Archive & Subscribe Share your news with us | Connect on LinkedIn