Industry giants and academics tackle
the harmful parts of imaging AI
[Scroll down for The imaging announcement highlights from Google I/O and the Snap Partner Summit]
|
|
The good news: awareness of these negative effects is mounting and major AI imaging players and academic researchers are developing solutions to eliminate – or at least reduce – the most negative aspects.
Just in the last two weeks alone, we’ve seen noteworthy announcements of initiatives to combat undesired AI imaging effects, including those from Google and Snap.
|
|
Oct. 19-20: Visual 1st 2021
Virtual interactive conference
Oct. 21: DIY Video Ecosystem Summit
Virtual interactive conference
Oct. 28: Meet & Greet,
in-person, San Francisco
Stay tuned for updates!
|
|
Google:
More equitable camera initiative
|
These changes will come to Google’s own Pixel smartphone cameras this fall, and the company says it will also share what it learns across the broader Android ecosystem.
|
|
|
Specifically, Google is making changes to its auto-white balance and exposure algorithms to improve accuracy for dark skin tones based on a broader data set of images featuring black and brown faces. With these tweaks, Google aims to avoid over-brightening and de-saturating people of color in photos for more accurate representation. Google has also made improvements for portrait mode selfies, creating a more accurate depth map for curly and wavy hair types — rather than simply cutting around the subject’s hair.
|
|
Snap:
Inclusive camera initiative
|
As Snapchat’s camera functionality also includes image creation and AR effects, non-biased image capture is extra crucial. Hence, Snapchat’s "Inclusive camera" initiative is broader than just capturing dark skin as well as light skin. It also includes identifying and removing a range of biased assumptions (for instance, that smaller, thinner noses are better) when automatically adjusting people's appearance.
|
|
|
|
Twitter:
Removing AI auto-cropping
|
Twitter shared the results of an internal study of the workings of its AI-based auto-cropping features, which showed racial and gender biases. Its underlaying AI algorithms were trained on human eye-tracking data and significantly favored white individuals over black people, and women over men.
|
|
|
In testing, compared to a 50-50 chance of "demographic parity", Twitter found:
- An 8% difference in favor of women over men
- A 4% difference favoring white people over black people of both sexes
- A 7% difference favoring white women over black women
- A 2% difference in favor of white men over black men
“We considered the tradeoffs between the speed and consistency of automated cropping with the potential risks we saw in this research,” Rumman Chowdhury, software engineering director, says. “One of our conclusions is that not everything on Twitter is a good candidate for an algorithm, and in this case, how to crop an image is a decision best made by people.”
In the interim, Twitter’s latest update includes a realistic preview of photos during Tweet creation, so that users will know exactly how the images will look prior to publishing.
|
|
Fawkes and LowKey:
Cloaking to spoof AI imaging tools
|
Still don’t trust the social media networks’ initiatives to eliminate biases in their AI imaging tools? Or don’t trust face recognition software in the hands of certain government agencies or enterprises?
Tools based on these methods make tiny changes to an image that are not visible to the human eye but can confuse an imaging AI tool (they "cloak"), forcing the software to make a mistake in identifying the person or the object in the image, or, even stopping it from realizing the image is a selfie. The Fawkes tool can be downloaded here. The LowKey solution can be tested here.
|
|
|
Snap Partner Summit takeaways
|
Snapchat now has 500M monthly active users, 40% of Snap's users these days residing outside of North America and Europe – apparently finally having an up to par Android version of their apps is paying off!
Each day 5 billion pictures are taken by using Snapchat's camera app.
|
|
|
250,000 developers are registered to use Snap's developer tools, Snap Kit, which helps developers build and distribute augmented reality lenses.
Nearly 2M augmented reality lenses have been created by Snapchat users using Snap's developer tools. Those lenses have been viewed by Snapchat users more than 2 trillion times.
More than 135M people use Snapchat's TikTok rival feature, Spotlight. Over 5,400 creators have collectively earned more than $130M dollars from payouts Snapchat has awarded to viral video makers. Snapchat will add "tipping" for creators later this year.
Snap announced new Spectacles, 3D glasses with AR features. The new glasses have 4 built in microphones, 2 stereo speakers and built-in touchpad controls that make it easy to send pictures and videos using AR to friends. Note that for now these Spectacles are only available to creators who want to make AR filters with the glasses using Snapchat's Lens Studio developer tools. Snap hasn’t announced if or when they will become available to the general market.
Snap announced Story Studio, an iOS video editing app for creating vertical videos for Spotlight. The Story Studio app not only does the usual (trimming, clipping, adding effects) but can also look through Snapchat Insights to determine what’s trending on the app across sounds, hashtags, and lenses. In addition, Story Studio can leverage the wealth of AR lenses available in the overall Snapchat platform.
|
|
Google Photos. Creating phodeos. Google announces the ability to create animated photos from just 2 similar still shots by using its AI smartness to add transitional frames. (More about the trends behind hybrid photo/video formats in our Videos and Phodeos study).
|
|
|
Project Starline. 1-on-1 life-size video chat booth. Imagine looking through a sort of magic window, and through that window, you see another person, life-size and in three dimensions. Featuring high-end cameras, custom depth sensors and a new type of light field display this 1-on-1 video chat booth is not going to replace Zoom anytime soon – or ever. Google is still in the early internal testing phase and hopes to expand Starline to partners in media and health care in the future. Check out the video.
Google Lens. Part of the camera. This camera-based search engine will be part of the Android camera app, with 11 smartphone models so far signed up. Lens’ latest update has more text-reading and additional language features and, of course, shopping suggestions – not just for the objects you point your camera at but whatever the camera can scan around you. Could easily become creepy if not annoying, so we’ll keep our fingers crossed. The race is on with other visual search providers, including Pinterest Lens (which powers Samsung’s Bixby Vision visual search tool).
|
|
Vimeo. IPO. Congratulations to video solution provider Vimeo, which went public yesterday, while spinning off from IAC. Vimeo previously acquired Visual 1st presenter Magisto and is looking to acquire more video startups in the future, according to CEO Anjali Sud. Vimeo has more than 200M users and as of the end of March had 1.6M paying subscribers, up 25% year over year. Its first day of trading was disappointing: down 12.8% from its price when it was trading on when-issued basis last week but still …. Giving it a market capitalization of about $7.2 billion.
Zenfolio. BookMe. Zenfolio launches BookMe, a comprehensive client management suite to automate bookings and payments, which can also be accessed through Zenfolio’s website builder solutions, Portfolio and PortfolioPlus. BookMe syncs with Google calendar to show the photographer’s availability in real-time. BookMe is the latest step in the modernizing Zenfolio’s original technology platform.
Snap. Biggest acquisition to date. Snap acquired AR startup WaveOptics for over $500M. The company, which represents Snap’s biggest acquisition to date, provides the waveguides and projectors used in Snap’s AR glasses, Spectacles. Who said that Spectacles are just a side project for Snap?
Pinterest. Pins going video. Pinterest is upgrading its Story Pins feature (and renaming it as Idea Pins) by centering Pins around the ability to share product or “how to” ideas though short-form videos, alongside other media, such as photos, music, text and stickers. Creators can edit their up to 60 second videos by adding their own voiceover or using a “ghost mode” transition tool to better showcase their before-and-afters by overlaying one part of a video on another. Like companies such as Snap, YouTube, TikTok and Instagram (for its Reels app), Pinterest also provides a fund to attract Creators. However, the $500K fund is only a fraction of what the other video-promoting platforms offer.
Poparazzi. Photo app bans selfies. No more “Look at this photo I took of what I’m doing.” The idea behind the new Poparazzi app: you propagate your friend’s social profile with your photos, and they do so with yours. You are your friends' “poparazzi,” and they are yours. You'll receive a notification when a friend adds a photo of you, while the app gives you control to remove this photo from your profile if you don't like it.
|
|
|
|
|
|
|