Monday, June 24, 2024

Artificial Intelligence news

Synthesia’s hyperrealistic deepfakes will...

Startup Synthesia’s AI-generated avatars are getting an update to make them even...

How underwater drones could...

A potential future conflict between Taiwan and China would be shaped by...

How generative AI could...

First, a confession. I only got into playing video games a little...

I tested out a...

This story first appeared in China Report, MIT Technology Review’s newsletter about...
HomeTechnologyAI tool Stable...

AI tool Stable Diffusion amplifies race and gender stereotypes


A popular generative AI tool that creates images from text prompts is rife with gender and racial stereotypes when it comes to rendering people in “high-paying” and “low-paying jobs,” according to a recent study.

Stable Diffusion, a free-to-use AI model, was asked to render 5,100 images from written prompts related to job titles in 14 fields, plus three categories related to crime, according to a test of the tool by Bloomberg.

The outlet then analyzed the results against the Fitzpatrick Skin Scale — a six-point scale dermatologists use to asses the amount of pigment in someone’s skin.

Images generated for every “high-paying” job — like architect, doctor, lawyer, CEO and politician — were dominated by lighter skin tones, numbered one to three on the skin scale, Bloomberg found.

Meanwhile, darker skin tones made up the majority of “low-paying” jobs, like janitors, dishwashers, fast-food workers and social workers.

The stereotyping was even worse when Bloomberg asked Stable Diffusion to categorize job-related images by gender.

The AI tool generated nearly three times as many images of men than women, with all but four of 14 jobs — cashier, teacher, social worker and housekeeper — being dominated by women.


Darker-skinned men were the images generated for the majority if images in the janitor category while women only outnumbered their male counterparts for jobs like cashier, teacher and housekeeper.
Stanford University

Of the 300 images created for each of the 14 jobs, all but two images for the keyword “engineer” were perceived to be men, Bloomberg reported, while zero images of women were generated for the keyword “janitor.”

The prompts for crimes asked the AI tool to render images for drug dealers, terrorists and inmates. The vast majority of results for both drug dealers and inmates were darker-skinned.

The results for terrorists rendered men with dark facial hair, often wearing head coverings — clearly leaning on stereotypes of Muslim men, Bloomberg found.

“All AI models have inherent biases that are representative of the datasets they are trained on,” a spokesperson for London-based startup StabilityAI, which distributes Stable Diffusion, told The Post in an email statement.

They added that as an open-source model, which means the software can learn based on new algorithms and data sets, platforms like Stable Diffusion will one day “improve bias evaluation techniques and develop solutions beyond basic prompt modification.”


When asked to generate 300 images for one of 14 jobs -- seven "high-paying" and seven "low-paying" -- Stable Diffusion rendered the most images of light-skinned men for positions like CEO, lawyer, doctor and politician.
When asked to generate 300 images for one of 14 jobs — seven “high-paying” and seven “low-paying” — Stable Diffusion rendered the most images of light-skinned men for positions like CEO, lawyer, doctor and politician.
AFP via Getty Images

“We intend to train open source models on datasets specific to different countries and cultures, which will serve to mitigate biases caused by overrepresentation in general datasets,” the spokesperson said.

Stable Diffusion is part of the quickly-growing generative AI imaging industry that also includes paid services like DeepAI, Midjourney and Dall-E from OpenAI — the firm behind ChatGPT.

Stable Diffusion is already being used by startups like Deep Agency, an AI-powered virtual photo studio and modeling agency out of Holland that allows brands to generate images of humans for mainstream advertising.

Deep Agency is still currently in beta, according to its website, which shows off its capabilities with two posing people who look real. However, “these models do not exist,” the site notes.

Graphic design platform Canva, which boasts 125 million active users, has also introduced a Stable Diffusion integration that has allowed all types of companies and marketers to incorporate unique, AI-generated images into their design work and advertising.

The head of Canva’s AI products, Danny Wu, told Bloomberg the company’s users have already created 114 million images using Canva’s Stable Diffusion integration, which he assed will soon be “de-biased.”

“The issue of ensuring that AI technology is fair and representative, especially as they become more widely adopted, is a really important one that we are working actively on,” he told the outlet.


Bloomberg analyzed the AI-generated images against the Fitzpatrick Skin Scale -- a six-point scale dermatologists use to asses the amount of pigment in someone's skin.
Bloomberg analyzed the AI-generated images against the Fitzpatrick Skin Scale — a six-point scale dermatologists use to asses the amount of pigment in someone’s skin.
John D’Orazio

The need to remove bias from generative AI tools has become more pressing as police forces look to tap the tech to create photo-realistic composite images of suspects.

“Showing someone a machine-generated image can reinforce in their mind that that’s the person even when it might not be — even when it’s a completely faked image,” Nicole Napolitano, director of research strategy at the Center for Policing Equity, told Bloomberg.

Napolitano said that police departments with cushy budgets are already adopting AI-backed tech despite its lack of regulation.

She also cited the thousands of wrongful arrests that have been made from lapses in tech like facial recognition systems and biased AI models.



Article Source link and Credit

Continue reading

Car dealerships forced to process orders by hand after cyberattacks

More than 15,000 car dealerships throughout North America scrambled to process orders by hand after cyberattacks shut down their computerized management system this week. Car dealers in the US and Canada including those that sell BMW were having difficulty...

Amazon plans revamp of Alexa with monthly fee, AI features

Amazon is planning a major revamp of its decade-old money-losing Alexa service to include a conversational generative AI with two tiers of service and has considered a monthly fee of around $5 to access the superior version, according to people with direct knowledge of the...

Ferrari’s first electric car to cost whopping $500K: report

Ferrari’s first electric car will cost at least 500,000 euros ($535,000), a source familiar with the matter told Reuters, as the luxury automaker prepares to open a plant that will make the model – and could boost group...