this post was submitted on 23 Nov 2024
462 points (99.4% liked)

Technology

60101 readers
2413 users here now

This is a most excellent place for technology news and articles.

Our Rules

  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots

founded 2 years ago
MODERATORS
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
462
AI tenant-screening tool SafeRent pays $2.28 million settlement for being massively racist (pivot-to-ai.com)
submitted 1 month ago by [email protected] to c/[email protected]
31 comments fedilink hide all child comments
 

SafeRent is a machine learning black box for landlords. It gives landlords a numerical rating of potential tenants and a yes/no result on whether to rent to them.

In May 2022, Massachusetts housing voucher recipients and the Community Action Agency of Somerville sued the company, claiming SafeRent gave Black and Hispanic rental applicants with housing vouchers disproportionately lower scores.

The tenants had no visibility into how the algorithm scored them. Appeals were rejected on the basis that this was what the computer output said.

you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 1 points 1 month ago

The article is fake news. I suggest looking elsewhere for proper information.

As for your questions: LLMs were certainly not involved here. I can't guess what techniques were used.

Racial discrimination is often hard to nail down. Race is implicit in any number of facts. Place of birth, current address, school, ... You could infer race from such data. If you do not look at race at all but the end result still discriminates, then it's probably still racial discrimination. I say probably because you are free to do what you like and discriminate based on any number of factors, as long as it isn't race, sex, and the like. You certainly may discriminate based on education or wealth. Things being as they are, that will discriminate against minorities. They have systematically lower credit ratings, for example.

In the case of generative AI, bias is often not clearly defined. For example, you type "US President" into an image generator. All US presidents so far were male, and all but one white. But half of all people who are eligible for the presidency are female and (I think) a little less than half non-white. So what's the non-biased output?