1119
Very much smart people (piefedimages.s3.eu-central-003.backblazeb2.com)
you are viewing a single comment's thread
view the rest of the comments
[-] catch22@programming.dev 10 points 5 months ago

Feature Visualization How neural networks build up their understanding of images

https://distill.pub/2017/feature-visualization/

[-] mrmacduggan@lemmy.ml 16 points 5 months ago

This method is definitely a great way to achieve some degree of explainability for images, but it is based on the assumption that nearby pixels will have correllated meanings. When AI is making connections between far-away features, or worse, in a feature space that cannot be readily visualized like images can, it can be very hard to decouple the nonlinear outputs into singular linear features. While AI explainability has come a long way in the last few years, the decision-making processes of AI are so different from human thought that even when it can "show its work" by showing which neurons contributed to the final result, it doesn't necessarily make any intuitive sense to us.

For example, an image-identification AI might identify subtle lens blur data to determine the brand of camera that took a photograph, and then use that data to make an educated guess about which country the image was taken in. It's a valid path of reasoning. But it would take a lot of effort for a human analyst to notice that the AI is using this process to slightly improve its chances of getting the image identification correct, and there are millions of such derived features that combine in unexpected ways, some logical and some irrationally overfitting to the training data.

this post was submitted on 22 Sep 2025
1119 points (99.0% liked)

Microblog Memes

10898 readers
2430 users here now

A place to share screenshots of Microblog posts, whether from Mastodon, tumblr, ~~Twitter~~ X, KBin, Threads or elsewhere.

Created as an evolution of White People Twitter and other tweet-capture subreddits.

RULES:

  1. Your post must be a screen capture of a microblog-type post that includes the UI of the site it came from, preferably also including the avatar and username of the original poster. Including relevant comments made to the original post is encouraged.
  2. Your post, included comments, or your title/comment should include some kind of commentary or remark on the subject of the screen capture. Your title must include at least one word relevant to your post.
  3. You are encouraged to provide a link back to the source of your screen capture in the body of your post.
  4. Current politics and news are allowed, but discouraged. There MUST be some kind of human commentary/reaction included (either by the original poster or you). Just news articles or headlines will be deleted.
  5. Doctored posts/images and AI are allowed, but discouraged. You MUST indicate this in your post (even if you didn't originally know). If an image is found to be fabricated or edited in any way and it is not properly labeled, it will be deleted.
  6. Absolutely no NSFL content.
  7. Be nice. Don't take anything personally. Take political debates to the appropriate communities. Take personal disagreements & arguments to private messages.
  8. No advertising, brand promotion, or guerrilla marketing.

RELATED COMMUNITIES:

founded 2 years ago
MODERATORS