Janelle Shane is a user on wandering.shop. You can follow them or interact with them if you have an account anywhere in the fediverse. If you don't, you can sign up here.
Janelle Shane @janellecshane

New blog post: When Visual Chatbot is wrong, it just keeps digging itself a hole.
Visual Chatbot is so much fun.

ยท 44 ยท 53

@janellecshane I'm fascinated that it just... lies. Even if that's a bit of anthropomorphism, there's no intent to deceive. It's easy to catch it in contradictions but the fact that it doesn't just fold and give up feels different somehow.

I'm way out of touch with how these actually work, I only played with making a NN like 8 years ago, but here's a weird question - if you were to use adversarial techniques on this, would you be making a better liar, a better description of photos, or both?

@floatoverblow I wonder if the lying is because it was trained to copy human responses to questions - and the humans very rarely folded in confusion. What the bot doesn't know is this is because the humans were very rarely confused. It copied the confidence but not the reason.

@janellecshane It reminds me of my 4 year old niece. She'll never say she doesn't know about a question, she'll figure out something plausible and say that instead. Makes teaching her to read very difficult, but entertaining - she'll often just read what she thinks things _probably_ say about the pictures.

@floatoverblow @janellecshane had the same thought...feels like the value of honesty is meta to its objectives to start

@janellecshane Thanks to this bot I now know that the #thinkingBread (I think I stole the picture from boringpeople.org) is actually saying (or thinking? :thaenkin:) "UNK UNK UNK UNK UNK UNK" ๐Ÿ˜…

@janellecshane That's really entertaining! Can anyone have conversations with Visual Chatbot?

@dominicduffin1 yes! Link near the beginning of the post. If it waits forever without giving you a caption, reload/try again later.

@janellecshane I read the blog post but, after playing with the chatbot a little myself, are you sure it's not just badly made (or, as you suggest, answering randomly)? I tested by just typing random letter strings, to which it always replied "yes."

In one interaction it appeared to be gaslighting me, though, so maybe it really is just a lying liar that lies.

@ink_slinger if you give it pedestrian datasets, like people holding snowboards or cats, it's not bad. And there are quirks & patterns. Try asking it how many giraffes there are.

@sconlan @janellecshane Yeah, the captions can be chalked up to a lack of diversity in the source material it was trained on. The "conversation," though, seems almost completely random.