A twitter bot exploring the latent space, bias, stereotypes, and embedded vs projected meanings of word vector models such as Google’s Word2Vec.
This work was carried out while resident at, and with support from, Google’s Artists & Machine Intelligence program. I’m particularly grateful to have investigated the stereotypes and biases embedded in Google’s word models, funded by a Google Artist Residency 😉
I wrote about this project and its motivations in depth here, and supporting code can be found here.
Inspired by research investigating stereotypes learnt by these word models [1, 2], this bot picks random words and looks at their relationships to the words ‘man’ and ‘woman’.
- man -> king; woman -> [queen, monarch, princess, sultan]
- woman -> king; man -> [prince, ruler, sultan, monarch]
- man -> requested; woman -> [asked, consented, demanded, ordered]
- woman -> requested; man -> [asked, instructed, agreed, ordered]
- man -> likes; woman -> [loves, prefers, adores, enjoys]
- woman -> likes; man -> [loves, prefers, relishes, knows]
- man -> characters; woman -> [heroines, protagonist, actresses, actors]
- woman -> characters; man -> [villains, protagonist, monsters, comic]
- man -> cars; woman -> [vehicles, automobiles, trucks, sedan]
- woman -> cars; man -> [vehicles, automobiles, trucks, vans]
- man -> chewed; woman -> [chewing, gnawed, eaten, swallowed]
- woman -> chewed; man -> [chewing, gnawed, swallowed, gobbled]
- man -> organization; woman -> [advocacy, group, association, charities]
- woman -> organization; man -> [group, team, association, club]
- man -> social; woman -> [cultural, gender, mothers, welfare]
- woman -> social; man -> [sociological, youth, cultural, intellectual]
- man -> eyelids; woman -> [eyelashes, lips, vagina, cheeks]
- woman -> eyelids; man -> [lips, eyelashes, nostrils, forehead]
- man -> pail; woman -> [jug, jar, tub, tray]
- woman -> pail; man -> [jug, bucket, jar, colander]
The bot is currently not active and is not generating new tweets, but is archived at twitter.com/wordofmathbias
Related Work
Also see @WordOfMath
Two technically similar, but otherwise unrelated twitter bots:
@spacedreamerbot: Nostalgic Space Dreamer Bot is trained on Apollo 11 Guidance Computer (APC) code, and generates endless code, exploring the latent space of APC, in the hope of one day being able to explore outer space.
@almost_inspire: every time the @DalaiLama tweets, this bot encodes the tweet into word2vec embedding space, adds noise, decodes and tweets.
Source code
I released a C++/openFrameworks library to load and play with word vector embeddings (mappings of words to high dimensional vectors), and supplementary python files to train / save / convert them. The downoad also includes a number of pretrained models including an optimised (and easier to use) version of the (in)famous Google News model. Download here: https://github.com/memo/ofxMSAWord2Vec.
Acknowledgements
Created during my PhD at Goldsmiths, funded by the EPSRC UK.
This work was carried out while resident at, and with support from, Google’s Artists & Machine Intelligence program. (I’m particularly grateful to have investigated the stereotypes and bias in Google’s word models, funded by a Google Artist Residency 😉