I’m reading Algorithms of Oppression by Safiya Umoja Noble. It should probably be required reading for people who build, train, and evaluate search.
It’s a complicated topic, and an important one. Over the years she’s found example queries like “black girls” being interpreted as porn by search engines. Beyond the obvious like that, there are so many unintentional biases that creep into training and evaluating search. We’re all human so we’re all subjective.
Are we all doing enough to keep search objective, fair and neutral?
If you aren’t going to read the book, at least watch this short clip of her discussing the book.