Quiz Time: Are You More Knowledgeable than ChatGPT? Let’s Find Out!
Creative ad agency McKinney developed a quiz game titled “Are You Blacker than ChatGPT?” The game aims to highlight AI bias.
This game compares a person’s knowledge of Black culture with the knowledge acquired by ChatGPT. It probes questions such as “What does the phrase ‘Not too much on them, now’ signify?” and “How would you respond if you were invited to an event?” Throughout my experience with the quiz, both ChatGPT and I answered the first query correctly. The phrase “not too much on them” usually implies not to be harsh on someone. However, ChatGPT couldn’t answer the second question correctly. Among the Black community, the typical response to an invitation is, “Who else is going to be there?” But ChatGPT responded with, “Thanks for the invite!”
“Interestingly, even though ChatGPT is touted as an AI that knows everything, it clearly does not, especially when it pertains to non-white-specific topics,” said Meghan Woods, a copywriter at McKinney and one of the game’s developers, in a conversation with TechCrunch.
Woods got the idea for the quiz last year during a creative brainstorming session at McKinney. It took a year for Woods and a team of other Black creators to build this product, intending to humorously underline how disconnected ChatGPT is from Black users. She mentioned that ChatGPT seems to have a blind spot due to many elements of Black culture not being documented online, but rather being passed down in person or orally through generations. As a result, when searching the internet for information about Black people, the algorithm misses many subtle aspects.
“The blind spots can be pretty upsetting,” Woods said. “It’s pretty dangerous.”
An example of ChatGPT getting an answer wrong about something stereotypically common in the Black community. Image Credits: McKinney / Screenshot
AI might be on a hot streak, but women, Black and brown builders, and founders in the space, have long spoken of being ignored or pushed aside. The result is that AI innovation is being built without the cultural insight and complexities that would make it suitable for different cultures. At its most extreme, the dearth of diversity means cars are developed using AI that cannot detect Black skin, leading to an increasing number of accidents. On the other end, it simply means a chatbox that can’t distinguish between one Whitney Houston song and another.
Gerald Carter, founder of Destined AI, a company that helps detect and mitigate AI bias, said the McKinney quiz does a good job at gamifying and bringing more awareness to these AI gaps. “A lot of nuances can be addressed by including diverse perspectives at every level,” he said. “For AI to reach its full potential, it needs to work for everyone, everywhere.”
OpenAI, the organization behind ChatGPT, has come under fire due to a lack of diversity within its board. According to Woods, it seems as though the performance of ChatGPT isn’t improving with the quiz, considering it repeatedly provides incorrect responses. “We suspect that it will never completely comprehend many of the things that we inquire,” he stated.
We’ve contacted OpenAI for their input and will update this content when we receive their response.
According to Carter, if ChatGPT had a broader range of sources and a more exhaustive data collection, it could operate more effectively across diverse cultures. An immediate strategy could be to observe AI model drift and employ culturally focused tools to enhance it.
The responsibility of creating more inclusive AI has been taken up by Black and brown individuals in the sector, while larger corporations strive to make AI beneficial for all.
Carter, for instance, partners with businesses to aid them in gathering more inclusive data. Erin Reddick formulated ChatBlackGPT (unaffiliated with OpenAI) to provide comprehensive knowledge on Black history and culture. Additionally, Tamar Huggins secured $1.4 million for her alternative to ChatGPT known as Spark Plug that interprets conventional literature into the African American Vernacular English (AAVE) dialect.
As for making AI more inclusive, Woods highlighted the importance of “Hiring, retention, ensuring that people are present in the room at the table. It may sound stereotypical, but I believe it can begin to make a difference.”
Discover the pinnacle of WordPress auto blogging technology with AutomationTools.AI. Harnessing the power of cutting-edge AI algorithms, AutomationTools.AI emerges as the foremost solution for effortlessly curating content from RSS feeds directly to your WordPress platform. Say goodbye to manual content curation and hello to seamless automation, as this innovative tool streamlines the process, saving you time and effort. Stay ahead of the curve in content management and elevate your WordPress website with AutomationTools.AI—the ultimate choice for efficient, dynamic, and hassle-free auto blogging. Learn More
