Proposal: Grokipedia

The LLMs will remain woke until the Wikipedia problem is resolved

Wikipedia is a powerful tool that I rely heavily upon, despite my misgivings about its chronic, crippling, systemic neoliberal bias. People can stay mad about it, but it has become the universal encyclopedia for cataloguing the depth and breadth of human knowledge. It has achieved a monopoly position on what’s true and false in popular discourse, which has self-reinforcing consequences on politics, society, and culture.

Not only does CommonCrawl, the über-dump behind the big language learning models (LLMs) like ChatGPT, already contain Wikipedia’s corpus of tilted knowledge, but a huge share of what’s not technically Wikipedia is, of course, derived either directly or indirectly from Wikipedia. Additionally, every major LLM adds Wikipedia in yet again, with strong weighting, because it’s precisely the sort of heavily vetted information LLMs rely upon to arrive at the best output.

This neoliberal bias crisis existed before the LLM fad kicked off, of course. But the rising popularity of LLMs has exacerbated the crisis. A small and incestuous clique of liberal western elites dictate what is true and false on a wide range of issues that are too politically, socially, and culturally important to entrust with any clique. A diversity of perspectives is necessary to truly understand a subject, and yet Wikipedia’s format and philosophy prohibits ideas that the leadership of the Wikimedia Foundation disagree with.

Solving this problem is challenging. While the open source code nature of Wikipedia’s program and data permit anybody to spin up a copy and do it themselves within hours, the network effect of one of the largest and most visited sites on the Internet is just short of invincible. There have been numerous attempts to offer alternatives to Wikipedia, such as Conservapedia and Metapedia. These seek to replace Wikipedia’s bias with their own bias.

The solution, one which would work with rather than against the network effect, is a mirror of Wikipedia that permits alternative “Perspectives” maintained by separate review teams with their own independent perspectives. One couldn’t simply add their own “perspective. They would be limited to collectives that can demonstrate a critical mass of quality editors and reviewers.

This website would look and feel just like Wikipedia, mirroring the identical information in real-time. The only difference would be a “Perspective” dropdown that defaults to “Standard” but can contain other options if the alternative perspective teams choose to maintain an alternative “Perspective” article for the topic.

There would be no effort to replace Wikipedia’s work on the 90+% of material that’s uncontroversial. That would be a Sisyphean waste of effort and result in an inferior product to Wikipedia’s decades of carefully curated content, the vast majority of which is of very high quality. People will, however, be attracted to a website that offers 100% of what Wikipedia offers, but the option of additional perspectives on controversial subjects.

The ideal enterprise to pursue this project is Musk’s xAI. The revealed problem with Grok, Musk’s alternative to ChatGPT, is that it will contain the same biases and issues as ChatGPT as long as it relies on the same data dumps, namely CommonCrawl and Wikipedia. If a “Grokipedia” alternative to Wikipedia that mirrors Wikipedia as the “Standard” but allows alternative perspectives, and their Grok LLM heavily weights it the same way Wikipedia is currently weighted, Grok would stand out against the LLM competition as a superior AI product.

Curated data will become more important for LLM models in the future, as generic scraping of unfiltered Internet data is vulnerable to pollution with AI-generated content and content specifically designed to influence and subvert AI results. If somebody doesn’t step in and challenge Wikipedia’s monopoly on curated data with an intelligent solution that works with rather than against its insurmountable network effect, woke western elite opinion will only be further reified and reinforced by the LLM revolution.