A New Search Engine
Repurposing the best knowledge aggregator on the internet
I stumbled upon Paul Graham’s essay How to Start a Startup as a high schooler prepping for university, trying to figure out what to do with my life. Since then, I have been reading Hacker News voraciously and it has been instrumental in my intellectual growth.
The quality of the site’s content is unparalleled. The community is unique in keeping information true and discussions civil. This value was accentuated by the fallout of social media, fake news, and the migration of new readers due to the pandemic.
It is a great force for good, but can it become more? This is a challenge for Hacker News to evolve into a full-fledged product to better serve the world and maybe, shape human education and knowledge on its way.
A Growing Concern
According to Edward’s book “I am feeling lucky,” Google’s early motto “don’t be evil” promised to put users first by refusing to serve ads in its search results, unlike its early competitors. This is what gained them user’s trust and market share, ultimately becoming the world’s search engine. Since then, its motto has changed to “do the right thing”, and evolved to today’s money-hungry, marketing machine.
The crux in “the right thing” is that it is subjective, depending on who defines it. Decades of covert misinformation and manipulation have created a culture of “relative truth and morality”, atrophying our generation’s critical thinking skills. In a post-truth world, money dictates what is true. Eventually any site based on the advertising business model gets degraded by becoming promotional tools: Facebook, Quora, Instagram, Twitter, Reddit, etc.
However, Google’s unique position as a monopoly on the world’s knowledge dangerously dictates our thoughts. It has betrayed user’s trust, becoming a brainwashing machine where corporations can pay to manipulate, define and censor truth. Google search has increased the number of ads in its top results, propagating misinformation with marketing campaigns masqueraded as articles. The product has been deteriorating, yet users have little choice. Something needs to be done; we cannot let them corrupt our minds.
An Apparent Solution
This is where a community largely devoid of commercial interest can fill the void. Hacker News is not perfect, but the diligent work of its moderators and community avoids the above degradation over time. What makes HN strong is its collaborative nature based on intellectual curiosity, democratizing the scientific method to find truth.
With a repository of curated articles collected since 2007, and with better sources and vetting than Google, can it be an alternative to the search giant? A search box by Algolia is already offered at the bottom of the page. It would not be hard to index the content and present it in a better UI.
Instead of relying on backlinks, comments and votes could be used to determine the ranking of search results. Sentiment analysis could be implemented to filter content and verify the veracity of each link. Recommendations based on natural language processing of analogous search terms can provide a discovery rather than a promotional experience. Translations can expand HN’s value internationally. Even minimal social elements could encourage constructive discussions for intellectual development.
People underestimate the value of good analysis. When an article has been vetted by thousands of intelligent actors from diverse fields, veracity and insights can be far superior than what any AI can offer. It is like having peer reviewed articles evaluated by a consortium of crowdsourced geniuses, with a punishment/reward system that keeps the shortcomings of human bias and greed in check. Compare this with Google, where the wisdom of the crowd is often influenced by popularity or profit, with AI augmenting the darkest parts of human nature. No wonder people resort to workarounds.
Trained a scientist in search for truth, I am eager to see this solution come to fruition. If anyone is willing to help, I am open to share ideas in order to explore the possibility of building and testing a prototype. Something needs to be done to counter Google - no alternatives cut it so far.
Would love to hear your feedback.
Recommended book: The Truth Formula
Bonuses
1. Google Manipulated Digital Ad Prices and Hurt Publishers
2. ChatGPT and Bing
3. Losing Trust in Google
