Not a technical person, as I understand it, Google and Bing are full search engines while options like Seaexng, Ecosia, Duckduckgo are meta search engine that depend on Google or Bing. Are there other search engines that are fully independent, and if there aren’t, what are the barriers that stop other search engines from emerging
Yes there is. Kagi is probally the most well known I can think of, and even that still uses Google and Bing as secondary sources.
The biggest barriers is data/context. The biggest being the primary index.
Google has a lot of web scrapers/indexers and also offers hosting platforms. They also partner with big hosting companies for index trees to be able to easily show web sites reliably AND have been around for years finding it.
This is sadly also one of the primary damages that AI is currently doing to the internet field, because not only is it decreasing web traffic for web hosts due to AI summaries and searches, but it’s also forcing web hosts to have to block or restrict indexers. Which is making it even harder to establish search platforms. Because these same agents are abusing the user agent system to try to pretend that it’s a normal indexer, so web hosts are faced with either having their platform spammed so many bot traffic that it takes their website down, or block indexers, which means that they don’t appear in web searches. It’s a lose lose.
This is sadly also one of the primary damages that AI is currently doing to the internet field, because not only is it decreasing web traffic for web hosts due to AI summaries and searches, but it’s also forcing web hosts to have to block or restrict indexers
Not to mention the damage it’s doing to content quality, as websites become increasingly written in a way that’s meant to optimise for AI placement, and are increasingly being populated with content written by an AI in the first place. Information, written by machines, for machines. It’s depressing as hell (and hard to see how it ultimately helps engage real actual people, who seem to be an afterthought sometimes).
Thanks for this. Its very detailed. I heard abt Kagi but I thought it was a meta search engine, thanks for the clarification.
Thanks for letting me know about the barriers ad well and the current damage ai companies are doing. Hopefully, we’ll think of something to beat back the ai companies and Google
Mojeek is an option I’ve not seen others mention:
Kagi has independent indexes.
Kagi Search Sources( source ):
Kagi is known for delivering a unique flavor of high-quality search results, sourced from our own web index (internally named “Teclis”) and news index (internally named “TinyGem”). Kagi’s indexes provide distinctive results that help you discover non-commercial websites and engage with “small web” discussions surrounding a particular topic.
We don’t stop there; we are always trying new things to surface relevant, high-quality results. For example, we recently launched the Kagi Small Web initiative, which showcases content from personal blogs and discussions around the web. Discovering high-quality content written without the motive of financial gain gives Kagi’s search results a unique flavor and makes it feel more humane to use.
Our search results also include anonymized API calls to all major search result providers worldwide, specialized search engines like Marginalia, and sources of vertical information such as Wolfram Alpha, Apple, Wikipedia, Open Meteo, Yelp, TripAdvisor, and other APIs. Typically, every search query on Kagi will call a dozen or so different sources simultaneously, all with the purpose of bringing the best possible search results to the user in a split second.
Our unique algorithms down-rank pages with a lot of ads and trackers (which we have found correlate with a decrease in content quality) and promote content from independent, ad-free sources and personal websites. This ensures that Kagi shows results that delight users and are worth paying for. Subscriptions from our members pay for search results, allowing Kagi to remain ad-free and 100% privacy-respecting.
Yeah. I just… it costs too much. I can’t justify it, even though I tried. It should be 1000/mo. for $3. That I would pay.
I think part of why the internet sucks now is we’ve grown accustomed to too much for free.
The $10 I spend on Kagi is the best subscription I have, and one of the only ones I still keep.
Your search engine in a very real way is your internet. Nearly everything you see online starts there.
If you aren’t paying for it, someone else is. And the reason they’re paying, is to make sure you get the internet they want; Not the one you want. If you want the internet to be what you want, you have to be the one to pay for it.
I don’t know where you live, but in most of the developed world $10 is roughly the cost of a single lunch. Not even a fancy one. What’s worth more to you? An inexpensive lunch? Or making sure the internet you see in your search, is what you want to see, instead of what someone else wants you to see?
That’s actually a good way of quantifying the price.
QFT right here. I just started using Kagi this month after being very impressed by the trial period. The $5/month starter plan gives an average of 10 searches per day. It doesnt seem like much but that’s a decent amount of searching for my personal use case. Truth be told I’ll probably end up going for the family plan at $20/month. Up to six users with unlimited search for less than a Netflix subscription.
I think the best model would be that you top up an account and pay some small amount per search. That’s also a bit more transparent, and directly linked to use
There’s also YaCy, which is a peer-to-peer search engine. You index whatever you want, and when you search, you connect to lots of other indices as well as your own.
I want to set up my own node at some point. My understanding is that the search results are hit and miss, but you have the power to improve things by building your own index of pages you want to search regularly. And this benefits everyone else too.
Since search indexing is so intensive, I think it’s the right technical solution if you want a truly open and independent alternative.
Thanks for sharing! This my first time hearing about YaCy. Im always excited about peer to peer, and I hope this or a similar solution grows in popularity. About to try it now!
Ecosia at least partially uses the European Search Perspective (EUSP) search index which they develop in collaboration with Qwant.
Ecosia and Qwant have their own search index.
Well, you gotta index the whole internet
Before you can make an apple pie from scratch. Or something.
I’ve not used it, but have heard decent things about Kagi, a paid search engine. Supposedly, it finds things like how old Yahoo or old Google worked, without AI (but is optionally available?), and no ads.
I would think the major barrier to entry is the business model: ad revenue goes to those that can deliver results. Google AdSense has dominated that realm for years, so it would take a major upfront investment to challenge them on that. Not much different than how it’s hard to compete with established airlines to a particular airport that they already serve. Economies of scale tend towards consolidation.
Someone recently posted this:
The usual example is Kagi.
I made a video about a few others this week: UDM-14 (Though, note that UDM-14 is basically just google search with a different overlay and no ads/ai), Mojeek, and Marginalia. (Though Marginalia focuses on non-commercial content and is a little different.)
I’ve been using Lite Duckduckgo. It gives 10 text link per page. No video, shorts, pictures, ads. Just link. Someone on lemmy mentioned it and I’ve been using it since.
DDG uses Bing though and doesn’t have its own index.
That’s a good thing to know about!
deleted by creator
yandex maybe, i think mojeek is also independent
Brave delivers results from its own, built-from-scratch index.
Lemmy has an unhealthy obsession with shiting on Brave. (Responding to your downvotes)
Brave Search privacy notice
Brave Search is designed to be private by default. We don’t collect personal information about you, your device or your searches. We also don’t transmit information to the web that could be used to profile you or track you or learn anything about you. Your searches are private to YOU.
