I search for words, can even indicate I want search results with a keyword included and it will be ignored. And then I have to sift between what is the search result, and what is an ad.
And if I get another quora answer....
But, this post? it was a waste. We do some hand wavy stuff, come try us.
Fair point, I probably should have provided more context in the post.
MatterRank uses LLMs to rank pages based on criteria you provide it with, not SEO tricks. It’s not meant to replace Google, but helps when you're looking for something specific and don't want to wade through tons of results that you don't care about. Still early, but useful for deeper searches.
Transformer models like BERT have been lurking in Google search for a few years now (and non-transformer language models before that). The distinction between LLM and chatbot is pretty thin. Dismissing "chatbots with web access" when you are actually using LLMs is not a very clear or useful differentiation, even if the way you use a LLM really is different. More end-user control over results is a good thing, but there is an opportunity to engage much more clearly.
> I guess there's just too much data now to search stuff like that.
That seems extremely unlikely as the reason.
It's far more likely that some executives looked at the numbers and decided that removing search operators would make people more likely to click on ads, while leaving them in would make people click on the actual results that they were searching for.
i was being glib. Of course it's possible to have functional search for people that remember the good old days of google dorks and switching search engines to find deeper links. That's gone away, for the reasons you and sibling(s) mentioned - ad revenue.
I am unsure if it is possible to run a "free" web search without having a benevolent benefactor paying for the scraping and maintenance and staffing. Furthermore, someone has to play mouse and mousetrap with the "gaming" of whatever "algo" one chooses to use to rank results. Maybe a list is the wrong way to display search results. maybe a contemporary snapshot of the page with the search text highlighted might work better. It might even convince a lot of sites to clean up their landing pages and their blog/article formats.
I know how to stand up and start a web search engine, and probably could implement a decent chunk of functionality myself. it'd be slow and fall down if 100,000 people hit it at once, but nonetheless, the hard part isn't getting one running and starting the scraping. The hard part is results and funding.
I envisioned, last night, in a fever dream: maybe some metadata that the crawler and the sites share, to encourage Value for Value. If a site is willing to be scraped, but would like some nominal bandwidth costs recouped, or perhaps some sort of data agreement that is mutually - mutually - beneficial; or a site like NYT chipping in to the search hosting costs if the search company has really good results, like better than NYT could implement, then there could be some value for value there, too.
Search engines provide a valuable service for humanity, as a general concept. Search engines as they exist now provide a valuable service for their shareholders. remove the shareholders, make the service valuable for humans, and the human stakeholders in the search company (employees, vendors, etc) might not be so greedy or "legally obligated to make numbers go up".
Encarta and Britannica existed. Wikipedia exists - as well as the forks and archives.
Does it? I understand there are issues with spam in search, but assuming we don't know what we want is not at all the conclusion I draw from using search engines.
Yeah that's fair, "doesn't know what we want" might have been oversimplifying. Better phrasing would have been that there is a very hard limit on the context you're able to give when using a search engine. It's mainly keywords, and then maybe some tricks like `site:` or quotes.
I think you're right that there's limited context, but I'd still disagree on "doesn't know what you want". I think search engines know what users want within the scope of the context they're able to provide. There are two issues with that, one is the deeper examples you gave in another reply may be better, and the other being differentiation between legitimate search matches, and bad actors trying to match for things they shouldn't do.
For the former, I'm intrigued but unconvinced that it's what I actually want in a search engine.
For the latter, I imagine that's something that this search engine will need to contend with, although it could "just" be an LLM compute trade-off, where if you give enough results to an LLM to analyse you'll eventually find the good stuff. That said, SEO is going to rapidly become LLMEO and ruin the day again.
Credit to @ziftface — I should’ve included more examples in the original post. MatterRank is useful when you want results with specific qualitative traits that go beyond keyword matching. You can ask for stuff like “written by a woman,” “mentions these specific lines from a movie,” or “talks about X/Y/Z but avoids A/B.” Since it reads the full content, not just metadata or SEO signals, it lets you be a lot more precise in ways that traditional search engines just don’t support.
Is this kind of promotional post even allowed? It doesn't have any actual content that discusses how technically to make search better, only that MatterRank has solved it. If doing content marketing, remember to include some content.
It doesn't even explain why it's better than Perplexity.
Using an LLM isn’t the worst way to rank, but it’s pretty darn slow. The speed could be improved a lot by just distilling into deep neural nets though.
The results for me were fairly high quality and moderately relevant but I think they could be improved as well.
You get pretty far by just blocking low quality blogspam and Medium, which would be a lot faster and could even be done on the frontend with a chrome plugin.
Yeah LLMs were the easiest way to get a proof of context running, but replacing it with a specialized distilled model/classifier should hopefully make it way quicker.
As for the results, it's tough because we've made the deliberate decision to have no control over the reranking. What that means is that if your criteria is "written by a woman", for instance, then any result that meets that will be ranked equally at the top. In all engines I've built for myself, I have a relevance criteria that's weighted relative to how much I care that the result is exactly what I'm looking for. It's probably important to make that clearer to the end user.
I mean, it's not just search that assumes we don't know what we want. A huge amount of technology these days has shifted to telling us what to want rather than letting us obtain what we have independently decided we want.
I actually completely agree with this. Search is a good example, but in general it seems that general consensus has become that consumers don't know what they want, which is pretty frustrating, and probably a product of the success of the TikTok algorithm and similar software.
I'm hoping that as LLMs become more mainstream more functionality is built into tech that doesn't treat consumers as idiots. This is one stab at it, but there's so many other opportunities imo.
Because LLMs understand language, we can start building algorithms that respond to what users say they want. Instead of reverse-engineering user intent from behavior, you can just tell a system “more of X, less of Y” and it listens. Way more flexible than hard-coded workflows.
Interesting. That doesn't align with my experience with LLMs. I tend to find "smarter" interfaces (like LLM-based ones) more frustrating because they are black boxes and I find myself struggling to understand how to get what I want from them. I've had a fair number of maddening conversations with LLMs where I ask them for something and they just regurgitate non-answers back over and over.
What I prefer is interfaces that are more systematic and based on comprehensible principles. Like, for search (as someone mentioned in another comment), I want to be able to search for pages (or records, or whatever) that contain the text I searched for. I don't want an interface that tries to understand what I mean, I just want it to use the data I give it in a way that's deterministic enough that I can figure out how to make it do what I want.
I think in a lot of cases that's because the meta with LLMs right now is to "have them do things for you", which generally means that obfuscating what's actually happening behind the scenes can make them seem "smarter" to the average user. Also, engineers are used to full control over deterministic input-output pipelines, which is a framework they try to force on LLM applications that fails miserably for the reasons you've listed.
In my opinion, the best applications of LLM UX will have full clarity for the end user (something we're trying to do with MatterRank). The non-determinism should be something the user can control to get better results, not something the engineer has prompted that takes control away from the user.
Now, if the use case you're looking for is "give me results with x text", then yes I agree with you that LLMs are just getting in the way. But that's not always the case.
MatterRank is pretty slow still since it runs LLM evaluations on each webpage as markdown content. Wouldn't really consider it a Kagi alternative (which I haven't used but have heard great things about!), as that's more of a search engine in the traditional sense.
I think where MatterRank shines right now is for finding results where you wouldn't mind waiting an extra 20-30 seconds for an added layer of vetting, as opposed to just wanting a quick answer.
Having said that, we are definitely working on making it faster and more useful for everyday queries.
> I think where MatterRank shines right now is for finding results where you wouldn't mind waiting an extra 20-30 seconds for an added layer of vetting, as opposed to just wanting a quick answer.
I've not used it, but anecdotally, I can refine my own search query to get what I want, or conclude it doesn't exist, within 20-30s. Assuming ~5s per search to write, search, read, decide, that's 4-6 searches.
Do you think you're getting more value than 4 iterations on the initial search term? Are you always getting it in one search, or do you end up still needing to refine the search term, extending it beyond that 20-30s?
> Do you think you're getting more value than 4 iterations on the initial search term?
Definitely not for all cases, but in some cases yes. Where it really makes a difference is when you're looking for qualitative attributes of the webpage, rather than what words show up in it (e.g. “written by a woman", "is likely to convince someone who supports Trump", "talks about X/Y/Z but not A/B.”) It reads the actual content, so you can get oddly niche in a way you just can’t with keywords alone.
Give me a way to filter out results with ads on them please.
Edit (hn doesn’t let me post this fast): is finding places to buy shit really an issue? How many times in your life have you thought “damn I know what I want to buy, I just don’t know from which site to buy it”? That’s hard to imagine of anyone. This user story just seems like a problem made up by search indexes to court capital.
If you're referring to sponsored content, then you can actually use MatterRank to configure an engine to devalue content that's trying to sell you something or is sponsored.
If you mean pop-ups, MatterRank can't handle that at the moment because it evaluates markdown content, but it's something we're looking at adding. In the meantime, I'd recommend a good ad-blocker.
I don't want to argue for ads, but, as someone trying to cut out Amazon from my life, it genuinely is hard to figure out where else to buy stuff these days. Slightly less common things that nonetheless used to be stocked in any halfway decent electronics store just aren't any more.
Would you mind giving more details about your use case? I'm curious.
I've been Amazon-free for a while and generally I've had very good luck simply going directly to manufacturer's websites, but it seems like you might be searching for a class of products for which that strategy is ineffective?
> Would you mind giving more details about your use case? I'm curious.
That would definitely have enriched my comment, but, unfortunately, I couldn't think of anything in particular in the moment, and can't now. seb1204 (https://news.ycombinator.com/item?id=43564922) mentions one common kind of use case for me: I want to buy some small utility item that I'm used to finding in the hardware store, but it's sufficiently specialty that it's not worth it for the hardware store to carry it, and it's sufficiently small that its price would triple or quadruple if I paid the manufacturer's shipping costs.
> I've been Amazon-free for a while and generally I've had very good luck simply going directly to manufacturer's websites, but it seems like you might be searching for a class of products for which that strategy is ineffective?
As I say, I'm stuck in the situation of being vague because I can't think of the last specific time this affected me, but I have definitely dealt with relatively small sellers where the purchase option on their webaite is "here's a link to buy from our Amazon store."
Conversely, let me only see sites where I can buy something. Too much of my life is consumed by trying to see if something can be bought and if so how much it costs.
This is just an ad without any context?
Search could be better? Yes, yes it could.
I search for words, can even indicate I want search results with a keyword included and it will be ignored. And then I have to sift between what is the search result, and what is an ad.
And if I get another quora answer....
But, this post? it was a waste. We do some hand wavy stuff, come try us.
Fair point, I probably should have provided more context in the post.
MatterRank uses LLMs to rank pages based on criteria you provide it with, not SEO tricks. It’s not meant to replace Google, but helps when you're looking for something specific and don't want to wade through tons of results that you don't care about. Still early, but useful for deeper searches.
Transformer models like BERT have been lurking in Google search for a few years now (and non-transformer language models before that). The distinction between LLM and chatbot is pretty thin. Dismissing "chatbots with web access" when you are actually using LLMs is not a very clear or useful differentiation, even if the way you use a LLM really is different. More end-user control over results is a good thing, but there is an opportunity to engage much more clearly.
But often the problem "is" google, garbage in, garbage out
Search could be better if it was a paid service and the content got paid.
[dead]
I want a search engine that only returns results containing words I specify. Is that asking too much? Google is not that search engine.
I think google several years ago had gotten very good at matching on related concepts, but it just fell off a cliff after that.
Maybe try Kagi. I am pretty happy with the results.
I felt the same frustration: I just want keyword matching without any filtering. I'm building https://greppr.org/ to scratch that itch.
Isn’t that what quotes do?
I feel stupid for asking, but do quotes even do anything anymore? I feel like I try them and it just gives me the same results.
For me, quotes no longer are exact match. Google Search is kind of a bust.
They always seem to work for me. I regularly over-specify obscure error messages and get no results.
On Bing I think you have to put a plus symbol immediately following the quoted word: “keyword”+
No. But it's what verbatim mode does.
remember when "Human speech" -robots -alien worked? those were the days. I guess there's just too much data now to search stuff like that.
> I guess there's just too much data now to search stuff like that.
That seems extremely unlikely as the reason.
It's far more likely that some executives looked at the numbers and decided that removing search operators would make people more likely to click on ads, while leaving them in would make people click on the actual results that they were searching for.
i was being glib. Of course it's possible to have functional search for people that remember the good old days of google dorks and switching search engines to find deeper links. That's gone away, for the reasons you and sibling(s) mentioned - ad revenue.
I am unsure if it is possible to run a "free" web search without having a benevolent benefactor paying for the scraping and maintenance and staffing. Furthermore, someone has to play mouse and mousetrap with the "gaming" of whatever "algo" one chooses to use to rank results. Maybe a list is the wrong way to display search results. maybe a contemporary snapshot of the page with the search text highlighted might work better. It might even convince a lot of sites to clean up their landing pages and their blog/article formats.
I know how to stand up and start a web search engine, and probably could implement a decent chunk of functionality myself. it'd be slow and fall down if 100,000 people hit it at once, but nonetheless, the hard part isn't getting one running and starting the scraping. The hard part is results and funding.
I envisioned, last night, in a fever dream: maybe some metadata that the crawler and the sites share, to encourage Value for Value. If a site is willing to be scraped, but would like some nominal bandwidth costs recouped, or perhaps some sort of data agreement that is mutually - mutually - beneficial; or a site like NYT chipping in to the search hosting costs if the search company has really good results, like better than NYT could implement, then there could be some value for value there, too.
Search engines provide a valuable service for humanity, as a general concept. Search engines as they exist now provide a valuable service for their shareholders. remove the shareholders, make the service valuable for humans, and the human stakeholders in the search company (employees, vendors, etc) might not be so greedy or "legally obligated to make numbers go up".
Encarta and Britannica existed. Wikipedia exists - as well as the forks and archives.
It's not only actual clicks, it's impressions too.
make search worse = more searches are performed = more ads shown
It's anti-consumer but every company is like this now.
Use Google verbatim mode.
https://www.google.com/search?q=beer&tbs=li:1
That is unbelievably better. The ads are even off to the side!
I have been playing with idea of one big SQLite for domains. I can search it relatively fast, find things related to "amiga", "emulator" etc.
https://github.com/rumca-js/Internet-Places-Database
I must admit, that this is a difficult task. There are many domains for "hotels", "casinos", so I have to protect myself, just as google agains spam.
> It assumes we don't know what we want.
Does it? I understand there are issues with spam in search, but assuming we don't know what we want is not at all the conclusion I draw from using search engines.
Yeah that's fair, "doesn't know what we want" might have been oversimplifying. Better phrasing would have been that there is a very hard limit on the context you're able to give when using a search engine. It's mainly keywords, and then maybe some tricks like `site:` or quotes.
I think you're right that there's limited context, but I'd still disagree on "doesn't know what you want". I think search engines know what users want within the scope of the context they're able to provide. There are two issues with that, one is the deeper examples you gave in another reply may be better, and the other being differentiation between legitimate search matches, and bad actors trying to match for things they shouldn't do.
For the former, I'm intrigued but unconvinced that it's what I actually want in a search engine.
For the latter, I imagine that's something that this search engine will need to contend with, although it could "just" be an LLM compute trade-off, where if you give enough results to an LLM to analyse you'll eventually find the good stuff. That said, SEO is going to rapidly become LLMEO and ruin the day again.
Credit to @ziftface — I should’ve included more examples in the original post. MatterRank is useful when you want results with specific qualitative traits that go beyond keyword matching. You can ask for stuff like “written by a woman,” “mentions these specific lines from a movie,” or “talks about X/Y/Z but avoids A/B.” Since it reads the full content, not just metadata or SEO signals, it lets you be a lot more precise in ways that traditional search engines just don’t support.
Too many steps, why do I have to signup? Why do I have to create an engine.
Remove all of this, just let me directly use your app, I want to search and create engines on the fly.
I don't need to save them for future uses, if I am not going to use your app even once.
If you want this to take off, it needs to just work, no extra steps unless I want to.
Is this kind of promotional post even allowed? It doesn't have any actual content that discusses how technically to make search better, only that MatterRank has solved it. If doing content marketing, remember to include some content.
It doesn't even explain why it's better than Perplexity.
Using an LLM isn’t the worst way to rank, but it’s pretty darn slow. The speed could be improved a lot by just distilling into deep neural nets though.
The results for me were fairly high quality and moderately relevant but I think they could be improved as well.
You get pretty far by just blocking low quality blogspam and Medium, which would be a lot faster and could even be done on the frontend with a chrome plugin.
Yeah LLMs were the easiest way to get a proof of context running, but replacing it with a specialized distilled model/classifier should hopefully make it way quicker.
As for the results, it's tough because we've made the deliberate decision to have no control over the reranking. What that means is that if your criteria is "written by a woman", for instance, then any result that meets that will be ranked equally at the top. In all engines I've built for myself, I have a relevance criteria that's weighted relative to how much I care that the result is exactly what I'm looking for. It's probably important to make that clearer to the end user.
I mean, it's not just search that assumes we don't know what we want. A huge amount of technology these days has shifted to telling us what to want rather than letting us obtain what we have independently decided we want.
I actually completely agree with this. Search is a good example, but in general it seems that general consensus has become that consumers don't know what they want, which is pretty frustrating, and probably a product of the success of the TikTok algorithm and similar software.
I'm hoping that as LLMs become more mainstream more functionality is built into tech that doesn't treat consumers as idiots. This is one stab at it, but there's so many other opportunities imo.
How do you think that LLMs will help that?
Because LLMs understand language, we can start building algorithms that respond to what users say they want. Instead of reverse-engineering user intent from behavior, you can just tell a system “more of X, less of Y” and it listens. Way more flexible than hard-coded workflows.
Interesting. That doesn't align with my experience with LLMs. I tend to find "smarter" interfaces (like LLM-based ones) more frustrating because they are black boxes and I find myself struggling to understand how to get what I want from them. I've had a fair number of maddening conversations with LLMs where I ask them for something and they just regurgitate non-answers back over and over.
What I prefer is interfaces that are more systematic and based on comprehensible principles. Like, for search (as someone mentioned in another comment), I want to be able to search for pages (or records, or whatever) that contain the text I searched for. I don't want an interface that tries to understand what I mean, I just want it to use the data I give it in a way that's deterministic enough that I can figure out how to make it do what I want.
I think in a lot of cases that's because the meta with LLMs right now is to "have them do things for you", which generally means that obfuscating what's actually happening behind the scenes can make them seem "smarter" to the average user. Also, engineers are used to full control over deterministic input-output pipelines, which is a framework they try to force on LLM applications that fails miserably for the reasons you've listed.
In my opinion, the best applications of LLM UX will have full clarity for the end user (something we're trying to do with MatterRank). The non-determinism should be something the user can control to get better results, not something the engineer has prompted that takes control away from the user.
Now, if the use case you're looking for is "give me results with x text", then yes I agree with you that LLMs are just getting in the way. But that's not always the case.
Isn't it super expensive to run an LLM on each result?
kagi vs matterrank anyone?
MatterRank is pretty slow still since it runs LLM evaluations on each webpage as markdown content. Wouldn't really consider it a Kagi alternative (which I haven't used but have heard great things about!), as that's more of a search engine in the traditional sense.
I think where MatterRank shines right now is for finding results where you wouldn't mind waiting an extra 20-30 seconds for an added layer of vetting, as opposed to just wanting a quick answer.
Having said that, we are definitely working on making it faster and more useful for everyday queries.
> I think where MatterRank shines right now is for finding results where you wouldn't mind waiting an extra 20-30 seconds for an added layer of vetting, as opposed to just wanting a quick answer.
I've not used it, but anecdotally, I can refine my own search query to get what I want, or conclude it doesn't exist, within 20-30s. Assuming ~5s per search to write, search, read, decide, that's 4-6 searches.
Do you think you're getting more value than 4 iterations on the initial search term? Are you always getting it in one search, or do you end up still needing to refine the search term, extending it beyond that 20-30s?
> Do you think you're getting more value than 4 iterations on the initial search term?
Definitely not for all cases, but in some cases yes. Where it really makes a difference is when you're looking for qualitative attributes of the webpage, rather than what words show up in it (e.g. “written by a woman", "is likely to convince someone who supports Trump", "talks about X/Y/Z but not A/B.”) It reads the actual content, so you can get oddly niche in a way you just can’t with keywords alone.
It might be worth putting those examples and other use cases you've found in the post. That is certainly something Google can't do.
So there's a cross encoder of some kind which accepts a prompt for how to score?
The last thing I want is a search engine where I have to create an account to use it.
You don't have to create an account
[dead]
Give me a way to filter out results with ads on them please.
Edit (hn doesn’t let me post this fast): is finding places to buy shit really an issue? How many times in your life have you thought “damn I know what I want to buy, I just don’t know from which site to buy it”? That’s hard to imagine of anyone. This user story just seems like a problem made up by search indexes to court capital.
Edit2: Kagi is great. I'm a full subscriber.
>> How many times in your life have you thought “damn I know what I want to buy, I just don’t know from which site to buy it”?
I find I do it quite a lot. When I was researching solar. When I needed some actuators recently. Now I'm looking for a trailer. And so on.
Obviously not groceries, but whenever um investigating something new I find commercial sites to be very helpful.
If you're referring to sponsored content, then you can actually use MatterRank to configure an engine to devalue content that's trying to sell you something or is sponsored.
If you mean pop-ups, MatterRank can't handle that at the moment because it evaluates markdown content, but it's something we're looking at adding. In the meantime, I'd recommend a good ad-blocker.
Kagi lets you sort by ad/tracker count. You can also downrank or block results from sites you know to be particularly bad with ads, but good at SEO.
I don't want to argue for ads, but, as someone trying to cut out Amazon from my life, it genuinely is hard to figure out where else to buy stuff these days. Slightly less common things that nonetheless used to be stocked in any halfway decent electronics store just aren't any more.
Would you mind giving more details about your use case? I'm curious.
I've been Amazon-free for a while and generally I've had very good luck simply going directly to manufacturer's websites, but it seems like you might be searching for a class of products for which that strategy is ineffective?
A packet of rubber feet that go underneath the cast iron part on my gas stove. Add stove brand name if needed.
> Would you mind giving more details about your use case? I'm curious.
That would definitely have enriched my comment, but, unfortunately, I couldn't think of anything in particular in the moment, and can't now. seb1204 (https://news.ycombinator.com/item?id=43564922) mentions one common kind of use case for me: I want to buy some small utility item that I'm used to finding in the hardware store, but it's sufficiently specialty that it's not worth it for the hardware store to carry it, and it's sufficiently small that its price would triple or quadruple if I paid the manufacturer's shipping costs.
> I've been Amazon-free for a while and generally I've had very good luck simply going directly to manufacturer's websites, but it seems like you might be searching for a class of products for which that strategy is ineffective?
As I say, I'm stuck in the situation of being vague because I can't think of the last specific time this affected me, but I have definitely dealt with relatively small sellers where the purchase option on their webaite is "here's a link to buy from our Amazon store."
I’m surprised most manufactures even offer individual sales!
> Give me a way to filter out results with ads on them please.
Use Brave Search with Goggles (https://search.brave.com/goggles/discover). It's great.
Conversely, let me only see sites where I can buy something. Too much of my life is consumed by trying to see if something can be bought and if so how much it costs.
Was the home page vibe coded? The "Take me to Tutorial" button does nothing.
Actually the tutorial was breaking the homepage so took it down while it was being fixed. Should be back up now.