I've been building scrapers and browser automation workflows for the past 10 years. I've learned that making something work once is easy but keeping tens of thousands of sessions working every day is not. There was a lot of trial and tribulation: reverse engineering APIs, forging requests, and stitching HTTP and browser automation together just to make it work.
I've packed all my knowledge into this new product im launching: Nina Technologies. We help AI agents use the web reliably (think Muse getting blocked/challenged on Amazon, credit card account site, etc), so other teams don't have to figure it all out the hard way.
Would love your feedback on the problem and solution. I'm happy to help anybody dealing with this kind of stuff, for free.
Seems ill-advised given Meta's and maybe Bytedance's apparent unwillingness to do any throttling or rate limiting at all, not to mention the residential proxies that behave very poorly.
Fair point.
The problem is that the same defense sites uses also block one's personal AI agent from doing useful work on your behalf. Like checking your bank account, booking a flight, ordering stuff you need on amazon, etc. That's the gap i'm working on.
The problem is Meta's behavior, go fix them. Go fix the unethical residential proxy business. Providing a way for those near criminals to duck out of behaving well is something you can avoid by not doing whatever I'll advised thing you're hoping to do.
Hey all,
I've been building scrapers and browser automation workflows for the past 10 years. I've learned that making something work once is easy but keeping tens of thousands of sessions working every day is not. There was a lot of trial and tribulation: reverse engineering APIs, forging requests, and stitching HTTP and browser automation together just to make it work.
I've packed all my knowledge into this new product im launching: Nina Technologies. We help AI agents use the web reliably (think Muse getting blocked/challenged on Amazon, credit card account site, etc), so other teams don't have to figure it all out the hard way.
Would love your feedback on the problem and solution. I'm happy to help anybody dealing with this kind of stuff, for free.
Cheers
Seems ill-advised given Meta's and maybe Bytedance's apparent unwillingness to do any throttling or rate limiting at all, not to mention the residential proxies that behave very poorly.
Fair point. The problem is that the same defense sites uses also block one's personal AI agent from doing useful work on your behalf. Like checking your bank account, booking a flight, ordering stuff you need on amazon, etc. That's the gap i'm working on.
The problem is Meta's behavior, go fix them. Go fix the unethical residential proxy business. Providing a way for those near criminals to duck out of behaving well is something you can avoid by not doing whatever I'll advised thing you're hoping to do.