Yeah, of course we are paying for the compute on our end. I think we are waiting for our users to stress test our theory, however our secret sauce is optimizing inference so that this is a feasible product on both ends.
Like I alluded to in my initial comment, I've never understood why the inference providers of today charge per token. Whichever marketing department got our industry to be ok with being charged for the output of a REST call, I applaud.
I'm not sure it will all hold up at scale, but I am still waiting for the test to break. It helps to be a curious and optimistic person in this endeavor.
Not currently implementing any rate limits. Using an assortment of open models.
I have never understood the per token pricing model, so I built WavexAI to see what I was missing. Maybe I'll find something at scale, but turns out you can provide fixed cost inference without the fear of running out of tokens.
What's your cost model / pricing based on here? If you're going to provide $20/month for unlimited APi calls, I can see this getting abused very reasily.
Pricing is largely based on how many users we think we can fit per our baseline modset, and how much the cost of that modset is determines where we can work back from in price. From there, we would scale on a per modset basis.
It may not turn out to be a perfect science, but in the name of shipping something and getting feedback, it's out there
It is very new (launched a couple days ago), so still trying to figure out how to get it out there. I'm a software nerd not a marketing genius unfortunately.
As for how it works, its a pretty standard inference provider, where we offer a chatbot and an API, at unlimited usage for a fixed cost. Since its very new, we're still wondering if there's something we're missing as for why fixed cost AI is not the norm.
Hoping it is similar to the music industry, where you once had to pay $1.99 to download a song, and now you can stream all you want for $12.99. We want to begin that new trend.
You are yourself in some pay paying for the compute right? Isn't this fix price an opportunity for many to abuse?
How are you managing rate limits and what model are you using? if you would like to disclose that.
Yeah, of course we are paying for the compute on our end. I think we are waiting for our users to stress test our theory, however our secret sauce is optimizing inference so that this is a feasible product on both ends.
Like I alluded to in my initial comment, I've never understood why the inference providers of today charge per token. Whichever marketing department got our industry to be ok with being charged for the output of a REST call, I applaud.
I'm not sure it will all hold up at scale, but I am still waiting for the test to break. It helps to be a curious and optimistic person in this endeavor.
Not currently implementing any rate limits. Using an assortment of open models.
I have never understood the per token pricing model, so I built WavexAI to see what I was missing. Maybe I'll find something at scale, but turns out you can provide fixed cost inference without the fear of running out of tokens.
Would love feedback, and testers on the site!
What's your cost model / pricing based on here? If you're going to provide $20/month for unlimited APi calls, I can see this getting abused very reasily.
Pricing is largely based on how many users we think we can fit per our baseline modset, and how much the cost of that modset is determines where we can work back from in price. From there, we would scale on a per modset basis.
It may not turn out to be a perfect science, but in the name of shipping something and getting feedback, it's out there
Sorry, but what's a "modset"? I _assume_ you're renting GPU(s) from a IaaS provider?
What models are available for use?
How does it work and how are you getting more people to discover it
It is very new (launched a couple days ago), so still trying to figure out how to get it out there. I'm a software nerd not a marketing genius unfortunately.
As for how it works, its a pretty standard inference provider, where we offer a chatbot and an API, at unlimited usage for a fixed cost. Since its very new, we're still wondering if there's something we're missing as for why fixed cost AI is not the norm.
Hoping it is similar to the music industry, where you once had to pay $1.99 to download a song, and now you can stream all you want for $12.99. We want to begin that new trend.