This is a good idea, although IMO source control, compute, and MLOps integration are bigger but less flashy pain points for data scientists than AI in notebooks.
If you're going to market Erdos as open source, then IMO there should be a github link somewhere on your website.
Thanks for the suggestions - we'll definitely add those to the dev list. Also, the GitHub is https://github.com/lotas-ai/erdos (and it's on the download page but a bit small).
Very cool. Any plans to add support for local models? This has what has prevented us from adopting Positron so far. We have sensitive data and sending to third party APIs is not an option (regardless of their stated retention policies).
Yeah, we just added support for local models. As I mentioned in an earlier comment, if you have a local model with an OpenAI-compatible v1/chat/completions endpoint (most local models have this option), you can route Erdos to use it in the Erdos AI settings.
When models edit the raw JSON behind a Jupyter notebook, they often mess up the cell structure by adding extra cells, misaligning code, or making bad edits. We fix this by giving the model the notebook in Jupytext format instead, which tends to make its edits cleaner and more accurate.
Give me this, but with a very efficient, opinionated path to put models into production. Give me accessible PM and customer friendly documentation about features and model choices at every stage. Make it reusable and easy to modify. Make it robust and scalable at inference time, with metrics and dashboards tracking performance over time. This seems like optimising the bit that's already fun, but I see a lot of value in hand-holding a department through all the stodgy boring bits and getting high quality analysis repeatably into customer hands.
No specific reason. Mainly because he was one of the most productive and collaborative mathematicians of all time. We actually considered "Poisson" at some point but ended up going with Erdos.
A bunch of specific things below, but the main point is that it integrates a bunch of features that data scientists use that don't come with Cursor.
Specifics (mostly reproduced from above):
1. R/Python/Julia consoles accessible by the user and AI
2. Optimized jupytext system for editing notebooks efficiently
3. Plots pane for viewing and tracking plots
4. Databases pane for managing SQL/FTP connections
5. Environment pane for managing Python/R/Julia packages and environments
6. Help pane for documentation
7. An AI that interacts with all of that.
8. Open source AGPLv3
For me, the biggest difference in the AI usage is that the AI doesn't need to write one-off python scripts for everything and run them from the terminal because it can just use the console directly.
Have you done any fine-tuning or prompt-customization for the R-specific work? I've found the models worse on R when compared to Python, especially for more complex tasks. This looks cool, thanks for sharing!
Nothing R specific. In my experience, Claude is pretty good about using tidyverse for everything. What was is flopping on for you? Our thought on not fine tuning models is that whatever comes out in 6 months is just going to be better than whatever we fine tuned.
Admittedly this was probably ~2 years ago, previous gig used a lot more R and the earlier models were (in my memory) worse in R than in Python. But makes sense that this would've come a long way. Love this!
I think Rao is more appealing to me since Positron already has that kind of integration, while RStudio doesn’t. Plus, Posit probably won’t ever add an AI Chat feature to RStudio anyway.
FWIW there's a bunch of stuff Erdos has that Positron doesn't (including having solved Positron's top 5 open GitHub issues):
1. Remote development via SSH or containers
2. AI that can connect to ChatGPT, local models, or our backend
3. In-line code execution for Qmd/Rmd files
4. Julia as a first class citizen
5. Multi-agent chats: as many AI sessions as you want and they’ll all run in parallel
6. Windows ARM64 builds
7. Open source AGPLv3 license
8. A bunch of other misc items including read-write data explorer for CSVs and TSVs, plots history sorted by file and time, searchable help, a command history tab, etc
Maybe the biggest difference going forward is that Positron was a ~2 year dev project, whereas Erdos reached feature parity (plus or minus some features) in about ~2 months and is now adding substantial brand new functionality every week.
Yep — if you have a local model with an OpenAI-compatible v1/chat/completions endpoint (most local models have this option), you can route Erdos to use it in the Erdos AI settings.
We started with a product like this at Definite (https://www.definite.app/), but it became clear there weren't enough people willing to spend real money on a product like it when Cursor / VS Code already have good coverage on data science.
Not sure if self-promoting on every single analytics- or data-related thread is in line with the ethos of HN: "Please don't use HN primarily for promotion."
Erdos is also widely considered as the most prolific and productive mathematician of all time (in terms of publications and collaborations). Hopefully you can be as productive with Erdos :)
But productive with it in a different field from the person it's named after? That's weird. It seems disrespectful to him to name a product after him when its purpose is pretty much unrelated to his work.
This is a good idea, although IMO source control, compute, and MLOps integration are bigger but less flashy pain points for data scientists than AI in notebooks.
If you're going to market Erdos as open source, then IMO there should be a github link somewhere on your website.
Thanks for the suggestions - we'll definitely add those to the dev list. Also, the GitHub is https://github.com/lotas-ai/erdos (and it's on the download page but a bit small).
Will echo that one thing that would prevent me from trying this is def the source control. Otherwise it does look pretty slick!
Very cool. Any plans to add support for local models? This has what has prevented us from adopting Positron so far. We have sensitive data and sending to third party APIs is not an option (regardless of their stated retention policies).
Yeah, we just added support for local models. As I mentioned in an earlier comment, if you have a local model with an OpenAI-compatible v1/chat/completions endpoint (most local models have this option), you can route Erdos to use it in the Erdos AI settings.
Looks interesting but i'm unclear what makes it "more accurate"?
When models edit the raw JSON behind a Jupyter notebook, they often mess up the cell structure by adding extra cells, misaligning code, or making bad edits. We fix this by giving the model the notebook in Jupytext format instead, which tends to make its edits cleaner and more accurate.
Give me this, but with a very efficient, opinionated path to put models into production. Give me accessible PM and customer friendly documentation about features and model choices at every stage. Make it reusable and easy to modify. Make it robust and scalable at inference time, with metrics and dashboards tracking performance over time. This seems like optimising the bit that's already fun, but I see a lot of value in hand-holding a department through all the stodgy boring bits and getting high quality analysis repeatably into customer hands.
This looks very cool, I’m gonna try it later today.
Out of curiosity, why the name Erdos? AFAIK Erdos was neither a statistician, data scientist nor AI researcher.
He sure solved many probability/combinatorics problems and famously had many many collaborators.
No specific reason. Mainly because he was one of the most productive and collaborative mathematicians of all time. We actually considered "Poisson" at some point but ended up going with Erdos.
I can't tell how this differs to Cursor from your website. How is it different?
A bunch of specific things below, but the main point is that it integrates a bunch of features that data scientists use that don't come with Cursor.
Specifics (mostly reproduced from above):
1. R/Python/Julia consoles accessible by the user and AI
2. Optimized jupytext system for editing notebooks efficiently
3. Plots pane for viewing and tracking plots
4. Databases pane for managing SQL/FTP connections
5. Environment pane for managing Python/R/Julia packages and environments
6. Help pane for documentation
7. An AI that interacts with all of that.
8. Open source AGPLv3
For me, the biggest difference in the AI usage is that the AI doesn't need to write one-off python scripts for everything and run them from the terminal because it can just use the console directly.
Have you done any fine-tuning or prompt-customization for the R-specific work? I've found the models worse on R when compared to Python, especially for more complex tasks. This looks cool, thanks for sharing!
Nothing R specific. In my experience, Claude is pretty good about using tidyverse for everything. What was is flopping on for you? Our thought on not fine tuning models is that whatever comes out in 6 months is just going to be better than whatever we fine tuned.
Admittedly this was probably ~2 years ago, previous gig used a lot more R and the earlier models were (in my memory) worse in R than in Python. But makes sense that this would've come a long way. Love this!
Apple Silicon only, might be worth mentioning on the download link.
Thanks for pointing that out - will fix it asap
I'm seeing a Windows download link?
The download button on the erdos/ page is OS specific, but you can also find all the download links in the download-erdos/ page.
I think Rao is more appealing to me since Positron already has that kind of integration, while RStudio doesn’t. Plus, Posit probably won’t ever add an AI Chat feature to RStudio anyway.
FWIW there's a bunch of stuff Erdos has that Positron doesn't (including having solved Positron's top 5 open GitHub issues):
1. Remote development via SSH or containers
2. AI that can connect to ChatGPT, local models, or our backend
3. In-line code execution for Qmd/Rmd files
4. Julia as a first class citizen
5. Multi-agent chats: as many AI sessions as you want and they’ll all run in parallel
6. Windows ARM64 builds
7. Open source AGPLv3 license
8. A bunch of other misc items including read-write data explorer for CSVs and TSVs, plots history sorted by file and time, searchable help, a command history tab, etc
Maybe the biggest difference going forward is that Positron was a ~2 year dev project, whereas Erdos reached feature parity (plus or minus some features) in about ~2 months and is now adding substantial brand new functionality every week.
Will, thanks for the explanation. This changes my view a lot. Will give it a try.
Do you have the option to run on a local model? Lots of firms don't want data or prompts going outside the local network
Yep — if you have a local model with an OpenAI-compatible v1/chat/completions endpoint (most local models have this option), you can route Erdos to use it in the Erdos AI settings.
Does it support OpenRouter? I tried configuring OpenRouter as a "local model" but it seems to silently fail.
Not yet - we need to change the header configuration for that to work (versus connecting to local models), but we'll have it available soon.
We started with a product like this at Definite (https://www.definite.app/), but it became clear there weren't enough people willing to spend real money on a product like it when Cursor / VS Code already have good coverage on data science.
Not sure if self-promoting on every single analytics- or data-related thread is in line with the ethos of HN: "Please don't use HN primarily for promotion."
I see Google acquiring Iotas in the future, that's how good it gets
The choice of name seems pretty bizarre. The famous Erdos [1] was a mathematician, not data scientist, computer scientist, or statistician.
[1] https://en.wikipedia.org/wiki/Paul_Erd%C5%91s
He did contribute to/utilize probability theory. He came up during my undergrad probability class because of this: https://en.wikipedia.org/wiki/Probabilistic_method
Erdos is also widely considered as the most prolific and productive mathematician of all time (in terms of publications and collaborations). Hopefully you can be as productive with Erdos :)
But productive with it in a different field from the person it's named after? That's weird. It seems disrespectful to him to name a product after him when its purpose is pretty much unrelated to his work.