If anyone here hasn't read Borges, I'd strongly recommend him. Pretty much everything he wrote was short, <20 pages, and so it's really easy to sit down and read one of his stories over a lunch break. The common recommendation would be to try out Tlön, Uqbar, Orbis Tertius and see if you like it. If so, it's part of Labyrinths, which is (in my opinion) his best collection of short stories. The best edition in English is probably Penguin's Collected Fictions.
Regarding the content of this interview:
>If you compiled an enormous dataset of everything Borges read, and combined it with an exquisitely sensitive record of every sensory experience he ever had, could you create a Borges LLM?
This is my Kantian way of thinking about epistemology, but I don't think that LLMs can create synthetic a priori knowledge. Such knowledge would be necessary to create Borges out of a world without Borges.
In this interview, Simon's view feels much more like the way Hume viewed people as mechanical "bundles of sensations" rather than possessing a transcendent "self". This led to his philosophical skepticism, which was (and still is I guess) a philosophical dead end for a lot of people. I think such epistemological skepticism is accurate when applied to machines, at least until some way of creating synthetic a priori knowledge is established (Kant did so with categories for humans, what would the LLM version of this be?)
Somewhat relevant to this overall conversation is "pierre menard author of the quixote". Which takes the concept of death of the author in an amusing direction.
> Pretty much everything he wrote was short, <20 pages, and so it's really easy to sit down and read one of his stories over a lunch break.
Yes, his writings are short, but man they are dense!
To anyone who cares, do this exercise: read short story by Borges, probably the shorter the better. Then go ahead say, next day, and try to write it down again in your own words. I tried a couple of times, and I ended with at least twice the number of pages. Amazing.
Tlön is one of my favorite short stories. Weirdly (and perhaps appropriately) that's despite being unable to remember basically anything about it once I've finished reading.
I first encountered Borges in high school, reading “The House of Asterion” as an assignment. Probably not one of his most well known short stories, but I would still recommend it.
>but I don't think that LLMs can create synthetic a priori knowledge.
Do you think that a LLM has the ability to identify a new a priori knowledge?
It seems like it would be a lower threshold to meet but If you combine that with a stochastic process then it seems inevitable that it would be able to ruminate until it came up with new a priori knowledge.
I've said this in another comment but an example would be to train an LLM on a corpus with ALL mathematic content removed. Nothing at all. Then ask it what the shortest distance between two points is. That's an example of synthetic a priori knowledge.
Reading Borges is anything but easy. It requires a certain state of mind. I myself would pick Cortazar over Borges any day, buy I have appreciated some of his writings.
I've tried, but never made it all the way through. Cool to realize the author's sister (Poe) made a hit song "Haunted" when inspired by the same house iirc. There's my random fact of the day.
Let me give you some probably bad advice. Skip the Johnny Truant parts and skim past all the creative layout stuff. It's just decoration, and decoration is often suspicious. Focus on the core story. It’s fantastic. There’s a shot at building a full-on American mythology, Lovecraft-style, from that alone.
Sadly, almost no one talks about it. Ditch the form and embrace the substance. ← It also nods to the mystery behind The Navidson Record.
I was interested in what this house looks like but after a quick internet search it seems that her album was inspired by the novel itself, not any particular house.
> If you compiled an enormous dataset of everything Borges read, and combined it with an exquisitely sensitive record of every sensory experience he ever had, could you create a Borges LLM?
Hmm, what if you could recreate, word-for-word, the great works of an author like Borges (or, say, Cervantes) by so thoroughly understanding their life that the words themselves came out of you, not memorized and recapitulated, but naturally and unbidden? What an interesting idea for a story, maybe an LLM will be able to write that one day.
There already is a story like that in The Cyberiad by Stanislaw Lem. One of the robot characters in the book decides to make a poet robot. They reason that a poet is "programmed" by their culture, and a culture is programmed by the previous culture, so the robot has to simulate the evolution of the world from the beginning of time in order to produce the AI poet. It's a wonderful and hilarious story.
It could be that Lem was influenced by Borges? The original poster is referencing a specific Borges short story called "Pierre Menard, Author of the Quixote" which he published in 1939. It influenced a number of other notable authors
This reads exactly like the plot of a story Borges might write, maybe someone more familiar with his ouevre can shine a light on which stories of his touch on this kind of theme.
Borges is totally recommended, of course, but after reading him in the original language I think his English translations lack the poetry and music of his writings. For once I am happy Spanish is my first language.
The last few months I've been picking up Spanish language editions of Borges's short stories and poems from used book stores. Two decades ago during school I took two years of Spanish, so reading native Borges would be way beyond my comprehension.
With AI tools, though, I can "read" Borges in his native language: with my phone + OCR + translate I have an English language companion. Or, using the voice interface I can try narrating the Spanish text and ask clarifying questions whenever I'm confused.
An author like Borges makes it well worth the extra effort. And, his puzzles often involve language, so the extra layer of mental translation can mirror the work itself, e.g. in his poem La luna [1]. (though, I envy your native Spanish)
My Spanish is more advanced (I was fluent in everyday Spanish, not so much in the more formal use), and I'm reading El Aleph now. It sure takes quite some effort. So many unknown words. And he can turn a phrase quite concisely. But worthwhile.
Congrats on both of you for putting the effort! I am from Buenos Aires, Argentina, so 100% same language as Borges. However, as both of you already said, it isn't even easy for us to read Borges. He has a complex sentence structure, and uses a lot of not-so-common words, so you have to read it really carefully, paying extra attention (not as I'd read any other text). But in the end is really worthwile, everytime I read something by Borges (new or not), I found it fascinating.
Herb Simon certainly was great, but it is weird that the site uses a picture of what is obviously (despite his face being cut off) Claude Shannon working on his robotic mouse Theseus.
Hofstadter should have written Gödel, Escher, Bach, Borges.
I wrote about the connection between Borges, AI, Wikipedia, Kafka (the messaging system, not the author), GPUs, and cryptography in the small print on page 7 of this:
Considering Borges' stories (some written as if they're reports of actual events), I had to wonder for a long while if this is a "reporting" of a "what if" scenario. It would've been a great homage to him.
I see no hint of an audio recording having been made, much less surviving & digitized, in the original article's description: https://www.cambridge.org/core/journals/journal-of-the-histo... It's too detailed to be retroactive notes by Simon or Borges, so I would guess Borges's secretary or a student simply transcribed it as they went.
Gwern and others who have dug into it this far might be interested by this footnote in the Crespo article: "I have tried to lay my hands on the original version of the conversation, as I am sure Simon did, too. I contacted Gabriel Zadunaisky, who, as the article explains, participated in the meeting. He is a professional translator. I asked him for the original version, and he replied on WhatsApp: 'Mr. Crespo: I am very sick. Unfortunately, I am unable to provide you with the information requested.' My hypothesis is that Zadunaisky translated the conversation directly from the recorded version and that this original version has been lost."
My read is that most likely, it was recorded on an old school reel-to-reel tape recorder. It's entirely possible that the tapes are still sitting on a shelf somewhere in Argentina, though the chances of actually tracking them down are pretty low. I worked with some reel-to-reel tapes that Alan Ginsberg made (now held at Stanford) in the mid-60s (including one where he is talking to Bob Dylan!) and they held up pretty well. Had to use audio editing software to remove tape hiss, but they were not as badly preserved as I expected.
That's unfortunate about Mr Zadunaisky, but it does suggest there's no hope of a recording unless someone should stumble across it in the Borges papers (although given how his estate has been abused, little hope of that being useful, one way or another) or the Argentine library archives...
Fascinating and very accessible read. While in jail I tried to get through Sartre’s “Being and Nothingness” (new translation) and some of the big concepts are echoed in this dialogue.
An LLM trained on Sartre would be amazing because the logical extensions of many of his positions and postulations would be uncomfortable in polite society. Even as a human being he quite frequently espoused concepts counter the grain of civility or notions of what ethics are or should be. An unrestrained, uncensored LLM in this vein could be scary and gut wrenching and yet a good reminder of our less-than-ideal state of refinement of thought and behavior as a species.
Borges and Herbert Simons are two great minds, but their conversation is not deep since is mostly shared view about the meaning of human and machine intelligence. Today, with LLMs we have a tool to explore the relation between intelligence and language, between number of parameters, neural nets architectures and much more. So that conversation give us no new insight but is delightful to share time with such great people.
If anyone here hasn't read Borges, I'd strongly recommend him. Pretty much everything he wrote was short, <20 pages, and so it's really easy to sit down and read one of his stories over a lunch break. The common recommendation would be to try out Tlön, Uqbar, Orbis Tertius and see if you like it. If so, it's part of Labyrinths, which is (in my opinion) his best collection of short stories. The best edition in English is probably Penguin's Collected Fictions.
Regarding the content of this interview:
>If you compiled an enormous dataset of everything Borges read, and combined it with an exquisitely sensitive record of every sensory experience he ever had, could you create a Borges LLM?
This is my Kantian way of thinking about epistemology, but I don't think that LLMs can create synthetic a priori knowledge. Such knowledge would be necessary to create Borges out of a world without Borges.
In this interview, Simon's view feels much more like the way Hume viewed people as mechanical "bundles of sensations" rather than possessing a transcendent "self". This led to his philosophical skepticism, which was (and still is I guess) a philosophical dead end for a lot of people. I think such epistemological skepticism is accurate when applied to machines, at least until some way of creating synthetic a priori knowledge is established (Kant did so with categories for humans, what would the LLM version of this be?)
Somewhat relevant to this overall conversation is "pierre menard author of the quixote". Which takes the concept of death of the author in an amusing direction.
> Pretty much everything he wrote was short, <20 pages, and so it's really easy to sit down and read one of his stories over a lunch break.
Yes, his writings are short, but man they are dense!
To anyone who cares, do this exercise: read short story by Borges, probably the shorter the better. Then go ahead say, next day, and try to write it down again in your own words. I tried a couple of times, and I ended with at least twice the number of pages. Amazing.
There's definitely a Pierre Menard joke in here somewhere
Tlön is one of my favorite short stories. Weirdly (and perhaps appropriately) that's despite being unable to remember basically anything about it once I've finished reading.
I first encountered Borges in high school, reading “The House of Asterion” as an assignment. Probably not one of his most well known short stories, but I would still recommend it.
>but I don't think that LLMs can create synthetic a priori knowledge.
Do you think that a LLM has the ability to identify a new a priori knowledge?
It seems like it would be a lower threshold to meet but If you combine that with a stochastic process then it seems inevitable that it would be able to ruminate until it came up with new a priori knowledge.
I've said this in another comment but an example would be to train an LLM on a corpus with ALL mathematic content removed. Nothing at all. Then ask it what the shortest distance between two points is. That's an example of synthetic a priori knowledge.
Reading Borges is anything but easy. It requires a certain state of mind. I myself would pick Cortazar over Borges any day, buy I have appreciated some of his writings.
I have the opposite experience. I find Cortazar impenetrable, but the Borges stories "speak to me".
Highly recommend his Fictions too. Grabbed a worn copy for 1€ on the street months ago, and I still think about Uqbar from time to time.
Did you ever read House of Leaves?
I've tried, but never made it all the way through. Cool to realize the author's sister (Poe) made a hit song "Haunted" when inspired by the same house iirc. There's my random fact of the day.
Let me give you some probably bad advice. Skip the Johnny Truant parts and skim past all the creative layout stuff. It's just decoration, and decoration is often suspicious. Focus on the core story. It’s fantastic. There’s a shot at building a full-on American mythology, Lovecraft-style, from that alone.
Sadly, almost no one talks about it. Ditch the form and embrace the substance. ← It also nods to the mystery behind The Navidson Record.
I wish I had known this when I first read it.
I agree, I loved the story (I also did kind of enjoy the form, but agree it can get in the way of enjoying the story).
The Navidson Record is on youtube if you dare =P
OMG! ੧[⁰o⁰]ʋ
Thanks!
I was interested in what this house looks like but after a quick internet search it seems that her album was inspired by the novel itself, not any particular house.
> If you compiled an enormous dataset of everything Borges read, and combined it with an exquisitely sensitive record of every sensory experience he ever had, could you create a Borges LLM?
Hmm, what if you could recreate, word-for-word, the great works of an author like Borges (or, say, Cervantes) by so thoroughly understanding their life that the words themselves came out of you, not memorized and recapitulated, but naturally and unbidden? What an interesting idea for a story, maybe an LLM will be able to write that one day.
There already is a story like that in The Cyberiad by Stanislaw Lem. One of the robot characters in the book decides to make a poet robot. They reason that a poet is "programmed" by their culture, and a culture is programmed by the previous culture, so the robot has to simulate the evolution of the world from the beginning of time in order to produce the AI poet. It's a wonderful and hilarious story.
It could be that Lem was influenced by Borges? The original poster is referencing a specific Borges short story called "Pierre Menard, Author of the Quixote" which he published in 1939. It influenced a number of other notable authors
The joke the previous comment is making is that Borges already wrote that story. "Pierre Menard, the Author of the Quixote."
ChatGPT, Author of "Pierre Menard, Author of the Quixote"
I see what you did there...should your username be Pierre Menard, perhaps?
This reads exactly like the plot of a story Borges might write, maybe someone more familiar with his ouevre can shine a light on which stories of his touch on this kind of theme.
I think the comment is referring to “Pierre Menard, Author of the Quixote”.
Yes indeed. This thread seems to indicate more people should read more Borges!
Borges is totally recommended, of course, but after reading him in the original language I think his English translations lack the poetry and music of his writings. For once I am happy Spanish is my first language.
The last few months I've been picking up Spanish language editions of Borges's short stories and poems from used book stores. Two decades ago during school I took two years of Spanish, so reading native Borges would be way beyond my comprehension.
With AI tools, though, I can "read" Borges in his native language: with my phone + OCR + translate I have an English language companion. Or, using the voice interface I can try narrating the Spanish text and ask clarifying questions whenever I'm confused.
An author like Borges makes it well worth the extra effort. And, his puzzles often involve language, so the extra layer of mental translation can mirror the work itself, e.g. in his poem La luna [1]. (though, I envy your native Spanish)
1. https://www.gaceta.unam.mx/la-luna-un-poema-de-borges/
My Spanish is more advanced (I was fluent in everyday Spanish, not so much in the more formal use), and I'm reading El Aleph now. It sure takes quite some effort. So many unknown words. And he can turn a phrase quite concisely. But worthwhile.
Congrats on both of you for putting the effort! I am from Buenos Aires, Argentina, so 100% same language as Borges. However, as both of you already said, it isn't even easy for us to read Borges. He has a complex sentence structure, and uses a lot of not-so-common words, so you have to read it really carefully, paying extra attention (not as I'd read any other text). But in the end is really worthwile, everytime I read something by Borges (new or not), I found it fascinating.
I had the same issue with Stephen King, reading it in English after reading Spanish translations is a different world.
Which editions? The Iberian Spanish ones are not that different in tone/speech to the original English ones.
Hey! Opino lo mismo!
(I agree!)
Herb Simon certainly was great, but it is weird that the site uses a picture of what is obviously (despite his face being cut off) Claude Shannon working on his robotic mouse Theseus.
https://www.futilitycloset.com/2018/08/23/shannons-mouse/
I’m a huge fan of his short story Funes the Memorious. Link: https://ia801405.us.archive.org/10/items/HeliganSecretsOfThe...
Which is in Borges words "a long allegory of insomnia" :)
https://philosophy.as.uky.edu/sites/default/files/The%20Libr...
I really enjoyed and have recommended to others this very short paper, 'Borges and AI' [1], that was also discussed on HN a couple years back [2].
[1]: https://arxiv.org/abs/2310.01425
[2]: https://news.ycombinator.com/item?id=38693120
Hofstadter should have written Gödel, Escher, Bach, Borges.
I wrote about the connection between Borges, AI, Wikipedia, Kafka (the messaging system, not the author), GPUs, and cryptography in the small print on page 7 of this:
https://lab6.com/4#page=7
Hofstadter was defiantly a fan of Borges' work. https://themindi.blogspot.com/2007/02/chapter-1-borges-and-i...
What, when did he stop being so?
I didn't mean to imply that he stopped being a Borges fan.
you beat me to this
Considering Borges' stories (some written as if they're reports of actual events), I had to wonder for a long while if this is a "reporting" of a "what if" scenario. It would've been a great homage to him.
Is there an audio file of this interview? I'd prefer listening to the original (in the background).
I see no hint of an audio recording having been made, much less surviving & digitized, in the original article's description: https://www.cambridge.org/core/journals/journal-of-the-histo... It's too detailed to be retroactive notes by Simon or Borges, so I would guess Borges's secretary or a student simply transcribed it as they went.
Gwern and others who have dug into it this far might be interested by this footnote in the Crespo article: "I have tried to lay my hands on the original version of the conversation, as I am sure Simon did, too. I contacted Gabriel Zadunaisky, who, as the article explains, participated in the meeting. He is a professional translator. I asked him for the original version, and he replied on WhatsApp: 'Mr. Crespo: I am very sick. Unfortunately, I am unable to provide you with the information requested.' My hypothesis is that Zadunaisky translated the conversation directly from the recorded version and that this original version has been lost."
My read is that most likely, it was recorded on an old school reel-to-reel tape recorder. It's entirely possible that the tapes are still sitting on a shelf somewhere in Argentina, though the chances of actually tracking them down are pretty low. I worked with some reel-to-reel tapes that Alan Ginsberg made (now held at Stanford) in the mid-60s (including one where he is talking to Bob Dylan!) and they held up pretty well. Had to use audio editing software to remove tape hiss, but they were not as badly preserved as I expected.
That's unfortunate about Mr Zadunaisky, but it does suggest there's no hope of a recording unless someone should stumble across it in the Borges papers (although given how his estate has been abused, little hope of that being useful, one way or another) or the Argentine library archives...
Anyway, I uploaded the Simon book chapter at https://gwern.net/doc/borges/1996-simon-2.pdf
That's awesome, thank you for uploading.
This seems to be the version published in an Argentinian magazine:
https://borgestodoelanio.blogspot.com/2017/05/jorge-luis-bor...
Fascinating and very accessible read. While in jail I tried to get through Sartre’s “Being and Nothingness” (new translation) and some of the big concepts are echoed in this dialogue.
An LLM trained on Sartre would be amazing because the logical extensions of many of his positions and postulations would be uncomfortable in polite society. Even as a human being he quite frequently espoused concepts counter the grain of civility or notions of what ethics are or should be. An unrestrained, uncensored LLM in this vein could be scary and gut wrenching and yet a good reminder of our less-than-ideal state of refinement of thought and behavior as a species.
Borges and Herbert Simons are two great minds, but their conversation is not deep since is mostly shared view about the meaning of human and machine intelligence. Today, with LLMs we have a tool to explore the relation between intelligence and language, between number of parameters, neural nets architectures and much more. So that conversation give us no new insight but is delightful to share time with such great people.