Reading list 2024: On Consciousness, Knowledge, Behavior, and AI
My reading list for this year includes books on AI, language evolution, linguistics, Alignment, Chomsky, Sapolsky, and more. Perfect Thanksgiving gift for those curious about languages & AI.
Languages have fascinated me since I was very young. Growing up in South India, I was destined to be trilingual. However, moving across several Indian states made me proficient in 5 languages. I’ve heard several arguments for unifying the nation using a single language. China’s cultural revolution is often cited as an example, but I never liked that movement or its implications. Every language I know is rich with literature, poetry, insights, and knowledge locked in for over a thousand years. Until recently, most of this knowledge was passed on orally. How could we converge to one without losing all that accrued value?
In the recent past, we’ve lost many languages due to the population dying or due to a lack of institutional support. Similarly, I know of linguists who study ancient languages like the Egyptian Hieroglyphs or the infamous Harappan script, hoping to uncover knowledge captured and disseminated by their peoples.
I also observed the power of stories across languages. I own a copy of the Epic of Gilgamesh, considered the first novel from 2100 B.C.E. It describes the story of a hero who saved people from a deluge and sought eternal life. I’m positive you’ve read or seen multiple variants of this famous epic in your culture, as I have.
This fascination with languages has also influenced my hobby as a naturalist; consequently, I can identify more than thirty bird species through their calls. When the LLM & AI movement started, you know I was paying attention. Especially the part where it was all linguistic combinatorics.
Language as a foundation for AI
For me, 2024 was all about AI and learning about unsolved problems and why they remain unsolved. I read many research papers describing the state of the art, some of which I’ll share slowly as I broach each subtopic that fascinated me.
I’ve been lucky to be at the center of the AI revolution through my time in Silicon Valley and the multiple AI-powered products I helped build. All the magic of the last few years came down to "next token prediction,” as Karpathy called it.
A few seminal books helped me understand the fundamentals of language theory, human and animal behavior, and their influence on speech, symbolism, epistemology, and meaning. I highly recommend reading these if you want to work with language systems or Artificial Intelligence.
With Thanksgiving approaching, please consider gifting these to your friends, family, and others working in AI or linguistics. My list includes a short note about what I learned from the book.
1. What kind of creatures are we? by Noam Chomsky
(Amazon link) A collection of essays by Noam Chomsky explaining many fundamental concepts around the brain, the history of thought, and models of thought. An important takeaway from this book - “speech is externalized thought”.
2. The Alignment Problem by Brian Christian
(Amazon link) Good introduction book describing the problem of AI alignment. Bias in systems, our unstated assumptions, and “first responders” of Alignment.
3. Behave by Robert Sapolsky
(Amazon link) Trying to answer the question: “Why do we do the things we do?”. This interdisciplinary book examines neurobiology, evolution, and culture to explain human behavior. Prof Sapolsky is a fantastic and humorous writer who will keep you entertained.
4. Human Compatible by Stuart Russel
(Amazon link) I loved reading this book by Prof Stuart Russel. While every researcher and frontier model company is trying to achieve AGI, this one starts by asking the critical question: “What if we succeed”?
5. Kingdom of Speech by Tom Wolfe
(Amazon link) Excellent summary of the work done in linguistics & philosophy, starting with the Greeks and ending with Daniel Everett’s work refuting Chomsky’s universal grammar and recursion.
6. Content by Cory Doctorow
(Amazon link) This book, and others by Cory, inspired me to write to him to thank him for sharing his wisdom. To my surprise, he replied. This is a fantastic take on the state of content rights in the US and why and how platforms have helped or hindered progress and creativity in the arts.
7. Beginning of Infinity by David Deutsch
(Amazon link) From award-winning physicist Prof David Deutsch, this one explains the creation of knowledge, the history of civilization, art, and morals and discusses the necessity for creating new knowledge and explanatory theories about everything.
8. Why Animals talk? by Arik Kershenbaum
(Amazon link) One of the most fantastic books I read this year is Prof Kershenbaum’s Studies from the Field, detailing accounts of conversations between gibbons, parrots, dolphins, and wolves and breaking down why it’s down to evolution to explain animal communication. The seemingly bizarre vocal expressions are far from mere noise and serve an important evolutionary purpose.
9. Consilience by E O Wilson
(Amazon link) This book is considered Prof Wilson's magnum opus. It explains the evolution of different scientific disciplines and their importance, trying to define a unified theory of knowledge (not unlike the work of David Deutsch). Something I took away from this book is how scientists worldwide discovered climate change and how it was an interdisciplinary discovery that no one discipline could confirm. Almost 30 years since its discovery, it still has not found support among large swaths of people and state institutions.
10. Don’t sleep, there are snakes by Daniel Everett
(Amazon link) Considered one of the best works of Prof Daniel Everett, this book details his life’s work working with the Piraha people and their conception of life, language, and adventure. Key takeaway: The Piraha don’t have a concept of time, so there is no tomorrow, yesterday, or numbers. This simple omission seems to change the nature of their language and challenges Chomsky’s life’s work on the theory of linguistics.
Getting these books for cheap/free
Many of these books are expensive; if you have limited means, here are some ways to access them cheaply.
I love the Libby app and encourage everyone in the US to get it and map their free Library card to it. I read three of these books for free on it. Thanks to San Mateo County’s library system for their efforts.
Buy second-hand or used copies of this book on Abebooks. Their service is available in the UK, France, Canada, New Zealand, and Germany. While it’s not free, you can get used copies of these books for an absolute bargain.
Try accessing your local university or Alumni library. You might have access to these books for free, owing to your local county or alumni rules.
You could also try the Honey savings plugin, which offers between $2-$5 of savings for some books.