On Programming Purpose: Part 1
We're at peak AI. What does it mean to build towards AGI? I'd like to begin where American Linguist Daniel Leonard Everett left off. How do we go from text to intelligence?
Everything I know about humans I learned from birds. No, seriously. Birds have been an obsession for over half my life. It all started when “Life of Birds” sparked my curiosity about behavior. What followed was countless hours of waiting, observing, and often listening to the real Twitterverse.
Language
It’s fair to say that human language has come full circle. Beginning from pictures (see Fig A), and back to pictures again. English serving as programming of sorts. Very soon you can show a computer pictures and program that way too. Great day for symbols and symbolism.
The cuneiform script (depicted in Fig A) was in use from 2500 BC to ~1000 AD. That’s over 3000 years in popular usage. Here’s a thought exercise: What could a hypothetical LLM built in this era have accomplished for humanity, I wonder?
What could a machine learning system from the year 500 AD have learned? Would it achieve AGI/ASI (Artifical Superintelligence) by now?
Today's machine learning systems only learn our preferences. At best, they are learning to ape humans. If we want a monkey, we’ll get one. One that can be subjugated to do tricks, but I’m positive we don’t get intelligence.
Language: Intuition
Breaking down the Chomsky vs Everett debate
Everyone understands how you acquire a language. You have acquired it yourself, seen your kids, relatives, and friends acquire it, and you know when one can say complex things or feign proficiency. I hope.
Chomsky theorizes that, just like we grow biceps and armpit hair, we also “grow” language. It begins when you’re a child, and your language faculties mature around early adolescence. He also posits that a universal grammar must exist, epigenetically encoded in our hardware, which gives us this magical ability with words. Our ability to create it, think with it as a toolset, and string together multiple words that convey meaning. Sometimes, rarely, someone strings together words in a compelling way and revolutionizes human thought.
The theory also feels true. Language is intrinsically social. Kids seem to possess this magical ability to pick up as many languages as you throw at them provided they are surrounded by speakers.
As a technical linguist who worked on context-rich grammars for Google Search, I lived and breathed this symbolic NLP-first philosophy until I read Everett’s work. It was my personal “strong opinions, loosely held” moment.
Everett worked with the Pirahã people from the Amazon and documented their language & grammar (containing this cool phoneme). “Pirahã language lacked certain features, such as recursion, that Chomsky and others believed were universal to all human languages. This challenged the idea that the human brain is hardwired with an innate universal grammar”.
He also claimed that human cognition is deeply influenced by the language and culture one grows up in. I agree with some of his work, and one that I personally resonate with is the one where language can exist minus innate structures.
To support Everett’s claim, I'm introducing you to a simple but cute language from another social species. One that you can speak to yourself (mimic) and make friends with across North America. First, a quick engagement break.
Language: function
In humans, the primary function of language is thinking. Communication seems like a happy accident. With writing, papyri, gutenberg, and so on, we devised intergenerational thinking. But, let’s switch to another species.
The chickadees are a new world bird family, and these ones from my backyard have a tiny range across the hills of Northern California. They’re a gregarious bunch and I’m lucky to see them at my feeders very often.
Always traveling in groups of 4 to 6, chatting all the while and making their presence known. They scope out new locations with one of them taking the lead, while the others quietly fly in close by as backup. A chance encounter introduced me to their memorable chirp - “Chick-a-chick - DEE DEE DEE”. DEE DEE DEE was loud and in my ear, a clear warning call to the others that a predator was out and about.
Chickadee’s have a simple language, “Fee Bee” a sort of hello there, and they have distinct sounds for gaggle i.e. casual talk, warnings, quiet warnings, and more. All Chickadee variants share this trait with minor regional variants based on noise levels and altitude. Here’s a reference chart. They even have a severity rating in the # of DEEs they vocalize in their warning call. The closer the predator and riskier the situation, the more the # of DEEs. I’ve spoken to a few chickadees, and they respond and actively strike up a conversation. Try it with a bird calls app like Merlin.
For the most part, we can speak Chickadee. Since they’re social animals, language plays an important role in keeping the group safe and talking. “In a 2006 study by Freeberg, chickadees placed into large captive flocks used calls with greater information content compared to chickadees placed into small captive flocks, suggesting that the diversity of messages is greater in more complex social groups.”[source]
Without being into linguistics, you can surmise that a language exists, shared by all the individuals of a species (epigenetically encoded), adaptable with group size, and increases the individual’s odds of survival in the world. All this minus a universal grammar and other assertions from Chomsky.
Two questions to ponder over:
Supposing the chickadee population dramatically declines, could we program bots to take their place and fulfill their function to preserve the balance of nature?
What does it mean for a chickadee clone to be intelligent? What biological imperatives does the clone need to mimic to understand a natural born bird’s intrinsic motivation?
Language: General Intelligence
“If you beat a horse enough, it’ll confess to being a lion” - Hindi saying
Eventually, LLMs will beat out every eval you throw at it.
How do you go from LLM to true emergent intelligence? Well, it turns out that the number of GPUs, sensors, or, sadly, parameters in the model do not matter. Yes, there are efficiencies that need to be found in computing. Machines will help those efficiencies. Mostly though, humans will architect that through specialized code running through special-purpose chips.
What does it really mean for a program to be intelligent? Does the system have to present a new, as yet unseen consilient theory and supporting proof? for e.g., think outside the system like Gödel? Or does it need to solve the Twin-primes conjecture? There’s no rational explanation for it all, nor are there set expectations about intelligence.
Until then, we have alignment - mechanism design to predictably control autonomous systems.
A way to control and direct the chaos of combinatrics.
That’ll require a lot more exploring and inventing.
Subscribe if you’re interested to read more. Let me know what you think in the comments, especially about topics you’d like me to spend time exploring.
Follow me on twitter: https://x.com/itsmeritesh
</NotaBot>
Cover image
