I do teach English as a Foreign language, I used to teach computer programming (at a beginner level), and sometimes I daydream about teaching math according to principles from the essay A Mathematician’s Lament by Paul Lockhart, but I am unlikely to be given the leeway to try.

I feel like “passing it through a statistical model”, while absolutely true on a technical implementation level, doesn’t get to the heart of what it is doing so that people understand. It’s using the math terms, potentially deliberately to obfuscate and make it seem either simpler than it is. It’s like reducing it to “it just predicts the next word”. Technically true, but I could implement a black box next word predictor by sticking a real person in the black box and ask them to predict the next word, and it’d still meet that description.

The statistical model seems to be building some sort of conceptual grid of word relationships that approximates something very much like actually understanding what the words mean, and how the words are used semantically, with some random noise thrown into the mix at just the right amounts to generate some surprises that look very much like creativity.

Decades before LLMs were a thing, the Zompist wrote a nice essay on the Chinese room thought experiment that I think provides some useful conceptual models: http://zompist.com/searle.html

Searle’s own proposed rule (“Take a squiggle-squiggle sign from basket number one…”) depends for its effectiveness on xenophobia. Apparently computers are as baffled at Chinese characters as most Westerners are; the implication is that all they can do is shuffle them around as wholes, or put them in boxes, or replace one with another, or at best chop them up into smaller squiggles. But pointers change everything. Shouldn’t Searle’s confidence be shaken if he encountered this rule?

If you see 马, write down horse.

If the man in the CR encountered enough such rules, could it really be maintained that he didn’t understand any Chinese?

Now, this particular rule still is, in a sense, “symbol manipulation”; it’s exchanging a Chinese symbol for an English one. But it suggests the power of pointers, which allow the computer to switch levels. It can move from analyzing Chinese brushstrokes to analyzing English words… or to anything else the programmer specifies: a manual on horse training, perhaps.

Searle is arguing from a false picture of what computers do. Computers aren’t restricted to turning 马 into “horse”; they can also relate “horse” to pictures of horses, or a database of facts about horses, or code to allow a robot to ride a horse. We may or may not be willing to describe this as semantics, but it sure as hell isn’t “syntax”.

Mine was just all repeated digits of whatever hour. 1:11, 2:22, 3:33, 4:44, 5:55, 11:11 all “counted” in my mind when I was entering university, and it happened so freaking often it was really weirding me out. It seemed like anytime I glanced at a clock without other intention, it would be one of those times. There were probably times I looked at a clock normally, but of course confirmation bias reinforces things. But it really did seem far more often than you’d expect. My bet is that my inner clock was prompting me to look at those times because I got an adreneline or dopamine or something spike, so my subconscious got trained into finding it.

Dusting and cleaning does not defeat the purpose. You’re making the mistake of thinking that cleanliness is boolean… true or false. It’s not that it’ll just get dusty again, it’s that it will get more dusty, and then even more dusty, and then dustier still, and there is actually no real practical limit to how filthy a place can get. Cleaning resets the progress to a point where you can live again.

Now, there is a related cleaning story that could be called defeating the purpose that stuck in my mind. It’s a bit Luddite in nature, but does have a point. It’s a micro-story from inside the book “Mrs Frisby and the Rats of Nimh”:

The story was about a woman in a small town who bought a vacuum cleaner. Her name was Mrs. Jones, and up until then she, like all of her neighbors, had kept her house spotlessly clean by using a broom and a mop.

But the vacuum cleaner did it faster and better, and soon Mrs. Jones was the envy of all the other housewives in town—so they bought vacuum cleaners, too.

The vacuum cleaner business was so brisk, in fact, that the company that made them opened a branch factory in the town. The factory used a lot of electricity, of course, and so did the women with their vacuum cleaners, so the local electric power company had to put up a big new plant to keep them all running.

In its furnaces the power plant burned coal, and out of its chimneys black smoke poured day and night, blanketing the town with soot and making all the floors dirtier than ever.

Still, by working twice as hard and twice as long, the women of the town were able to keep their floors almost as clean as they had been before Mrs. Jones every bought a vacuum cleaner in the first place.

That’s an example of defeating the purpose, where the thing you do actually makes it worse. A similar “defeating the purpose” is when a bunch of companies lowers wages to save money, making it so that people can no longer afford their products, meaning that they earn less money after all.