Featured

Can a neural net read a 5 year old’s handwriting?

I recently went up to visit my Mom for her birthday and while up there came across this pretty cute birthday card on the mantle: 

The author of this amusing well wish is my mom’s latest ward, 5 yr old Peter from up the block. This is a pretty bad ass dragon if you ask me! 5 year old me would nod with silent approval. 

It’s fun because you have to puzzle out the letters a bit. Some are pretty near the mark like the T and O but a few other such as the various e’s are more than a little obscure. To an outsider they might be illegible but being in possession of a brain and some context, not all together impossible to discern. My mother’s name is Rhea, it’s her birthday, therefore the meaning of the young author’s scribbles is “To Rhea, Peter”. 

This got me to thinking: What would a machine make of this birthday card? Put more provocatively: Can a neural net read a 5 year old’s handwriting? 

Well, we don’t have to guess the answer to this question, we can try it. 

The neural net we’ll use is the same one that’s all the rage these days: A convolutional neural net with several hidden layers. You’ve probably heard them go by the name of “deep learning” and other fancy buzzwords. More or less the same sort of nets that power the various machine learning implementations you’re familiar with like Alexa, Facebooks image classification and presumably what’s behind Tesla’s self driving cars. 

We’ll use pytorch as our library which is essentially a machine learning library written in python and developed by Facebook.  

A fair amount of massaging the data was required but the reader versed in machine learning will know that that’s most of the work. The code here took barley a hlf hr to cobble together. The image manipulation on the other hand took a few hrs (I probably could of sped that up by writing a script to do these operations but then I would have had to figure that out which would’ve took who knows how long. The dilemmas of the lazy man!). 

Here’s an example of the processed data: 

But isn’t that cheating?! No, not really, I’m working around the difference in conditions between the birthday card and the training data set: EMNIST. The data set is in black and white, the birthday card, green text on white background. 

So, now on to the experiment: Using a net with 2 convolutional layers, 2 fully connected layers, max pooling and 4 epochs (number of times you run over the training data), here are the results. 

Actual message: 

TO RHEA 

PETER 

Machine read: 

TO RHOA 

PPTOR 

At first glance this might not seem too good, but it definitely got a good portion of the message correct. 8/11 characters correct, or about 73%. If this was a test to pass the first grade we’d make the cut. Also if you consider it from the perspective of chance we can definitely see that out net is working to some extent. 

There’s 26 possible characters (letters in the alphabet) and 11 letters in the message. So if our program was just guessing the letters, the chances that it would correctly guess all letters is an astronomical 1/3,670,344,486,987,776‬ chance. Looking at it from the inverse: What are the chances that it would correctly guess 8/11 characters? 

1/208,827,064,576 or about a 1 in 200 Billion chance. 

So with that in mind we might not thumb up our nose at the little program so lightly.  

The astute reader will realize that this is a very bad experiment scientifically: The writers which contributed to the EMNIST data set were all adults and given specific paper and marker to write with. Those initial conditions are totally different than our young author: green crayola marker and construction paper. None the less I think the experiment is telling. 

Perhaps more interesting than evaluating the machine: we can evaluate the kid. Technically minded readers will notice this as what’s termed a reverse turing test. Given an individuals actions how sure are we that they are in fact a human and not a machine. 

In our case it would be more like: How close is our kid to being an adult in terms of handwriting. If he’s basically writing at an adult level we would expect our program to easily read his message 100% accurately or very near to. Well based on the test, I’d say pretty darn close. He’s got a few kinks to work out but our young author is well on his way. 

Like the handedness of his ‘e’s’: 

Face down e’s, inverted e’s and half finished e’s. This is interesting because it’s a trend. Although I have no 5 year old, I’ve seen enough examples of their writing to notice that they tend to have trouble with handedness. Backwards P’s, b’s and the like are common in young children. There’s something interesting there but it’s hard to say what. The same issues don’t seem to plague letters like T and A which are symmetrical and to which the writer need not pay attention to handedness. 

Terrapin dream

The Readers Club, New York

In the summer after my 33rd Ice time, when I was just a whippersnapper, I first met the Big No Furries.

It was on one of those hazy late summer afternoons. I was all the way across the lake, out in the deepest section, right before the big ol boulders on shore.

Life was good, I’d not a care in all Gawd’s blue waters. I floated idly along, riding an eddy or two when all of a sudden:

“Thunk, thunk…”

Something had slapped me upside the head like Mama had caught me stealin minnows from the ol Snake neck.

I quickly flipped up and to my surprise: there were two ungainly furries glaring at me while floating on a log.

My jaw dropped in utter surprise: what in the heckin was dis? I thought.

The one Furrie closest, wielded an oak frond which he beat the water with. The strangest thing was that these furries had no fur, excepting on the tippy top of their little heads!

My alarm was somewhat quieted by the realization that in their upright position even a little eddy would send em a tumbling.

And so I gave em the splash, fully expecting em to plunge into the eddies face first, but alas, they seemed not even a little perturbed!

At this, I thought it best to make a swift exit, not much caring for the rather unusual manner in which these furries were licking their lips.

And so I dove off. I thought first to check with ol Father Deep, as he was the wisest of the Shelled Folk, and had seen more life than any of us.

Word on the eddies was that Father Deep was so old he’d even been around back in the Big Ice Time. Though that was mostly hearsay, it was hard to discount it with all the things he knew’d about.

Getting to him, weren’t exactly easy: living true to his moniker, he was always aways, ways down…….. and it was quite hard to see down there.

Finding him was even further complicated by the fact that he never moved no moar.

Year by year, he seemed to become more and more akin to the mud and brambles of his little alcove. There was a trick though to getting his attention…. although sometimes using that method was followed by a prompt drubbing.

What you had to do was grab a hold of his little snare tongue and give it a good pinch.

Figuring it was worth the while, I sat still for a bit until I saw it: just this little pink wormy thing wriggling out of some big ol hole there.

And so I swam up and gave it a little pinch…

After no small amount of time had elapsed, a low and ponderous whisper of a voice spoke from the mud bank:

“Anddddddd whhhhoooooo iisss ittt, that awakkenssss myy slumberrrrrrrrrrr…….?”

“Hey Big Daddy, it’s Snap, how’s it shakin down here,” I answered.

And after some more time…….. the reply:
 

“Ohhhhhhhh, itttsss just you, Snappppp…. Beeen a while………. what bringsss you alll the way down here?”

“Well, I’m in need of your sagacious knowledge is all, you see just right now when I was sunbathing I got’s slapped upside the head by the most unusual of furries, and I’s wondering if you’d ever come across one of dem in your travels.

You see, the odd thing about this pair of furries is that they had no furs, excepting on the tops of their little brown heads. On top of all this they was floating on a log most unusual like, skimming like a big old water skimmer.”

There was a pause for quite sometime………. and then:

“YOU GOSH DARNED WHIPPISNAPPERS GOT THE NERVE TO COME DOWN HERE WITH YOUR TALL TAILS AND YOUR EMPTY SNAIL SHELLS.

WHO DO YOU THINK YOU ARE?!?!?!?!?

BIG NO FURRIES!!!! WHAT WILL THEY THINK OF NEXT…

I’VE HALF A MIND TO SLAP YOU SO HARD YOU’LL WAKE UP IN THE NEXT BIG ICE TIME,

GET GONE WITH YOU, YOU LITTLE RAPSCALLION.”

There was really no need for that last bit as the eddies of this gesticulation alone expunged me to the surface without so much as a splash.

My curio not quite quenched though, I thought it best to seek out the Mother.

I splashed over to the submerged log to see if Remmy or Yellow Belly had seen her, but they weren’t there. So I climbed up to the tip top of the log to get a good gandering. It took me a minute but after a whiles I spotted her ol mossy broadsides glimmering in the sun.

Mother was the yin to Father Deep’s yang: she couldn’t never stay put for long. She’d always be floating along in them eddies, a veritable Island. Like the ol legends say upholds the whole world.

I splashed over her way and skimmied on up the side, past the tree and a little group a loud honkers sittin all cosy like in her mossy beds.

It took me a moment to get my bearings, but afore long I found her head poking out a little above the eddies. She turned around,

“Well hullo Snap, whatya been up to? Come to visit Mama? Lemme fix you up some minnow sandwich’s, why don’t ya. “

“Oh, that sounds, absolutely fantastic Mama, I’m right starved and ain’t ate since yesterday morn.”

Her claws flashed out and went a rummaging through one of dem gourds she keeps on her big ol shell. Before long, the sandwiches, presented themselves: minnows on crappies with little web foot egg sauce. I took a big ol bite and went to em ravenously.

“So what’s on your mind Snap? You got a furitive look about you.” I swallowed a few more bites afore answering:

“Mama you ever seen a creature round these parts that’s like a Big No Furry? Walks on two legs, got two big gangly arms and a little furry fur on top, but like a shiny crappy on the outside otherwise?” Mama’s eyes went up and around for a sec and then she replied:

“Oh yes, you mean the two legs, they come all the time and take little nippy naps right on ol Mama…… Don’t suppose they even know I’m here, judging by the sight of things.”

“Well what is they Mama?”

“They’re new around here but you best be careful with them Snap, once about a week ago I saw something that really raised my eyebrow bout them. I’s over by the falling water that never ends, and I espied em sitting about in a big ol group having a little picnic seemed like. Most peculiar was they kept throwing little logs in a mini fiery fire like the one’s the sky flashes make.

But that weren’t the half of it…

You see, I watched em for a good long time, all through the night as they huddled around that little fiery fire and in the morning a Big Furry No Tail came upon em and it was most peculiar. He stood up and got one look at them and ran like the dickens. I figure if he had a tail it’d been between his legs just like the little red furry be doin when he come around.”

Now that was most peculiar. Big Furry No Tail didn’t play none.

Quite the opposite really: everyone knew he had claws as big as Father deep’s, a hide that even the sharpest beak couldn’t penetrate and big ol nasty fangs that’d cut right through a shelled one like a beak through whisker fish.

“Mama, he must’ve just remembered where he put the pointy stick head carcass or something, Big Furry No Tail ain’t fraid nobody on Gawd’s blue earth.”

“I saws what I saw, little Snap, I’m tellin you. Whatever they’s is they not to trifled with none.” I pondered this for a moment, then said:

“Well look at the big yellow circle… I think it’s about time I went splash and found my cozy cozy afore black and little white shinies time. Thanks Mama for the minnow sandwiches, they certainly were swell.” Mama gave me an appraising look then turned away,

“Take care of yourself Snap, an don’t be going anywhere nears them Two legs if you see em.”

And so I splashed off and made fast eddies over to where my cozy cozy was but afore I’d even got there I’d made up my mind to go over to the water that falls but never ends and do a little peeping. Chances were the Big No Furries weren’t even there anyhows, so what was the big deal?

A few splashes later I got close and dove down and come up right where the water comes down. Just to give me a little more cover to spy on em from.

Sure enough, there was the little fiery fire that Mama mentioned and around it a whole group of them Big No Furries, in some kinda big revelry. They was passing a gourd around and drinking from it, and then making noises like the Loud honkers do and getting red as the little red furry.

After a while I noticed something strange: There was Remmy, Yellow-belly and Sharp-back right in the middle of em,  but turned over like they were doing the flip. This was most strange I thought. But I had to get a closer look, it could be they’s was stuck upside down. According to the 5th rule of the Shelled folk, if one came across a shelled folk all upside down it was his Gawd given duty to set his brother aright, else all hell and damnation were on his head.

And so I splashed closer, and closer, till I was just close enough to get a better look.

Wish to Gawd I hadn’t though….

The horror of it was almost unbearable:

Remmy, Yellow-Belly and Sharp-back weren’t stuck. They’d been fashioned into Shelled folk Soup containers! The Big No Furries was dancing around em, dippin in their gourds and taking big ol sips of my pals insides!

The Horror!

I’d never splashed outta some place so fast in my life!

Marginalia of Creation III: The Eagle

John J. Audubon

There is a wisdom that is woe; but there is a woe that is madness. And there is a Catskill eagle in some souls that can alike dive down into the blackest gorges, and soar out of them again and become invisible in the sunny spaces. And even if he for ever flies within the gorge, that gorge is in the mountains; so that even in his lowest swoop the mountain eagle is still higher than other birds upon the plain, even though they soar.

Melville’s Moby Dick

At high noon on the fifth day, God made the eagle. In the fires of the forges of El he fashioned him. From the hottest part of the fire he beckoned him forth.

And to you oh great Eagle I grant these gifts:

“Thine eyes wilt shoot forth lightning, yea, your gaze will be like the morning. Piercing through the darkest of nights.

I give thee the strongest of wings to beat the wind with a alacrity: not lackadaisically like thy brother the Hawk.

Thy abode shall be the frozen north, the wastes where my Spirit dwells strongest. Few tread there, but I give you a constitution to bear it. And even more so, to thrive and soar over that north country.

You shall eat from mine own hand, plucking fishes from my deep.

All fowl and fish will live in fear of you, and every creeping thing that slithers through the wood.

Thou shalt be a King bird, and bear dominion over all fowls. Only my children will be over you…

And yet, even they will take inspiration from you: your wings and countenance will adorn their coins and banners, innumerable as grains of sand on the seashore.

Thy talons shall be sharper than any two edged sword for you alone are bestowed with my terrible strength.

But above all these things, I grant to you one final gift: you alone of all birds have gazed on the hem of my garment, and forthwith thou shalt have a white head.

Your form will be mighty, but above all these, it is your whiteness which will appall.”

Marginalia of creation I: The Songbird

The Goldfinch ~Fabritius

Are not two sparrows sold for a farthing? and one of them shall not fall on the ground without your Father. ~ The King James Bible

The first bird that God made was the Songbird.

Before departing for the earth, God summoned him out of the void, and said to him:

“Oh Songbird, I give to you these gifts with which to ply thy trade:

Though shalt be the smallest of fowls, but thereby shall all the earth’s bounty be open to you. Your wingtips will alight to all the four corners of the globe.

And I give you the most adaptable of all beaks with which to break the seeds of brambles.

You will be the most industrious of all birds, and by your work shall the whole of the earth bear fruits. For you will take the seed to every rock and cranny of the earth.

From the Garden of the Two rivers to the furthest specks of land in my vast oceans.

Yea, to the ends of the earth. Those specks of rock hemmed in by the sea, which shall be a Terra Incognita to my children until they have come of age and are ready to partake of the Fruit of Knowledge.

The last gift I give to you is this, and by far the greatest of them:

Though shalt be a Herald to mine Dawn, and the coming of my light. By your song shall all beasts and men breath hope from my infinite wellspring.

They shall breath a sigh of relief, for the banishment of the terrors of the night is nigh at hand.

Of all fowl you are most alike to my Angels. Though tellest the whole earth of my promises and their perennial immutability.

And by your song shall the creatures of night shrink, and take succor in darkness.”

Marginalia of creation II: The Hawk

Christopher Ciccone/Audubon

God’s own gunslingers ~ Stephen King’s the Dark Tower

And on the 5th day God make the Hawk. From the darkness he summoned him into his court.

“To you oh Hawk I give these gifts: Wings like blades to cut the wind in pursuit, and talons with which to dispense my justice upon the earth.

The mere shadow of your wing will strike Terror into the heart of the staunchest of sparrows.

You will be a hated bird, but I grant to you the gift of Indifference by which to accomplish your task.

Far seeing wilt be thine eyes, but by them, though shalt be given a certain detachment. 1

But it is good. For the task I give to you will require it. To hunt and to kill, relentlessly, all the day long, from when mine Sun rises, until it sets.

I make you my ordained killer on the earth.

That the Kingdom of Mice shall not overrun the Kingdom of Man.

You will be the most solitary of my fowl, but in that solitude, I grant you wisdom and eyes from which to behold the truth of my mine earth.

Take wing then, let lose thy talons upon the darkness.” 2

  1. Anne Rice’s Interview with the Vampire
  2. Bungie’s Destiny: Hawkmoon flavor text

Lakeside reverie: The Blue Dragonfly

In the summer in Tarrytown the dragonflies appear overnight. They can be seen in the afternoon in well lit alcoves along the lakes edge perched on branches that jut out of the water, and flitting about defending their little outpost from any interlopers.

The Blue Dasher species with their shimmering electric blue coloration is especially mesmerizing. Their coordination and aerial prowess are really something. Why do dragonflies never accidentally hit into you, when flies, bees and all manner of flying insect can’t seem to help themselves?

Their pose is reminiscent of a helicopter’s, head down, tail up and a tendency to swerve from side to side.

Occasionally, their hover will be interrupted and they will zip about in a seemingly random direction for a second, only to return to their original position. They will frequently harry any flying insect, dragonfly or other which enters their vicinity. At first glance, this can appear to be a predation move, if the insect is something small like a mayfly, but after watching for a few seconds, it becomes clear the little fly gets away without a scratch.

Dragonfly skirmishes are a real spectacle, a dog fight in miniature. Sometimes the Blue Dasher will spar with dragonflies of the same species, a duel in electric blue faster than the eye can track. Other times, their most frequent sparring partner appear to be the fiery red Eastern Amberwing, a worthy opponent. After the interloper is warded off, the Blue Dasher will return to its perch to regain its strength and soak up more sunlight.

This perch defense behavior appears to be an elaborate mating ritual of sorts. The Blue Dashers eck out its a little alcove, wooing a mate, zipping about, discomfiting any rival that wanders there.

Dragonflies have a head that is something like 80% compound eye, wrapped around nearly their whole head. This gives them excellent vision: They can see in nearly 360 degrees at a very high ‘refresh’ rate with something akin to binocular vision.

Dragonflies have very high success rates when hunting, but I’ve never seen a Blue Dasher catching anything or even hunting something. The closest behavior I can think of which might fit this, is when I’ve seen larger dragonflies seemingly hunting in fields (I can’t think of any other reason they would be here, and they tend to be at the same heights as swallows who also appear to be hunting).

Amazingly, they do have predators, despite their speed and agility. One such is the cunning Cat Bird. The Cat Bird will perch on the branch of a dead tree near where dragonflies are frolicking, pick out a target and then make a quick 1-2 feet dash to try and catch them. It’s hard to tell exactly what their trick is, but they seem to have some strategy other than brute force and speed which the dragonflies are susceptible to.

Another predator of dragonflies is yellow jackets and wasps who somehow manage to restrain and butcher them despite their larger size.

One mystery of dragonflies: have they been under the pressure of natural selection recently, or are their adaptations more static in nature. Are they just a ‘perfect’ organism for their niche, with no further improvements needed so long as the environment stays more or less the same?

Late Jurassic period

A remarkable thing about the Blue Dasher is that it only lives for the season. As summer ends you see fewer and fewer of them as they die off going into the Fall. They disappear slowly, and then all at once, with a few of their larger cousins hanging in longer, but eventually succumbing, leaving the little alcoves a little less frenetic in preparation for winter.

H.G Wells’s Time Machine

A tale of three centuries 

H.G Wells’s “The Time Machine” contains some incredible digressions.

1895:

“Seeing the ease and security in which these people were living, I felt that this close resemblance of the sexes was after all what one would expect; for the strength of a man and the softness of a woman, the institution of the family, and the differentiation of the occupations are mere militant necessities of an age of physical force. Where population is balanced and abundant, much child-bearing becomes evil rather than a blessing to the State; where violence comes but rarely and offspring are secure, there is less necessity – indeed there is no necessity – for an efficient family, and the specialization of the sexes” 

Here he’s speaking of the Eloi, the physically diminutive and mentally naive surface dwellers who are descendants of humanity. The Eloi sexes are harder to differentiate than modern persons. It’s a fun trend to explore, we see this lack of differentiation at least in professional ‘white collar’ occupation where the gender gap has become smaller over time.

“So, in the end, above ground you must have the Haves, pursuing pleasure and comfort and beauty, and below ground the Have-notes, the workers getting continually adapted to the conditions of their labor” 

Wells had an interest in socialism and you can see some Marxian influence here. The premise of the Eloi and Murlock divide was basically that the rich became richer and lived in refinement where as the murlocks struggled living miserable lives. The tables turn in this future though because the ‘aristocracy’ have become so adapted to gentle conditions that they have wholly lost their fear and ability to fight. A bit like the dodo bird who with no natural predators had no fear of man and was hunted to extinction in a decade because of it. The Murlocks on the other hand have maintained their cunning and ferocity, being on the razor edge of evolution and struggle (they actually eat the Eloi). 

The events of the time machine are in the far-flung future but it’s fun to think about this prediction or scenario in the 125 years since it was written. As it happens humanity is more or less enjoying a golden age now. Compared to the blustery war years or the social upheavals of the industrial revolution era things are pretty peachy now. This begs the question: Are we going down the Eloi path as a civilization? Has prosperity lead to a lethargic generation of do-nothing good timers? 

I would argue, No, that it hasn’t: The decade is near closing (December 2019 as I write this) and thinking back on just the last decade I see a wave of technical advancements and changes. Mobile computing stands out in my mind as just one example: In 2009 I didn’t use my phone to stream the music I was listening to, hail an uber, board a plane and order food. There’s a lot of efficiency gains there. Just taking the plane example: Before the internet age, you had to go into the airport, see the check-in desk to get checked in and print out your boarding pass, find a directory to navigate the airport and so forth. Afterwards, you could be checked in and know your seat number while still in your pajamas in bed!

“The too perfect security of the upper-worlders had led them to a slow movement of degeneration, to a general dwindling in size, strength and intelligence” 

“Nature never appeals to intelligence until habit and instinct are useless. There is no intelligence where there is no change and no need of change”

(Interesting theory on the origin of intelligence.) 

1960: 

The 1960 film adaptation of the book is also a masterpiece. It keeps very close to the source material with a couple fun riffs thrown in here and there, for example: when the time traveler first jumps into the future, he finds himself in the midst of some kind of nuclear attack. The populace is fleeing into underground bunkers or safe houses and the air raid sirens are blaring. 

The great fears of the post war world come true and the world is blown to smithereens in some apocalyptic “The Day after Tomorrow” style ending. It’s indicated that the Morlocks develop from those folks who took safety underground and presumably lived out the destruction above for a few generations to become the phantasmal Morlocks . 

2002 

This adaptation takes the most liberties with the source material but I think it’s my favorite. Guy Pearce plays the time traveler. There’s a beautiful tragic love story to kick things off where the time traveler’s fiancé is shot in Central park by a robber attempting to steal their engagement ring.

The time traveler attempts to fix this by going back in time and taking the fiancé through another segment of the park or avoiding the park all together. Regardless of what he does she dies. This spurs him on his quest to find out why he can’t change the past, he thinks that they would certainly know the answer to this question in the far-flung future…

The Eloi of this future are neither diminutive like the novel nor simpering like the 1960 film. Here we have a mulatto race of future hunter gatherers. This bit was probably influenced in part by the expectation that the skin color differences that we have today will eventually average out yielding up a face like this: 

Wells would probably appreciate this expansion of the source material, his own contemporaries had just uncovered The law of regression to the mean a few years prior (Sir Francis Galton). Galton even gave a widely publicized demonstration at the legendary Royal Society on the topic. 

The 2002 version has its own insights on the future. Or at a minimum a glimpse into the odd manias and obsessions of the double aughts: 

In future New York, people ride around on bicycles presumably because the world’s oil supply has been depleted and cars are no longer economical. 

The future is struck not by war but by environmental disaster. The time traveler spends a few eons in a frozen waste engulfed in some glacier. (This brings to mind the Laurentide ice sheet which buried New York and most of New England under a mile-thick sheet of ice, tens of thousands of years ago.)

The film shows the influence of computers and is the first adaptation where they play a central role in the plot: 

The Eloi speak of a ghost hidden in a cavern who turns out to be the AI of the long lost New York Public Library. The idea of AI as an enduring memory of humanity is briefly explored here. 

Also striking are the CGI sequences: breathtaking in scope and a gentle tease of the kinds of things that machines can now do!