I am fascinated by Artificial Intelligence and Neural Networks. I've been getting into them (by which I mean reading about them on an extremely casual basis) for a few years, and in the last few months I started to get a little bit obsessed.
And now, I am about to embark on a journey in which I hope to eventually train a Neural Network to write tickling stories, and I thought it might be fun to chronicle it here in my blog.
To sort of set the stage for this, I should explain what I mean by those terms, especially since I am the furthest thing from an expert and I'm probably using them at least a little bit incorrectly.
Our brain is a neural network, in lowercase because I'm referring to the general concept, meaning literally a "network of neurons." Neurons, and the connections between them, are what our brain is made of; they form everything from our most basic instincts like breathing, to our most advanced thought-processes. They're simultaneously the hardware AND the software of our brain.
If you've ever seen a baby learn how to feed itself cheerios, it's a good example of how neural networks such as our brains learn a new task - at first, the baby is just swinging it's arm randomly, but over time it will get incrementally closer and closer to its target. Soon the baby can reliably smack the cheerios around, and it starts learning how to use its fingers. Again the effort will be mostly random at first, but through trial and error it will eventually perfect the task and start shoveling cereal into its relentlessly gawping food-hole. (I don't have kids, but I assume that's what it's like when they eat.)
And according to my best understanding, a Neural Network (capitalized because I'm referring to the specific software now) is a piece of software that is designed to emulate the way our brains work. They're given a set of tools by a "trainer" and they attempt to reach a goal through trial and error. For example, a trainer might give a NN a million pictures of furniture and attempt to get it to be able to identify them by type (chair, sofa, table etc...)
It would do that by guessing randomly at first and comparing its results to the labels on each picture. It would attempt to learn from its mistakes and successes and improve itself, by rewriting its own math before trying again. And because its a computer, it can do that millions of times in a matter of hours or days.
People have trained NNs to do amazing things recently, but the best example is voice recognition software. Engineers had been trying to program a computer to recognize speech for almost as long as computers have existed, but it was seemingly impossible because of how many ways there are for people to say the same thing.
It was only after the perfection of Neural Nets that they were able to do it - by giving the computer a massive database of audio files and letting it teach itself to process them. It tried over and over again, literally billions of times, getting slightly better each time until it could hear speech 95% as accurately as a human.
Which brings me to the topic of Artificial Intelligence - to the best of my knowledge, an AI is a piece of software that has been written by a Neural Network to accomplish a specific task. For example Siri would be an Artificial Intelligence created by Apple's largest Neural Network.
Artificial Intelligence can be a confusing term, because it makes you imagine a thinking machine, but at this point in their development they're a lot more artificial than intelligent. Neural Networks are closer to "thinking machines" but they only solve a specific problem and then they stop running - they don't sit around thinking.
An artificial intelligence might do that, run continuously to accomplish a task like managing a bank of phone lines and their connections, or being the personal assistant on a phone. But however well it imitates a human, it doesn't understand anything that it's doing. It doesn't "know" that it's running your phone any more than a calculator "knows" that it's adding 2+2. Someday that will certainly change, but we're still a long way away from that.
One of the easiest ways to discover Neural Networks is from pop culture websites playing with them - they'll feed it a database of all the ice cream flavors ever invented and have it create realistic-but-silly-sounding new flavors like Mango Cats and Pumpkin Trash Break - kottke.org/18/05/ask-an-ice-cream-professional-ai-generated-ice-cream-flavors
The key here is that nobody taught the neural network to do that. It was a completely blank slate that didn't know anything about anything, and they let it learn by a combination of example and trial-and-error. The exact same software could have instead been trained to recognize pictures of furniture.
The first hundred attempts (They call them "epochs." I'd have called it a "generation" but nobody asked me. Anyway, the first hundred epochs...) are total gibberish, like just a string of a hundred i's in a row. But with every attempt it will get a little closer.
Which finally brings me back to this blog - I've begun learning how to "train" a Neural Network. In fact, I'm writing this blog after my first extremely modest success in which I fed an AI the entire works of Shakespeare, and got back a bunch of Shakespear-y gibberish.
This was a predesigned test where I just followed instructions, but even still it took a few tries to understand and properly execute it. The result here was after ten "epochs" and just following the variables I was given for the test, to see if I could get the NN to produce the exact same results as the creator of the exercise.
Now I'll try tinkering around to see if I can make it better, or funnier, or anything-er. I'll try having it use more randomness and less randomness, and see what that does. I also want to see what happens if I give it another ten epochs to learn in, or a hundred, which would probably take all night.
And once I have a working knowledge of how to train it, my eventual goal will be to create a massive document with every tickling story on the TMF, and attempt to train the network to write its own. They might end up being gibberish like the Shakespeare, or they might be hilariously off - but to set some expectations, what I'm ultimately aiming for is to get it to write a readable story that could pass for something that was written by one of us.
I will update further as events warrant
Jeff
And now, I am about to embark on a journey in which I hope to eventually train a Neural Network to write tickling stories, and I thought it might be fun to chronicle it here in my blog.
To sort of set the stage for this, I should explain what I mean by those terms, especially since I am the furthest thing from an expert and I'm probably using them at least a little bit incorrectly.
Our brain is a neural network, in lowercase because I'm referring to the general concept, meaning literally a "network of neurons." Neurons, and the connections between them, are what our brain is made of; they form everything from our most basic instincts like breathing, to our most advanced thought-processes. They're simultaneously the hardware AND the software of our brain.
If you've ever seen a baby learn how to feed itself cheerios, it's a good example of how neural networks such as our brains learn a new task - at first, the baby is just swinging it's arm randomly, but over time it will get incrementally closer and closer to its target. Soon the baby can reliably smack the cheerios around, and it starts learning how to use its fingers. Again the effort will be mostly random at first, but through trial and error it will eventually perfect the task and start shoveling cereal into its relentlessly gawping food-hole. (I don't have kids, but I assume that's what it's like when they eat.)
And according to my best understanding, a Neural Network (capitalized because I'm referring to the specific software now) is a piece of software that is designed to emulate the way our brains work. They're given a set of tools by a "trainer" and they attempt to reach a goal through trial and error. For example, a trainer might give a NN a million pictures of furniture and attempt to get it to be able to identify them by type (chair, sofa, table etc...)
It would do that by guessing randomly at first and comparing its results to the labels on each picture. It would attempt to learn from its mistakes and successes and improve itself, by rewriting its own math before trying again. And because its a computer, it can do that millions of times in a matter of hours or days.
People have trained NNs to do amazing things recently, but the best example is voice recognition software. Engineers had been trying to program a computer to recognize speech for almost as long as computers have existed, but it was seemingly impossible because of how many ways there are for people to say the same thing.
It was only after the perfection of Neural Nets that they were able to do it - by giving the computer a massive database of audio files and letting it teach itself to process them. It tried over and over again, literally billions of times, getting slightly better each time until it could hear speech 95% as accurately as a human.
Which brings me to the topic of Artificial Intelligence - to the best of my knowledge, an AI is a piece of software that has been written by a Neural Network to accomplish a specific task. For example Siri would be an Artificial Intelligence created by Apple's largest Neural Network.
Artificial Intelligence can be a confusing term, because it makes you imagine a thinking machine, but at this point in their development they're a lot more artificial than intelligent. Neural Networks are closer to "thinking machines" but they only solve a specific problem and then they stop running - they don't sit around thinking.
An artificial intelligence might do that, run continuously to accomplish a task like managing a bank of phone lines and their connections, or being the personal assistant on a phone. But however well it imitates a human, it doesn't understand anything that it's doing. It doesn't "know" that it's running your phone any more than a calculator "knows" that it's adding 2+2. Someday that will certainly change, but we're still a long way away from that.
One of the easiest ways to discover Neural Networks is from pop culture websites playing with them - they'll feed it a database of all the ice cream flavors ever invented and have it create realistic-but-silly-sounding new flavors like Mango Cats and Pumpkin Trash Break - kottke.org/18/05/ask-an-ice-cream-professional-ai-generated-ice-cream-flavors
The key here is that nobody taught the neural network to do that. It was a completely blank slate that didn't know anything about anything, and they let it learn by a combination of example and trial-and-error. The exact same software could have instead been trained to recognize pictures of furniture.
The first hundred attempts (They call them "epochs." I'd have called it a "generation" but nobody asked me. Anyway, the first hundred epochs...) are total gibberish, like just a string of a hundred i's in a row. But with every attempt it will get a little closer.
Which finally brings me back to this blog - I've begun learning how to "train" a Neural Network. In fact, I'm writing this blog after my first extremely modest success in which I fed an AI the entire works of Shakespeare, and got back a bunch of Shakespear-y gibberish.
ShakespeareBot said:
This was a predesigned test where I just followed instructions, but even still it took a few tries to understand and properly execute it. The result here was after ten "epochs" and just following the variables I was given for the test, to see if I could get the NN to produce the exact same results as the creator of the exercise.
Now I'll try tinkering around to see if I can make it better, or funnier, or anything-er. I'll try having it use more randomness and less randomness, and see what that does. I also want to see what happens if I give it another ten epochs to learn in, or a hundred, which would probably take all night.
And once I have a working knowledge of how to train it, my eventual goal will be to create a massive document with every tickling story on the TMF, and attempt to train the network to write its own. They might end up being gibberish like the Shakespeare, or they might be hilariously off - but to set some expectations, what I'm ultimately aiming for is to get it to write a readable story that could pass for something that was written by one of us.
I will update further as events warrant
Jeff
