Hi there Sean,
This is going to be a bit tricky. While I do have experience with Tensorflow for Python and Processing, I have little experience with Javascript and even less with Tensorflow.js. However, I will try my best to help you out a bit conceptually. Please let me know how much experience you have with machine learning and Tensorflow.
In the meantime, I will explain what I would do to approach this problem.
Since this is a big enough project, I would start by breaking down the problem. The ultimate goal is that you want to train a ML algorithm-controlled ball to follow your mouse. You could write a program that has a ball follow a mouse completely using vectors and target vectors, but seeing as you started out with Tensorflow, I’ll assume you want to try out using a machine learning algorithm. All right. You will probably want to use some type of reinforcement learning algorithm
since you have no particular good “examples” for how the ball should follow the mouse. If you wanted to write this program in a language without a machine learning library (ex. Processing), I would recommend using a genetic-algorithm optimized neural network, since simple neural networks and genetic algorithms are relatively simple to write from scratch. However, you have the lovely hand of Tensorflow guiding you, so no need to do all that grunt work.
I would start by simplifying the problem. Conceptually, you want to teach something to follow a target. The logical first step would be to write a program that randomizes the path of a target, mimicking, in your case, a mouse movement. Writing a random generator for these paths helps to keep you from monotonously sitting at your desk for hours just moving your mouse.
Once you have set up this path generator, you need to set up what I call a “frame”. A frame is a set period of time that you can think of as a single step in the training process. Let’s make this frame a canvas of about 600 by 600 pixels with a length of about ten seconds. This would mean that every ten seconds, the target is reset to the center point, and then goes about its randomly generated path bounded within the window (600x600 pixels) for ten seconds, and then is reset at the end of the ten seconds to the center of the screen.
Thirdly, I would write a small program for the follower, which in this case is the ball. You have already done this. Yay! If I were writing the networks and such from scratch, I would include a “brain” (instance of a neural network class) in this follower class. But since you have Tensorflow, I wouldn’t worry about that. For now, just write a few functions in the follower class that change the ball’s velocity vector. I wouldn’t worry about acceleration yet, we’ll add that in later if the velocity thing works. If you want to simplify it further, set hte velocity to a constant “1”, and just write a function that changes the heading of said velocity vector. Write the regular display and update functions. You will also want to write a function that outputs the follower’s location on the screen, its velocity, and its heading, as numbers. You will use this later as inputs (confusing! but useful in the future). Another morsel for thought- write a function that judges how close the follower stays to the target constantly. I would use a constantly updating average- have the “fitness” of the follower be determined by the average distance it has been to the target over the course of a frame.
Now that you have all that framework stuff out of the way, test it. Write a function that resets the ball to a location every “frame” and another that constantly feeds the follower a random velocity vector (or in the simple case, a random heading). See how this works. Given some fiddling on your end, you should ultimately end up with a sketch that has a target moving in a random path and a “follower” moving randomly and constantly updating its fitness, resetting to base positions every ten seconds.
Now we get to the juicy part, the part with some real, honest, frustrating thinking and troubleshooting- the ML algorithm. I said earlier that we would be using some kind of reinforcement learning algorithm. And that’s true, but here’s where you have reached a difficult crossroad. You can take your model and easily expand and adapt it to train using a genetic algorithm- neural network combo, save the best neural network weights/biases to a file, then use those weights/biases in a neural network that controls a real ball that will follow a target (your mouse) in another program, no longer training and just existing. Let me know if that appeals to you. However, this is a genetic algorithm, and since Tensorflow is used for, well, tensors, GAs are kinda off-topic and not really supported…at all. Alternatively, if you still want to use Tensorflow, you could go the harder (but more interesting) route and implement a self-training deep net.
It’s kinda like going down the steeper slide at the park even though you might break an ankle in the process.
At this point, I warn you. I have much more experience with GAs and simple deep nets than I do with whatever I will talk about from this point on. This problem is much better suited to be optimized by a genetic algorithm- NN combo than basically any other machine learning algorithm that I know of. And unfortunately, this means that Tensorflow just ain’t gonna cut it. Machine learning is fantastic, and fun, and interesting, but it’s not a panacea. You can’t just take any old problem and throw a deep net at it, or download Tensorflow and try and manhandle it to do your bidding. Certain algorithms work for specific problems. Imagine, if you will, I needed to paint a fence, and I had two options. Pay Huck Finn an apple to do it, and not worry about it, or program a robot to do it for me. Paying the apple loses me my lunch, bummer, but the programming the robot to do something it was never meant to do costs me my entire day, and even then it does the work worse (that’s a really, really crude and poor metaphor, I’m sorry).
Now, back to the subject. If you are still dead-set on using Tensorflow to train your follower, use a simple deep net. Personally, I have found that similar networks need no more than one or two hidden layers, the first with about 2 more neurons than the input layer and the second with 2 more neurons than the output layer has. Now, let’s talk about your model. If your follower just gets the input of its own location, it will output an arbitrary vector every time. Now, it might take a little bit of watching your algorithm to figure this out, and once you do, it will be so frustrating to figure out what was wrong. I know because I slipped up here once as well. In order for a network to react to a stimulus (in this case, the target mouse position), it needs to know what the stimulus is/what it is doing. For how “smart” they are, deep nets are actually very silly and kinda dumb. You need to let them know as much as you can about their world as you can. To them, the world around them is just numbers. Remember that deep nets are not so complex in nature. They find and make patterns based on numbers and math. They know nothing about their world. So we take our models, extract numerical data from it, and then feed it to the network. The network takes these numbers, does some math, and spits out some more numbers. We then take these outputs and manually relate them to our models. They cannot reason on their own. So, with that in mind, you want to feed the net, in your case, the x and y positions of the follower, the x and y positions of the target, and the x and y components of the follower’s velocity vector and the target’s velocity vector, separately. So you end up with a deep net with eight inputs. Following our earlier rule, the first hidden layer has ten neurons. Since you want to output an x and y component for the velocity vector of the follower, you should have the second hidden layer contain four neurons. However, this is a bit lopsided, so personally, I would fudge the network a bit until you end up with an 8-8-4-2 network: 8 inputs, 8 neurons in H1, 4 in H2, and 2 output neurons. All this is just known as architecture.
Once you have the structure of the network, focus on the connection to the network. Set it up so that every time the screen updates with your simulated mouse and the follower, the network also updates. The first time you initialize the network, create random weights. Input the inputs into the network, run the network to produce outputs, and then use those outputs to change the velocity vector of your follower.
Now comes the fun bit. You are going to fudge the ol’ backpropagation. To do this, you are going to need to do a few things. First of all, disable the “frames” portion. Just have it so that the simulated mouse constantly moves across the screen. Next, you are going to want to define a few variables. You’ll want a previousFitness variable as well as a bestPossible vector. The best possible vector will be the x and y components of a vector that is the vector sum of the target’s velocity vector and the vector connecting the follower ball to the target point. This is the optimal vector for the follower to produce. This is what we will use to train the follower. If you want something to learn, you gotta teach it. In order to teach the follower’s network, we have to compare the vector it suggests the follower should take (based on the inputs) against the optimal vector. Then, you can use Tensorflow to compute an error vector, backpropagate the error through the network, and adjust the weights. Since it performs this gradient descent once a tick, your follower should catch on pretty soon. Theoretically, I think this should work. I also think it’s less interesting than a genetic algorithm simply because you have to provide it with the answer to teach it. Personally, I find that when a subject learns what it is supposed to do given nothing but a fitness, the result is fascinating.
In summary, I think that you should look some more into how machine learning algorithms work and what they are used for. I love machine learning, and I love that you are interested in it and I want you to continue with it. I realize now I have written a long essay that more or less says that you can do one of two things. I just wanted to let you know that I love being proved wrong, and I am pretty sure I got one or two things wrong in this essay. It sounds like a fun project and I don’t want to come across as pretentious, mean, or snobby. I could have probably written this whole thing in one paragraph, headlined as follows:
You can probably use a genetic algorithm to do this, or you can train the ball using a target vector.
But I didn’t, for better or for worse. I took this opportunity, a little bit selfishly, to try and explain my way of thinking through an ML problem. I apologize for (probably) boring you and making you skip through a lot of the tangential stuff, as well as for the general vagueness of my answers. Thanks for making it this far, and I wish you the best with your project. Most of all, have fun with it. I will be here to help with any problems you have, whichever route you choose to go on, or neither at all. If you want to try something other than what I suggested, go for it. I’m down to learn it with you.
TL:DR; You can use a genetic algorithm to optimize a simple deep net to solve this problem. Otherwise, you can create a model that randomly generates a simulated “mouse movement” continuously, then use a computed target vector to train a deep net that controls the following ball. Use the position and velocity vectors of the follower ball and target point as your NN inputs, have two hidden layers, then output a velocity vector for the follower ball. Compare this outputted vector to the target vector, then backpropagate the error to train the network. Have fun!
Best of luck, and best regards. Don’t give up! ML is fun once you get the hang of it, which you will shorter than you think. Don’t hesitate to contact me with questions, concerns, or really anything at all.
- Trilobyte