Neural Networks
Introduction
Does it take brains to profit from neural networks?
-------------------------------------------------------------------------------------------
--------------------------------------------
Connecting with Neural Networks
A neural network is a poor man's version of a brain, a mechanical device that solves pattern recognition problems, like finding price history patterns in the stock market. It provides optimal solutions to problems. But the solutions generally prove to be only local optima, unless special steps are taken. They can then be near-optimal global solutions. The basic structure allows for:
The system accepts relevant data -- like equity prices over a number of days -- learns to work the data intelligently, and comes up with a solution to the problem. This is generally the way we ourselves seem to solve problems, when we can -- i.e., when we have the skills.
A network consists of a layer of input nodes, a layer of two or three internal nodes, and an output node. The layer of input nodes consists of a set of variables that are input simultaneously, called a vector or pattern. The vector components might be different components of a face, various dimensions of an object, a tennis ball in motion, or a time series of prices of a stock, and so on. Each pattern produces a specific output.
More complex devices may involve a greater number of layers and more complex connecting systems. For example, you might include feed-forward loops and feedback loops (known in brain research circles as recurrent loops). These additions would bring the design closer to the structure of the living brain. Whether they could provide what we know as consciousness is another question. In some quarters it is believed that as yet unknown elements underpin the neuronal structure and serve as the basis of consciousness.
As a device to recognize a time-series pattern in the stock market, say, the input might consist of a sequence of daily, weekly, or monthly closing prices of a particular stock, and the output could be a projected price for the next period. The trick is to get the network to recognize the price pattern in the input sequence and predict what the price will be the next day. This is the dynamics problem in a nutshell. The critical aspect of the network and key to its problem-solving utility is the pattern of weightings assigned to the connecting links -- a pattern established through a course of training. In math terms this is to define the tensor that solves the problem. (A tensor is a generalized function that takes or transforms one vector into another by means of a matrix.) See Matrices, Tensors for details.
----------------------------------------------
You need to train a network to do your bidding. That is, you have to generate the weights for the neural connections -- a fruitful linkage would receive a strong weight and a fruitless link would get a weight of zero. You let the network's "experience" define the mapping from the inputs to the output -- the buy-sell rule for stocks, say. The training leads to an appropriate matrix and thus to a winning tensor.
If you knew from the start what weights to assign to the connecting links, what formula to use, why of course you wouldn't need to do the training. But you wouldn't need a neural network either. You'd be able to solve the problem without it, just by evaluating the tensor formula. After all, the idea is to get the artificial brain to learn the set of rules that do the job -- to gain knowledge you don't have -- not to teach it rules you already know. In the neural network the weighted structure implicitly defines the rules; you generate the weights by training it.
The training procedure is simplicity itself; it learns as we do, by trial and error -- at least for one type of network called the supervised network. You get the network to guess (a trial) and then you test the result by comparing it against the actual conditions, which is provided through your supervision. If the guess is wrong, you adjust the weights and try again. You continue to make the adjustments as long as it takes for your artificial student to get the correct -- profitable -- answer. There are different ways to make the weight adjustments, but no matter which method you apply, it is through trial and error that the artificial network learns which formula works.
In the time-series problem, for instance, the network would likely receive a sequence of price inputs for a stock for a number of periods (hours, days, weeks, ... ) and would output a price for the stock for the next period. The predicted value would then be compared with the actual value for that day. If the difference turns out to be greater than a desired amount, the weights are adjusted and another trial is run and again tested against the actual value. The technique known as minimizing the mean-square error is normally used to minimize the error. The process continues until the error lies within a desired range of expectations.
--------------------------------------------
How to Proceed with the Development
Compared to the brain, artificial neural networks are simple almost to the point of being trivial. Note I said almost. If they really were trivial, there would be no point continuing with this. They aren't trivial. But the non-trivial characteristic is preserved only by severely restricting the job assigned to them. The task set for a network has to be very narrow in scope and sharply defined, else the required complexity for the network would exceed our capacity to build them. The network becomes a specialized pattern recognition sensor.
To train and use such a network effectively you need to stick to some important steps. They are as follows, adopting the directives given by Edward Gately, in Neural Networks for Financial Forecasting:
1. Determine What Is to Be Forecast
Stick to a program and don't include anything that's not relevant. Also choose a target that's realistic. For example it would be difficult to predict the value of the Dow 30 in absolute terms to just a few points. The value would represent a very low percentage change for the Dow, one that would be too small to predict successfully. Better to pick the percent change as the projection target.
2. Collect Data Sets or Number Series That Have a Relationship to What Is Being Forecast.
It's reasonable to believe that, as Gately points out, your golf score or tennis victories and losses for the week, for instance, have nothing to do with the price of the S&P 500 for next week. They are unrelated. This is a selective process we ourselves follow. Much more relevant would be the price of oil or the short-term interest rate, either of which could affect the S&P. (This already requires a model, of sorts.) You pick data relating to the price of the S&P 500. And of course that's a big problem, right off the bat, because you need a bit of knowledge of the markets, as you'd expect.
3. Preprocess the Data to Combine Information or Make It More Useful (like percentages)
4. Set Minimums and Maximums (to the data)
5. Extract Test Set Data (that system hasn't seen)
6. Select Suitable Network Architecture
7. Select a Suitable Learning (Training) Algorithm
8. Train the Network
9. Use the Network
.
--------------------------------------------
It is said that neural networks can have no direct knowledge of anything, that they don't have an innate cognitive ability, that they have to be "trained" for the capability. This applies to knowledge it might have of its own functions, or what we call self-awareness, because it first has to be trained on its own inputs. This may be true. After all, networks are simply machines. But the same would apply to the human brain, if treated as a neural network, which it could be. Our brain would then not have immediate and direct knowledge of itself. It would have to be trained. In other words, it would need experience of itself to learn about itself.
Here's what Paul M Churchland says in The Engine of Reason, the Seat of the Soul, which treats the living brain as a neural network.
... [For] a neural network to have knowledge of any particular domain is for it to have acquired an expertise in discriminating some important and recurring set of features within that domain, and to have acquired some expertise in responding to them in some systematic way. This requires in turn the development of a suitable configuration of synaptic connection weights, one that partitions the network's neuronal activation space into a useful set of categories [i.e., concepts]. Once these categories are in place, the network can be said to have a general or background comprehension of the domain at issue. And once it begins to activate those categories on appropriate occasions, it can be said to have specific knowledge of the domain's unfolding activities.
...
The upshot is that immediate and absolutely certain knowledge is something that is not to be expected from a neural network. Quite aside from the many local perils that attend the local application of a given framework in a specific circumstance, there is the ever-present background peril that the general framework is faulty, inadequate, or suboptimal to begin with. This is true of all neural networks. It is therefore silly to expect such pristine knowledge in our own case [i.e., self-knowledge], assuming we are indeed just sophisticated neural networks [which seems to be true, though strong arguments can be made against it].
An alternative view of self knowledge, represented by Immanuel Kant and others argues that knowing yourself is directly related to knowing about the outside world, inasmuch as the nature of that world is largely of your own making -- the choices you adopt, the things you do, and the interpretations you give to (impose on) the world.. The more you know about the world, the more you know about yourself, because acquiring knowledge is a selective process, and your selections reveal important things about yourself. But this view is consistent with the network option if the brain is indeed a neural network, because finding out what you actually select only requires a self-directed sub-network.
---------------------------------------