Finite size neural networks

A paper by Hedi Soula and myself on the dynamics of a finite-size neural network (Soula and Chow, Neural Comp, 19:3262 (2007)) is out this month. You can also download it from my homepage. As you may infer, I've been preoccupied with finite-size effects lately. In this paper we consider a very simple Markov model of neuronal firing. We presume that the number of neurons that are active in a given time epoch depends only on the number that were active in the previous epoch. This would be valid in describing a network with recurrent excitation for example. Given this model we can calculate the equilibrium probability density function for a network of size N directly and from that all statistical quantities of interest. We also show that the model can describe the dynamics of a network of more biophysically plausible neuron models. The nice thing about our simple model is that we can then compare the exact results to mean field theory, which is the classical way of studying the dynamics of large numbers of coupled neurons. We show that the mean activity is generally well described by mean field theory, except near criticality as expected, but the variance is not. We also show that the network activity can possess a very long correlation time although the firing statistics of a single neuron does not.

Kinetic Theory of Coupled Oscillators

I have recently published two papers applying ideas from the kinetic theory of plasmas and nonequilibrium statistical mechanics to coupled oscillators. The first is Hildebrand, Buice and Chow, PRL 98:054101 and the second is Buice and Chow, PRE 76:031118. The main concern of both papers is understanding the dynamics of large but not infinite networks of oscillators. Generally, coupled oscillators are studied either in the small network limit where explicit calculations can be performed or in the infinite size "mean field" limit where fluctuations can be ignored. However, many networks are in between, i.e. large enough to be complicated but not so large that the effects of individual oscillators are not felt. This is the regime we were interested in and where the ideas of kinetic theory are useful.

In a nutshell, kinetic theory strives to explain macroscopic phenomenon of a many body system in terms of (the moments of the distribution function governing) the microscopic dynamics of the constituent particles (oscillators). In the coupled oscillator case, we actually have a macroscopic theory and what we want to understand is how the microscopic dynamics gave rise to that theory. For example, in the Kuramoto model of coupled oscillators, there is a phase transition from asynchrony to synchrony if the coupling strength is sufficiently strong. In the infinite oscillator limit where fluctuations can be ignored, an order parameter measuring the synchrony in the network can be shown to bifurcate from zero at a critical coupling strength. However, for a finite number of oscillators, the order parameter fluctuates and there is no longer a sharp transition from asynchrony to synchrony but rather a crossover. We show in the first paper that a moment expansion analogous to the BBGKY hierarchy can be derived for the coupled oscillator system and using a Leonard-Balescu-like approximation the variance of the order parameter can be computed explicitly.

The second paper shows that the moment hierarchy can be equivalently expressed in terms of a generating functional of the oscillator density (i.e. a density of the density if you like). Once expressed in this form, diagrammatic methods of field theory can be used to do perturbative expansions. In particular, we perform a one-loop expansion to show marginal modes in the mean field theory are stabilized by finite-size fluctuations. This problem of marginal modes had been a puzzle in the field for a number of years.

House of Cards

The US Comptroller General, David Walker, is currently on a "Fiscal Wake-up Tour" to alert the population about the impending US fiscal crisis. I encourage everyone, including non-Americans, to go to one of these events or at least to the Government Accountability Office website at www.gao.gov. In essence, current US fiscal policy is unsustainable. Promised obligations such as Medicare, Medicaid and Social Security as well as servicing the interest of the debt will be 40% of GDP by 2040. Currently, revenue coming mostly form taxes is less than 20% GDP. Hence, either revenues must increase or spending must be cut. Whatever choice is made could lead to dire consequences.

Raising taxes is one solution but that could have adverse effects on the economy. I think the US could sustain some tax increases, especially on the rich, without bad effects but not enough to cover the deficit spending. Much of the current economy is geared towards providing services and goods to the well heeled. If disposable income starts to go down the first purchases to go will be luxury items like yachts, expensive restaurants, ski vacations, and so forth. People in these professions and industries will then lose their jobs putting more strain on social services. The very rich will be insulated from tax increases but the upper middle class, from which most of the tax revenue is extracted, would have to cut down on consumption. Additionally, with the real estate bubble of the past five years, many people are stuck with mortgages that they can barely sustain. Any increase in taxes could push them over the edge. So while taxes can be increased, it really can't be increased too much.

The other option is to cut spending. The retirement age can be raised to reduce the social security obligation. However, social security is actually in pretty good shape compared to medicare and medicaid. Something eventually will be done about these two programs. Hence, medical services and reimbursements to health care professionals will both likely be reduced. Basically, the US could end up being a nation where the poor and elderly will not receive first world medical care. This may be one other reason a complete overhaul of the health care system may be necessary.