Scientific Clearing House

New Location for Scientific Clearing House

2008-11-30T23:12:00.006-05:00

Scientific Clearing House has a new web address. It is now at sciencehouse.wordpress.com. The reason I'm moving is because Wordpress supports latex commands. I've been wanting to have the option of putting equations and mathematical symbols into my blog posts but it is impossible to do so in blogger. I tried using images of equations in my last post but it was extremely painful and didn't work all that well. I have started to migrate my previous posts to the new address and I'll eventually move all of my posts over.

Unwinding CDS's

2008-11-26T18:09:00.004-05:00

As pointed out several times by Steve Hsu recently, a major instigator of our current financial crisis is the Credit Default Swap (CDS). As far as financial instruments go, this one is almost understandable. Basically, it is an insurance policy that is exchanged between two firms. So, say you just loaned a bunch of money and you want to insure it. Well, you can buy a CDS for some fee from someone who will pay you some agreed upon amount if the loan goes bad. It is a zero sum game, which is why the amounts insured (notional amount) can be larger than the GDP of the entire world. Although the notional amounts of these CDS's could be large, the big banks and hedge funds that traded them are often hedged so that the net gain or loss are manageable. The problem was that when Lehman Brothers went down, the whole network became unbalanced and some parties were exposed to huge losses. However, given that there is no market for these things, no one knows who is holding what. Steve drew a complex graph and an example to demonstrate how difficult it would be to unwind everything.

I like to visualize this as a problem of flux balance. For example, let be a CDS payout from party j to i. Then the total net gain or loss (i.e. flux) for party i is given by

We see that the sum over is zero verifying that it is a zero sum game. It actually would be a simple matter to unwind all the obligations if everyone agreed to do it all at once. This could even be done without anyone disclosing any of their trades. People may want to do this so others wouldn't know how weak they were. The way you would do it is for everyone to compute their net flux and disclose this amount to a central clearinghouse. The clearing house then checks to see if the sum of all the fluxes is zero. If the sum is not zero then either someone made a mistake or tried to cheat. If it sums to zero then those with a negative flux would deposit that amount to the clearinghouse and those with a positive flux could then withdraw from it. It is possible to cheat if you collude with someone else so that the net change to the sum of your two fluxes is zero. However, that would not affect anyone else so it would be like a private deal between two parties.

Formal Proof

2008-11-22T19:39:00.008-05:00

This month's Notices of the American Mathematical Society is highlighting formal proof, which is a proof where every step is justified with respect to the axioms of mathematics so there is no chance of error. Generally, this would be extremely tedious to do by hand so people have been developing computerized proof assistants to help do the job. My good friend and former colleague at the University of Pittsburgh, Tom Hales penned one of the articles. I actually lured Tom from Michigan to Pittsburgh several years ago. At that time, Tom had recently finished his proof of the Kepler conjecture, which asserts that the densest arrangement of balls in three dimensional space is face-centered cubic (FCC) packing. He had been battling with the editors of the Annals of Mathematics to get his proof published. The problem was that Tom's proof was quite long (300 pages) and used a computer. Tom's strategy for the proof was to show that all possible packings in infinite space could be reduced to a finite number of arrangements, which needed to be tested individually. Tom and a student then wrote a program with about forty thousand lines that used interval arithmetic to show that none of these other arrangements was ever denser than FCC. A panel of experts worked on reviewing the proof and essentially gave up. They were pretty sure it was right but were unwilling to certify it. Eventually, the Annals published the proof with an asterisk that it was not fully checked by referees. I may have played a small part in helping Tom finally get the proof published. I remember going down to Tom's office one day. He was slumped in his chair and declared that he was giving up on the Annals and writing a book instead. I convinced him not to give up and gave him some tips on the strategy to deal with the editors and referees. I'm not sure exactly what happened but the next time I saw Tom he was beaming that the work would finally be published.

However, Tom was so frustrated with the whole episode that he decided that he needed to prove the Kepler conjecture formally, so that the asterisk could be removed. He began what he called the Flyspeck project, which is a stylized acronym for Formal Proof of Kepler. He's already recruited a number of mathematicians around the world to work on Flyspeck and nearly half of the computer code used to prove the Kepler conjecture is now certified. He estimates that it may take as long as twenty work years to complete the project so if he gets enough people interested it can be done quite quickly in real time.

A formal proof essentially involves translating mathematics into symbol manipulation that is encapsulated within a foundational system. Tom uses a system called HOL Light (an acronym for lightweight implementation of Higher Order Logic), which I'll summarize here. The details are fairly technical; they involve using a new axiomatic or logical system that involves types (similar to what is used in computer languages like C). HOL Light differs slightly from the Zermelo-Fraenkel-Choice axioms of set theory used in traditional mathematics. The use of types means that certain incongruous operations that any mathematician would deem nonsensical (like taking the union of a real number and a function), would automatically be disallowed. The system then involves mathematical statements or objects called terms involving symbols and logical operations or inference rules. Theorems are expressed as a set of terms called the sequent that imply the truth (or more accurately the provability) of another term called the conclusion. Proofs are demonstrations that using the allowed inference rules and axioms, it is possible to arrive at the conclusion.

What I like about formal proofs is that it reduces mathematics to dynamical systems. Each term is a point in the theorem space of all terms (whatever that means). A proof is then a trajectory between the initial condition (the sequent) and the conclusion. Technically, a formal proof is not a true dynamical system because the next step is not uniquely specified by the current state. The fact that there are multiple choices at each step is why theorem proving is hard. Interestingly, this is connected to the famous computer science problem of whether or not P=NP. Theorem proving is in the complexity class NP because any proof can be verified in polynomial time. The question is whether or not a proof can be found polynomial time. If it can be shown that this is possible then you would have a proof of P=NP and get a million dollars from the Clay foundation. It would also mean that you could prove the Riemann Hypothesis and all the other Clay Millennium problems and collect for those as well. In fact, if P=NP you could prove all theorems in polynomial time. This is one of the reasons why most people (including myself) don't think that P=NP.

I think it would be very interesting to analyze the dynamics of formal proof. So many questions immediately come to my mind. For example, what are the properties of theorem space? We know that the set of all theorems is countable but the set of possible terms is uncountable. A formal system consists of the space of all points reachable from the axioms. What does this look like? Can we define a topology on the space of all terms? I suppose a metric could be the fewest number of steps to get from one term to another term, which might be undecidable. Do people even think about these questions?

Explanation versus narration

2008-11-16T12:07:00.008-05:00

Noted Harvard economist Greg Mankiw wrote an op-ed last week for the New York Times and posted to his blog, a letter to the president-elect. One of his recommendations was to listen to economists. Following up on Steve Hsu's post on intellectual honesty, I think this exhibits an element of hubris since the majority of economists did not foresee this current financial meltdown. There were certainly those that were warning about the collapse of the housing bubble, like Robert Shiller, but other than Nouriel Roubini, I didn't hear too much about it causing the worst crisis since the Great Depression. Even Paul Krugman admits that this took him by surprise. I could see that there was a housing bubble back in 2004, which is why I haven't bought a house yet, but I had no idea that the bursting of a bubble could cause so much damage. The bursting of the internet bubble caused a lot of pain to some people but did not destroy the financial system.

The current crisis first became public knowledge when Bear Stearns went under in March of this year. Federal Reserve Chairman Ben Bernanke quickly engineered a buy out of Bear by JPMorgan and the market calmed for a while. Then in quick succession starting in September came the bailout of Fannie Mae and Freddie Mac, the sale of Merrill Lynch to Bank of America, the bankruptcy of Lehman Brothers, and the bailout of A.I.G. Shortly afterwards, Treasury Secretary Hank Paulson went to Congress to announce that the entire financial system is in jeopardy and requested 700 billion dollars for a bail out. The thinking was that banks and financial institutions had stopped lending to each other because they weren't sure which banks were sound and which were on the verge of collapse. The money was originally intended to purchase suspect financial instruments in an attempt to restore confidence. The plan has changed since then and you can read Steve Hsu's blog for the details.

What I want to point out here is that a narrative of what happened is not the same as understanding the system. There were certainly a lot of key events and circumstances starting in the 1980's that may have contributed to this collapse. There was the gradual deregulation of the financial industry including the Gramm-Leach-Bliley Act in 1999 that allowed investment banks and commercial banks to coalesce and the Commodity-Futures-Modernization Act in 2000 that ensured that financial derivatives remained unregulated. There was the rise of hedge funds and the use of massive amounts of leveraging by financial institutions. There were low interest rates following the internet bubble that fueled the housing bubble. There was the immense trade deficit with China (and China's interest in keeping the US dollar high) that allowed low interest rates to persist. There was the general world savings glut that allowed so much capital to flow to the US. There was the flood of physicists and mathematicians to Wall Street and so on.

Anyone can create a nice story about what happened and depending on their prior beliefs they can be dramatically different - compare George Soros to Phil Gramm. We really would like to understand in general how the economy and financial markets operate but we only have one data point. We can never rerun history and obtain a distribution of outcomes. Thus, although we may be able to construct a plausible and consistent story for why an event happened we can never know if it were correct and even more importantly we don't know if that can tell us how to prevent it from happening again. It could be that no matter what we had done, a crisis would still have ensued. My father warned of a collapse of the capitalistic system his entire life. I'm not sure how he would have felt had he lived to see the current crisis but he probably would have said it was inevitable. Or perhaps, if interest rates were a few points higher nothing would have happened. The truth is probably somewhere in between.

Another way of saying this is that we have a very large complex dynamical system and we have one trajectory. What we want to know about are the attractors, the basins of attraction, and the structural stabilty of the system. These are things that are difficult to determine even if we had full knowledge of the underlying dynamical system. We are trying to construct the dynamical system and infer all these properties from the observation of a single trajectory. I'm not sure if this task is impossible (i.e. undecidable) but it is certainly intractable. I don't know how we should proceed but I do know that conventional economic dogma about efficient markets needs to be updated. Theorems are only as good as their axioms and we definitely don't know what the axioms are for sure. I think the sooner economists own up to the fact that they really don't know and can't know what is going on, the better we will be.

Dead Zones

2008-11-08T12:20:00.004-05:00

Marine dead zones are regions of the ocean, usually near the mouths of rivers or waterways, which receive a large amount of nutrient (phosphorous and nitrogen) run off mostly from fertilizer. This causes a great phytoplankton and algae bloom that takes up carbon dioxide. However, when they die they sink to the ocean floor where other aerobic bacteria break them down with such vigor that they deplete the oxygen supply leaving an anoxic zone that cannot support marine life. The environmental movement is striving to curb fertilizer use in an attempt to mitigate these dead zones. There are also theories that the increase in the number of these dead zones are related to global warming.

I have a heretical thought on this regard and I haven't been able to find any information on it so if anyone knows please enlighten me. The earth's oxygen was originally created by cyanobacteria, which make up the algae that are causing the dead zones. So, could these dead zones actually be removing and sequestering carbon dioxide? Once the ocean bottom becomes anoxic, would the phytoplankton and algae fecal matter and remains pile upon the ocean floor and turn into fossil fuels in a few hundred million years? I don't think we should necessarily encourage dead zones but is there any data out there that they could be mitigating global warming? I don't want to be another one of those staunchly leftist youths going conservative in their old age so please set me straight if I'm wrong.

Rationality and politics

2008-11-02T12:22:00.016-05:00

I've been trying to reconcile the current political environment in terms of a consistent framework. In particular, I've been interested in dissecting how issues have been divided between the so-called left and right in the United States. My premise is based on ideas set down in my previous post on the genetic basis of political orientation. In that post I proposed that the political thesis of the right is that the wealth should be distributed according to a person's direct contribution while the left's premise is that wealth should be distributed equitably regardless of standing in the community. I think these are fair definitions based on historical notions of the right and left. What I want to do now is to see how current issues should be divided between these positions in a perfectly rational world.

Let me first summarize some positions currently held by the US right: 1) low taxes, 2) small government, 3) deregulation of industries, 3) free trade, 4) gun rights, 5) strong military, 6) anti-abortion, 7) anti-gay rights, and 8) anti-immigration. I would say positions 1) through 4) seem consistent with the historical notion of the right (although regulation can be consistent with the right if it makes markets more transparent), position 5) is debatable, while positions 6) through 8) seem dissonant. The left generally but not always take the opposite positions except possibly on point 5), which is mixed. The strong military position was understood as a right wing position during the Cold War because of the opposition to communism. The rationale for a strong military waned after the fall of the Soviet Union but 9/11 changed the game again and now the military is justified as a bulwark against terrorism.

The question is how to reconcile positions 6) through 8) and we could add pro-death penalty and anti-evolution into the mix as well. These positions are aligned on the right because of several historical events. The first was that many of the early settlers to the United States came to escape religious persecution at home and this is why there is a significant US Christian fundamentalist population. The second is slavery and the Civil Rights movement. The third is that middle class whites fled the cities for the suburbs in the 50's and 60's. These people were probably religious but a genetic mix between left and right.

In the early 20th century, the Republicans were an economic right wing party while the Democrats under FDR veered to the left although they were mostly Keynesian and not socialist. The South had been Democratic because Lincoln was a Republican. The Civil Rights movement in the 60's angered and scared many suburban and southern whites and this was exploited by Nixon's "Southern Strategy", which flipped the South to the Republicans. This was also a time of economic prosperity for the middle class so they were more influenced by issues regarding crime, safety, religion, and keeping their communities "intact". Hence, as long as economic growth continued, there could be a coalition between the economic right and the religious right since the beliefs of both sides didn't really infringe on each other. Hence, culturally liberal New York bankers could coexist with culturally conservative southern factory workers.

Let me now go through each point and see if we can parse them rationally. I will not try to ascribe any moral or normative value to the positions, only on whether or not it would be consistent in a right or left worldview. I think low taxes, small government, deregulation and free trade certainly belong on the right without much argument. Gun rights seem to be consistent with the right since it is an anti-regulatory sentiment. Strong military is not so clear cut to me. It certainly helps to ensure that foreign markets remain open so that would help the right. However, it could also enforce rules on other people, which is more left. It is also a big government program, which is not so right. So my sense is that a strong military is neither right nor left. Abortion is quite difficult. From the point of view of the woman, I think being pro-choice is consistent with being on the right. Even if you believe that life begins at conception and I've argued before that defining when life begins is problematic, the fetus is also a part of the woman's body. From the point of view of the fetus, I think it's actually a left wing position to be pro-life. Gay rights seems to be clearly a right wing position in that there should not be any regulation on personal choice between consenting adults. However, if you view gay behavior as being very detrimental to society then as a left winger you could possibly justify disallowing it. So interestingly, I think being against gay rights is only viable from a left wing point of view. Anti-immigration is probably more consistent with the left since immigrants could be a competitive threat to one's job. A right winger should encourage immigration and more competition. Interestingly, anti-evolution used to be a left wing position. William Jennings Bryan, who was against evolution in the Scopes Monkey trial, was a populist Democrat. He was worried that evolution theory would justify why some people had more than others. Survival of the fittest is a very right wing concept.

I doubt that political parties will ever be completely self-consistent in their positions given accidents of history. However, the current economic crisis is forcing people to make economic considerations more of a priority. I think what will happen is that there will be a growth in socially conservative economic populism, which as I argued is probably more self-consistent. Republican presidential candidate Mike Huckabee is an example of someone in that category. The backlash against Johnson's Great Society and anti-poverty measures was largely racially motivated. However, as the generation that lived through the Cold War and the Civil Rights era shrinks in influence, I think a slow rationalizing realignment in the issues among the political parties may take place.

Addendum (11/4/2008)
I think it is appropriate to add on Election Day that I don't think either of the two major American political parties fall into the right or left camp as I've defined it. There are elements of both right and left (as well as a royalist bent) in both party's platforms.

Living in a simulation - part 2

2008-10-24T14:15:00.003-04:00

Suppose you are living in a simulation and you wanted to discover the theory of everything. What would that theory be? Probably in your (simulated) mind it would be the set of laws that govern all physical phenomena in your (simulated) observable universe. You would also want to understand how your universe came about and where it will end up. Let's suppose that the programmer of your universe came up with a set of physical laws and let it run. As I discussed before, the programmer really can't be sure what will happen in his simulation but let's say he was inspired or lucky and hit upon something that led to a universe that produced an inhabitant that could ask about the theory of everything.

What then is the theory of everything? Well, one answer would be the set of physical laws that the programmer put in. Now suppose that the programmer didn't come up with any laws but just started off a cellular automaton (CA) with some rules and an initial condition. An example of a CA, which Steve Wolfram's book "A new kind of science" describes in great detail, is a one dimensional grid of "cells" that can be in one of two states. At each time step, each cell is updated according to what state it and its nearest neighbors are in. There are thus 2^8=256 possible rule sets, since each of the 8 configurations that 3 contiguous cells can have yields two possible updated states for the middle cell. Wolfram has ennumerated all of them in his book.

One of Wolfram's former employees, Matthew Cook, proved that rule 110 is a universal computer. Hence, all possible computations (simulations) can be obtained by running through all possible initial conditions. One of these initial conditions corresponds to the program with the same physical laws that the inspired programer came up with. However, in this case the physical laws will be an emergent phenomenon of the CA. What then is the theory of everything? Is it the the set of rules of the cellular automaton? Is it the combination of the rules and the initial condition? Is it still the set of emergent physical laws? In all likelihood, the elementary constituents of the emergent theory will be comprised of some number of cells. Below this scale, the emergent theory will no longer hold and at the very lowest level, there will be rule 110.

Finally, as has been pointed out previously, the inhabitants of a simulation can never know that they are in a simulation. Thus, there is really no way for us to know if we are living in a simulation. So what does that say about our theory of everything? Will it be an uber string theory or the CA rule and initial condition? Would we want to have a theory of everything that included a theory of the programmer and the programmer's world? This is why I've come to adopt the notion that a theory of everything is a theory of computation and that doesn't really tell us much about our universe.

Absolute verus relative wealth

2008-10-19T15:38:00.002-04:00

There is a debate among sociologists, political scientists and economists on whether or not absolute wealth or relative wealth is more important. There seems to be a trend recently that happiness is linked more to relative wealth than absolute wealth. The number of people who say they are happy has not gone up with a rise in standard of living, and in fact it may have even come down. Also, a paper in the journal Science last year reported that activation in brain areas related to reward responded more to relative differences in wealth than absolute amounts. I recall reading an article recently about Silicon Valley millionaires feeling poor and unsatisfied because of the billionaires in their neighbourhood. There was a difference between being rich and being "plane"-rich.

However, the current economic turmoil is uncovering a more complex (or maybe obvious) interaction at play. The anti-correlation between the performance of the economy and the likelihood of a Democratic US president seems to indicate that there is a threshold effect for wealth. Happiness does not go up appreciably above this threshold but certainly goes down a great deal below it. For people above this threshold, other factors start to play a role in their political decisions and sense of well being. However, when you are below this threshold then the economy is the dominant issue.

This may be why the growth of income disparity did not create that much outcry over the past decade or so. When the majority of the population was above their wealth comfort threshold, they didn't particularly care about the new gilded age since the rich were largely isolated from them. It mostly caused unease among the rich that weren't keeping up with the super-rich. However, when the majority finally fell below their comfort threshold, the backlash came loud and strong. Suddenly, everyone was a populist. However, when (if?) the economy rights itself again then this regulatory fervor will subside in kind. The general public will tune out once again and the forces that pushed for policies favourable to unequal growth will dominate the political discourse.

The system may always be inherently unstable. Suppose that fervor for political activism and where you sit on the left-right divide are uncorrelated but have approximately equal representation. We can then divide people into four types - Active/Left, Active/Right, Nonactive/Left and Nonactive/Right. Also assume that when the economy is doing poorly, all the left are motivated but when the economy is doing well then all the right are motivated. A graph in the New York times yesterday showed that stock market growth was higher when democrats are in office, even when you don't count Herbert Hoover, who was in power during the crash of 1929. So let's assume that when the left is in power there is more total economic growth and less income disparity and when the right is in power there is less total growth but more income disparity. The nonactive fractions of the left and right determine the policy. When things are going well in the economy, the Nonactive/Left relax but the Nonactive/Right become motivated. Thus we have half the population pushing for more right leaning policies countered by only a quarter of the population, namely the Active/Left. This then leads to the right attaining power resulting in a widening of income disparity. When enough people fall below the wealth threshold, the Nonactive/Left become engaged while the Nonactive/Right disengages, which then allows the Left to come back into power. The interesting things is that the only way to break this cycle is for the right to enact policies that keep everyone above threshold. For the other stable fixed point, namely left wing policies failing completely and keeping everyone down would eventually lead to a breakdown of the system.

Genetic basis for political orientation

2008-10-17T20:27:00.003-04:00

I was listening to a podcast of Quirks and Quarks yesterday that featured an interview of political scientist James Fowler on his recent work showing that the likelihood to vote was partially genetic. (Fowler is the same person who recently argued in New England Journal of Medicine paper that obese people tend to have obese friends.) The likelihood that genes may play a role in politics has come up before most notably in a paper by John Alford, Carolyn Funk and John Hibbing in 2005 that argued that political leanings are heritable. That study looked at identical and fraternal twins and found that the heritability of political ideology was about 50%. The work didn't say that genes could predict party affiliation just how a person stood on the left-right divide on a number of issues. Fowler hypothesized that the reason politics has a genetic basis is that back in our hunter-gatherer days, figuring out how to divide the spoils of a hunt would be important to the survival of the troop.

Following Fowler, I can imagine how early humans could take two approaches to how to divide up a downed mastodon. The paleo-leftists would argue that the meat should be shared equally among everyone in the tribe. The rightwingers would argue that each tribe member's share should be based solely on how much they contributed to that hunt. My guess is that any ancient group that had approximately equal representation of these two opposing views would outcompete groups that had unanimous agreement of either viewpoint. In the rightwing society, the weaker members of the group simply wouldn't eat as much and hence would have a lesser chance of survival reducing the population and diversity of the group. The result may be a group of excellent hunters but perhaps they won't be so good at adapting to changing circumstances. Now in the proto-socialist group, the incentive to go out and hunt would be reduced since everyone would eat no matter what. This might make hunts less frequent and again weaken the group. The group with political tension may compromise on a solution where everyone gets some share of the spoils but there would be incentives or peer pressure to contribute. This may be why genes for left and right leanings have both persisted.

If this is true, then it would imply that we may always have political disagreement and the pendulum will continuously swing back and forth between left and right. However, this doesn't imply that progress can't take place. No one in a modern society tolerates slavery even though that was the central debate a hundred and fifty years ago. Hence, progress is made by moving the center and arguments between the left and the right lead to fluctuations around this center. A shrewd politician can take advantage of this fact by focusing on how to frame an issue instead of trying to win an argument. If she can create a situation where two sides argue about a tangential matter to the pertinent issue than the goal can still be achieved. For example, suppose a policy maker wanted to do something global warming. Then the strategy should not be to go out and try to convince people on what to do. Instead, it may be better to find a person on the opposite political spectrum (who also wants to do something about global warming) and then stage debates on their policy differences. One side could argue for strict regulations and the other could argue for tax incentives. They then achieve their aim by getting the country to take sides on how to deal with global warming, instead of arguing about whether or not it exists.

Complexity of art and science

2008-10-11T15:58:00.004-04:00

There seems to be a consensus that art cannot be compressed. A plot summary of Hamlet is not the same as Hamlet. A photo of Picasso's Guernica is not the same as the actual painting in Madrid. Music is a particulary interesting case. A Bach partita can be written down in a few thousand bits but reading the music is not the same as hearing it played by Heifetz or Menuhin. One could even argue that one performance by the same artist is not the same as a recording or even another performance.

This is in contrast to Science. We all know the theory of evolution but most of us have never read Darwin's On the Origin of Species. The three volumes of Newton's Principia Mathematica can now be reduced to F = ma and F = G m1 m2/r^2. Obviously, it takes some concerted study to understand these equations but one doesn't need to read Newton to do so. It is interesting that scientists tend to worry a lot about priority of a discovery while they are alive but unless their name is directly associated with a concept, theorem or equation, the provenance of many scientific ideas tend to get lost. Quantum mechanics is often taught before classical mechanics now so most starting students have no idea why the energy function is called a Hamiltonian. The concept of the conservation of energy is so natural to scientists now that most people don't realize how long it took to be established and who were the main players.

If art is not compressible then we can interpret the complexity of the brain in terms of the complexity of art. The complete works of Shakespeare runs a little over 1200 pages. Estimating 5000 characters per page and 8 bits per character leads to a total size of less than 50 million bits, which is not very much compared to the hard drive on your computer. Charles Dickens was much more prolific in terms of words generated. Bleak House alone is over 1000 pages. I haven't counted all the pages of all twenty plus novels but let's put his total output at say a billion bits.

If art is incompressible then that means there could not be an algorithm smaller than a billion bits that could have generated the work of Dickens. This would put a lower bound on the complexity of the "word generation" capabilities of the brain. Now perhaps if you are uncharitable (like some famous authors have been), you could argue that Dickens had a formula to generate his stories and so the complexity is actually less. One way to do this would be to take a stock set of themes, plots, characters, phrases and so on and then randomly assemble them. Some supermarket romances are supposedly written this way. However, no one would argue that they compare in anyway to Dickens, much less Shakespeare. Given that the Kolmogorov complexity is uncomputable we can never know for sure if art is compressible. So a challenge to computer scientists is to write a program that can generate literature with a program shorter than the work itself.

Modeling the financial crisis

2008-10-03T11:26:00.004-04:00

There is an interesting op-ed piece in the New York Times this week by physicist and science writer Mark Buchanan on predicting the current financial crisis. His argument is that traditional economists were unable to predict or handle the current situation (Nouriel Rubini notwithstanding) since their worldviews are shaped by equilibrium theorems, which unfortunately are either incomplete or wrong. Buchanan writes:

Well, part of the reason is that economists still try to understand markets by using ideas from traditional economics, especially so-called equilibrium theory. This theory views markets as reflecting a balance of forces, and says that market values change only in response to new information — the sudden revelation of problems about a company, for example, or a real change in the housing supply. Markets are otherwise supposed to have no real internal dynamics of their own. Too bad for the theory, things don’t seem to work that way.
Nearly two decades ago, a classic economic study found that of the 50 largest single-day price movements since World War II, most happened on days when there was no significant news, and that news in general seemed to account for only about a third of the overall variance in stock returns. A recent study by some physicists found much the same thing — financial news lacked any clear link with the larger movements of stock values.
Certainly, markets have internal dynamics. They’re self-propelling systems driven in large part by what investors believe other investors believe; participants trade on rumors and gossip, on fears and expectations, and traders speak for good reason of the market’s optimism or pessimism. It’s these internal dynamics that make it possible for billions to evaporate from portfolios in a few short months just because people suddenly begin remembering that housing values do not always go up.
Really understanding what’s going on means going beyond equilibrium thinking and getting some insight into the underlying ecology of beliefs and expectations, perceptions and misperceptions, that drive market swings.

He then goes on to describe the work of some pioneers who are trying to model the actual dynamics of markets. A Yale economist with two physicists (Doyne Farmer being one of them) used an agent-based model to simulate a credit market. They found that as the leverage (amount of money borrowed to amplify gains) increases there is a phase transition or bifurcation from a functioning credit market to an unstable situation that results in a financial meltdown.

I found this article interesting on two points. The first is the attempt to contrast two worldviews: the theorem proving mathematician economist versus the computational physicist modeler. The second is the premise that the collective dynamics of a group of individuals can be simpler than the behavior of a single individual. A thousand brains may have a lower Kolmogorov complexity than a single brain. My guess is that biologists (Jim Bower?) may not buy this. Although, my worldview is more in line with Buchanan's, in many ways his view is on less stable ground than traditional economics. With an efficient market of rational players, you can at least make some precise statements. Whereas with the agent-based model there is little understanding as to how the models scale and how sensitive the outcomes depend on the rules. Sometimes it is better to be wrong with full knowledge than be accidentally right.

I've always been intrigued by agent-based models but have never figured how to use them effectively. My work has tended to rely on differential equation models (deterministic and stochastic) because I generally know what to expect from them. With an agent-based model, I don't have a feel for how they scale or how sensitive they are to changes in the rules. However, this lack of certainty (which also exists for nonlinear differential equations, just look at Navier-Stokes for example), may be inherent in the systems they describe. It could simply be that some complex problems are so intractable that any models of them will rely on having good prior information (gleened from any and all sources) or plain blind luck.

Living in a simulation

2008-09-26T13:07:00.005-04:00

There has been a lot of press lately (summarized here) on the possibility that we are living inside of a computer simulation. Much of the attention has been focused on whether or not you could know if you lived in a simulation. Here, I will focus on what you could know (or compute) if you were running the simulation. Before I proceed, I'll briefly summarize some points about the theory of computation. I've alluded to these ideas in several of my recent posts but have never formally introduced them. Obviously, this is very deep area so I'll just briefly summarize some important points.

The theory of computation is basically about repeated operations on a finite set of symbols according to some rule. The paradigm is the Turing machine, which consists of a finite number of internal states and a tape on which symbols can be written or erased. The Turing machine then makes transitions from state to state based on its current state and the symbols on the tape. One of the states is for the Turing machine to halt, which signals the end of a computation. The important thing to take away is the Church-Turing thesis, which basically states that all forms of computation on finite symbol sets are basically equivalent. For example, the computer on your desk is equivalent to a Turing machine. (Actually, it is even less powerful because it is finite but I digress).

One of the things that Turing proved was that there is a universal Turing machine, which can emulate any other Turing machine. The input to a Turing machine is a set of symbols on the tape, i.e. an input string. Some of the symbols code for instructions or rules and the others code for an initial condition. Turing also showed that there are problems for which a Turing machine can never solve. The most famous is the Halting Problem, which states that there does not exist a Turing machine that can decide if another Turing machine will halt given some input. Turing actually showed that it was impossible to produce a general algorithm to decide if a given input string to a Turing machine will ever cause it to print a given symbol. In other words, there is no general algorithm to decide if a computation will have an infinite loop or perform some specific task. This doesn't imply that you couldn't prove that a specific program has this property, just that there isn't a way to do it generally. The proof of the Halting problem is similar to Cantor's diagonal proof that the set of real numbers is uncountable.

One of the consequences of Turing's work is that the total number of possible computations is countable. You simply take all strings of length 1, then 2, etc and feed it to a Universal Turing machine. Every possible computation or numerical simulation will be included. Thus, any simulation of the universe is coded in one of these strings. Some of these inputs will lead to computations that will halt and some will run forever. Some will make the Turing machine print a particular symbol and some will not. However, there is no way to decide, which of the input strings are on any of these lists.

The question is then, given an input string, can you determine if it will produce a universe that has some property such as supporting life. There are actually two separate issues regarding this question. The first is, how would you even define life or recognize it in the computation. I will deal with this in a future post. The second is, given that you have a definition of life, then can you know ahead of time whether or not your simulation will produce it. The answer to this question is no because if it were yes then you could solve the Halting problem. This is easy to see because any definition of life must involve some pattern of symbols to be printed on the tape and there is no way to decide if an input string will ever produce a symbol much less a pattern. This doesn't mean that a simulator couldn't come up with a simulation of our universe, it just means that she could never come up with a general algorithm to guarantee it. So, in the infinite but countable list of possible compuations, some produce simulations of universes, perhaps even ours, but we can never know for sure which.

Complexity of the brain

2008-09-20T20:17:00.003-04:00

The Kolmogorov complexity of an object is the length of the minimal description of that object. In terms of the brain, it would correspond to the length of the smallest computer program that could reproduce the brain. It could also be thought of as the amount of information necessary to model the brain. Computing the Kolmogorov complexity is not possible since it is an undecidable problem but we can estimate it. If we presume that molecular biology is computable then one estimate of the Kolmogorov complexity of the brain is given by the length of the genome, which is 3 billion base pairs long or 6 billion bits. To be conservative, we could also include the genome of the mother and baby, which implies 12 billion bits. This corresponds to less than two billion bytes and easily fits on a DVD. Hence in principle, we could potentially grow a brain with less than 12 billion bits of information and this is probably an upper bound.

However, this would not imply that we could describe a brain with this amount of information since it ignores the modifications due to external inputs. For example, the visual system cannot fully develop if the brain does not receive visual inputs. So we also need to estimate how much input the brain receives during development. The amount of information available in the external world is immense so it is safe to assume that the amount received is limited by the brain and not the source. However, there is no way to estimate this in a principled way since we don't know how the brain actually works. Depending on what you assume to be the neural code (see previous post), you could end up with a very wide range of answers. Nonetheless, let's suppose that it is a rate code with a window of 10 ms. Generally, neurons fire at rates less than 100 Hz so this corresponds to the presence or absence of a spike in a 10 ms window. This corresponds to 100 bits per neuron per second. The brain has about 10^11 neurons so the maximum amount of information that could be input to the brain is 10^13 bits per second. There are over 30 million seconds in a year, so that is a lot of information and can easily dwarf the genomic contribution.

However, this does lead us to a potential means to quantify the influence of genes versus environment on intelligence and behaviour debate. If the complexity of the brain is less than 12 billion bits then we are basically genetically determined. If it is greater, then we are mostly shaped by the environment. So what do you think?

Contrasting worldviews - Part 2

2008-09-13T16:09:00.022-04:00

Compared to biologists and mathematicians, physicists are much more uniform in their worldview. There is a central canon of physics that everyone is taught. With respect to how physicists approach biology especially with regards to how many details to include, their ideas are shaped by the concept of universality and the renormalization group, which I will explain below. This is partly what gives physicists the confidence that complex phenomena can have simple explanations although this physics worldview hegemony is starting to break as physicists become more immersed in biology. In fact, I've even noticed a backlash of some physicists cum biologists towards their colleagues that espouse the notion that details can be dispensed with. Some physicists are very much of the notion that biologically detailed modeling is necessary to make progress. In this sense, what I'm describing might be more appropriately called the old physics world view.

The concept of universality arose from the study of phase transitions and critical phenomena with inspiration from quantum field theory. In a nutshell, it says that for certain systems in regimes where there is no obvious length scale (usually indicated by power law scaling), such as at the critical point of a second order phase transition, the large scale behavior of the system is independent of the microscopic details and only depends on general properties such as the number of dimensions of the space and symmetries in the system. Hence, systems can be classified into what are called universality classes. Although, the theory was developed for critical phenomena in phase transitions, it has since been generalized to apply to a wide range of dynamical situations such as earthquakes, avalanches, flow through porous media, reaction diffusion systems and so forth.

The paradigmatic system for critical phenomena is magnetism. Bulk (ferro)magnetism arises when the atoms (each of which have a small magnetic moment) align and produce macroscopically observable magnetization. However, this only occurs for low temperatures. For high enough temperature, the random motions of the atoms can destroy the alignment and magnetization is lost (material becomes paramagnetic). The change from a state of ferromagnetism to paramagnetism is called a phase transition and occurs at a critical temperature (the Curie temperature).

These systems are understood by considering the energy associated with different states. The probability of occupying a given state is then given by the Boltzmann weight, which is exp(-H(m)/kT), where H(m) is the internal energy of the state with magnetization m (also called the Hamiltonian), T is the temperature, and k is the Boltzmann constant. Given the Boltzmann factor, the partition function (sum of Boltzmann weight over all states) can be constructed from which all quantities of interest can be obtained. Now this particular system was studied over a century ago by notables such as Pierre Curie, who using known microscopic laws of magnetism and mean field theory, found that below a critical temperature Tc, m is nonzero and above Tc m is zero.

However, the modern way of how we think of phase transitions starts with Landau, who first applied it to the onset of superfluidity of helium. Instead of trying to derive the energy from first principles, Landau said let's write out a general form based on the symmetries of an order parameter, which in this example is the magnetization m(x), at spatial location x. Since the energy must be a scalar, it can only depend on terms like |m|^2 or (grad m)^2. The first few terms then obey H ~ \int dx q (grad m)^2 - (T-Tc) m^2 + u m^4 + ..., for parameters q, T, and u. The (grad m)^2 term is due to fluctuations. If fluctuations are ignored, then this is called mean field theory, in which case H ~ -(T -Tc)/2 m^2 + u m^4. The partition function can be estimated by using a saddle point approximtion, which in the mean field limit amounts to evaluting the critical points of H, which are m=0 and m^2=(Tc-T)/4 u. They correspond to the equilibrium states of the system, so if T is greater than Tc then the only solution is m=0 and if T is less than Tc then the magnitude of magnetization is nonzero.

The partition function cannot be explicitly computed in the presence of fluctuations. This is where Ken Wilson and the renormalization group comes in. What Wilson said, following people before him like Murray Gell-Mann, Francis Low and Leo Kadanoff, is suppose we have scale invariance, which is true near a critical point. Then if we integrate out small length scales (or high spatial frequency scales), rescale in x, and then renormalize m, we will end up with a new partition function, but with slightly different parameters. These operations form a group action (i.e dynamical system) on the parameters of the partition function. Thus, a scale invariant system should be at a fixed point of the renormalization group action. In other words, if you keep applying the renormalization group, the parameters can flow to a fixed point and the location of the fixed point only depends on the symmetry of the order parameter and dimension of the space. Many different systems can flow to the same fixed point. The most important element of the renormalization group in terms of the physics worldview is that terms in the Hamiltonian are renormalized in different ways. Some grow, these are called the relevant operators, some stay the same, these are marginal operators, and some decrease, these are called irrelevant operators. For critical systems, only a small number of terms in the Hamiltonian are relevant and this is why microscopic details do not matter at large scales.

Now, these ideas were originally developed just for behavior near a critical point, which is pretty specialized. If it were only applicable to an equilibrium phase transition, then physicists really wouldn't have a leg to stand on in terms of ignoring details. However, these ideas were later generalized to dynamical systems with critical behavior. What also motivates them is that power laws (also called 1/f or fractal scaling) seem to be ubiquitous. They can been found in the size distribution of earthquakes, thermal noise in resistors, size of river meanders, the coastline of Norway, size of hubs in the Internet, connectivity of protein networks, and even neural firing patterns, to name a few. Although there is not an agreement as to why these systems exhibit power laws (many theories have been proposed), the spectre of the renormalization group and universality permeates the air and influences the physicist world view.

My personal view is that some details matter immensely while others do not. However, there is no a priori systematic way of deducing which is which. There are only rules of thumb and experience that can assist us. Hence, even if you buy into the details-may-not-matter worldview, there is no prescription for how to implement it. What it does do is give me less confidence that there is such a thing as the "correct" theory for a system. I'm more inclined to believe that given the current state of knowledge and a specific set of questions, some theories perform better than others. With more information, we can refine our theories. However, I don't think this process ever converges to "the" theory because specifying what a system is is somewhat arbitrary. Nothing is purely isolated from its surroundings, so drawing a boundary is always going to involve a choice. These could be very logical and well informed choices but choices nonetheless. Also, we can never have full control of all the external inputs that can affect a system. In this way, I have a Bayesian viewpoint in that we only make progress by updating our priors.

Contrasting worldviews - Part 1

2008-09-06T09:48:00.011-04:00

In my previous post, I talked about how we probably needed a new worldview before we would be prepared to understand the brain. What I thought I would do here is to introduce what I think forms the worldviews of people who do dynamical systems (which forms a sizable contingent of the mathematical neuroscience community) and physics (in particular statistical mechanics and field theory). Having trained in physics and applied mathematics, served on the faculty in a math department and worked on biology, I've gotten a chance to see how these different groups view science. The interesting thing is that in many ways biologists and mathematicians can sometimes understand each other better than physicists and mathematicians. I was led to this belief after hearing Alla Borisyuk, who is an applied mathematician, exclaiming at a conference I helped organize in 2000 for young researchers, that she had no trouble talking to biologists but had no idea what the physicists were talking about.

The reason why biologists may have more in common with mathematicians than physicists is because unlike physics, biology has no guiding laws other than evolution, which is not quantitative. They rarely will say, "Oh, that can't be true because it would violate conservation of momentum," which was how Pauli predicted the neutrino. Given that there are no sweeping generalizations to make they are forced to pay attention to all the details. They apply deductive logic to form hypotheses and try to prove their hypotheses are true by constructing new experiments. Pure mathematicians are trained to take some axiomatic framework and prove things are true based on them. Except for a small set of mathematicians and logicians, most mathematicians don't take a stance on the "moral value" of their axioms. They just deduce conclusions within some well defined framework. Hence, in a collaboration with a biologist, a mathematician may take everything a biologist says with equal weight and then go on from there. On the other hand, a physicist may bring a lot of preconceived notions to the table (Applied mathematicians are a heterogeneous group and their world views lie on a continuum between physicists and pure mathematicians.) Physicists also don't need to depend as much on deductive logic since they have laws and equations to rely on. This may be what frustrates biologists (and mathematicians) when they talk to physicists. They can't understand why the physicists can be so cavalier with the details and be so confident about it.

However, when physicists (and applied mathematicians) are cavalier with details, it is not because of Newton or Maxwell or even Einstein. The reason they feel that they can sometimes dispense with details is because their worldviews are shaped by Poincare, Landau and Ken Wilson. What do I mean by this? I'll cover Poincare (and here I use Poincare to represent several mathematicians near the turn of the penultimate century) in this post and get to Landau and Wilson in the next one. Poincare, among his many contributions, showed how dynamical systems can be understood in terms of geometry and topology. Prior to Poincare, dynamical systems were treated using the tools of analysis. The question was: Given an initial condition, what are the analytical properties of the solutions as a function of time? Poincare said, let's not focus on the notion of movement with respect to time but look at the shape of trajectories in phase space. For a dynamical system with smooth enough properties, the families of solutions map out a surface in phase space with tiny arrows pointing in the direction the solutions would move on this surface (i.e. vector field). The study of dynamical systems becomes the study of differential geometry and topology.

Hence, any time dependent system including those in biology that can be described by a (nice enough) system of differential equations, is represented by a surface in some high dimensional space. Now, given some differential equation, we can always make some change of variables and if this variable transformation is smooth then the result will just be a smooth change of shape of the surface. Thus, what is really important is the topology of the surface, i.e. how many singularities or holes are in them. The singularities are defined by places where the vector field vanishes, in other words the fixed points. Given that the vector field is smooth outside of the fixed points then the global dynamics can be reconstructed by carefully examining the dynamics near to the fixed points. The important thing to keep track of when changing parameters is the appearance and disappearance of fixed points or the change of dynamics (stability) of fixed points. These discrete changes are called bifurcations. The dynamics near fixed points and bifurcations can be classified systematically in terms of normal form equations. Even for some very complicated dynamical system, the action is focused at the bifurcations. These bifurcations and the equations describing them are standardized (e.g. pitchfork, transcritical, saddle node, Hopf, homoclinic) and do not depend on all the details of the original system. Thus, when a dynamical systems person comes to a problem, she immediately views things geometrically. She also believes that there may be underlying structures that capture the essential dynamics of the system. This is what gives her confidence that some details are more important than others. Statistical mechanics and field theory takes this idea to another level and I'll get to that in the next post.

Understanding the brain

2008-08-31T09:37:00.008-04:00

I think that sometimes philosophy is important and this may be true for neuroscience right now. I don't mean ivory tower, "what is life?" type philosophy (although that is important too) but trying to pin down what it would mean to say "we understand the brain." When would we know that the game is won? I think this is important for neuroscience now to help to guide research. What should we be doing?

So what does it mean to understand something? I would say there are two aspects. One is predictive power, which would mean that we would be able to know what drugs or therapies would be useful to cure a brain disorder. The second aspect is more difficult to pin down but would basically mean incorporating something seamlessly into your worldview. The simplest example I can give is a mathematical theorem. Predictive understanding would correspond to the ability to follow all the steps of the proof of the theorem and use the theorem to prove new theorems. Incorporative understanding would be the ability to summarize the proof in a way that relates it in a highly compressed form to things you already know. For example, we can understand bifurcations of complicated dynamical systems by reducing them to the behavior of solutions of simple polynomial equations.

Sometimes the two views can clash. Consider the proof of the Kepler Conjecture for sphere packing by my friend and former colleague Tom Hales. The theorem is difficult because there are an infinite number of ways to pack spheres in 3 dimensions. Hales made this manageable by showing that this could be reduced to solving a finite (albeit large) optimization problem. He then proceeded to solve the finite problem computationally. To some people, the proof is a done deal. The trick was to reduce it to a finite problem, after that it is just details. Even if you don't believe Hales's computation you could always repeat it. Others would say, it is not done until you have a complete pen and pencil proof. To me, I think the proof is understandable because Hales was able to reduce it to an algorithm. However this is not a view that everyone shares.

Now we come back to the brain. What would you consider understanding to entail? I'm not sure that we, namely people working in the field today, will ever have that satisfying incorporating understanding of the brain because we don't have anything in our current worldview that could encapsulate that understanding. We will never be able to say, "Oh right, I understand, the brain is like X." In that sense, it is like quantum mechanics (QM). This is a theory that is highly successful in the predictive sense. As a predictive theory, it is quite simple. There are just a few rules to apply and much of our modern technology like lasers and electronics rely on it. However, no one who has ever thought about it would claim any understanding of QM in the incorporation sense. The Copenhagen interpretation is basically a "Don't ask, don't tell" policy for the theory.

In this sense, trying to understand any complex system, no matter how unrelated it is to the brain, could help in the long run to provide a foundation for an incorporating understanding of the brain. That is not to say that I believe there are laws of complex system similar to classical and quantum mechanics. My own view is that there are no laws in complex systems such as the global climate, economics or the brain; there are just effective theories that sort of work in limited circumstances. However, it is by slowly creating effective theories and models that we will form a new worldview of what it means to understand complex systems like the brain. In the meantime, we should continue to try to build a predictive understanding so that we can cure diseases and treat disorders.

Materialism and meaning

2008-08-21T12:45:00.017-04:00

Let me first say that I am a die hard materialist in that I do believe that there is nothing beyond the physical world. I also believe that physics and hence the physical world is computable in that it can be simulated on a computer. However, helped along by Stuart Kauffman's new book Reinventing the Sacred, I have been gradually edging towards accepting that even in a purely materialistic world there is some amount of arbitrariness in our perception of reality. Kauffman argues that this arbitrariness is not "mathematizable". I will argue here that the question can be formulated mathematically and can be shown to be undecidable or at best intractable. Kauffman's thesis is that we should take advantage of this arbitrariness and make it the foundation of a new concept of the sacred.

The problem arises from how we assign meaning to things in the world. Philosophers like Wittgenstein and Saul Kripke have thought very deeply on this topic and I'll just barely scratch the surface here. The simple question to me is what do we consider to be real. I look outside my window and I see some people walking. To me the people are certainly real but is "walking" real as well? What exactly is walking? If you write a simple program that makes dots move around on a computer screen and show it to someone then depending on what the dots are doing, they will say the dots are walking, running, crawling and so forth. These verbs correspond to relationships between things rather than things themselves. Are verbs and relationships real then? They are certainly necessary for our lives. It would be hard to communicate with someone if you didn't use any verbs. I think they are necessary for an animal to survive in the world as well. A rabbit needs to classify if a wolf is running or walking to respond appropriately.

Now, once we accept that we can ascribe some reality or at least utility to relationships then this can lead to an embarrassment of riches. Suppose we live in a world with N objects that you care about. This can be at any level you want. The number of ways to relate objects in a set is the number of subsets you can form out of those objects. This is called the power set and has cardinality (size) 2^N. But it can get bigger than that. We can also build arbitrarily complex arrangements of things by using the objects more than once. For example, even if you only saw a single bird, you could still invent the term flock to describe a collection of birds. Another way of saying this is that given a finite set of things, there are an infinite number of ways to combine them. This then gives us a countable infinity of items. Now you can take the power set of that set and end up with an uncountable number of items and you can keep on going if you choose. (Cantor's great achievement was to show that the power set of a countable set is uncountable and the power set of an uncountable set is even bigger and so forth). However, we can probably only deal with a finite number of items or at most a countable list (if we are computable ). This finite or countable list encapsulates your perception of reality and if you believe this argument then the probability of obtaining our particular list is basically zero. In fact, given that the set of all possible lists is uncountable, this implies that not all lists can even be computed. Our perception of reality could be undecidable. To me this implies an arbitrariness in how we interact with the physical world which I call our prior. Kauffman calls this the sacred.

Now you could argue that the laws of the material world will lead us to a natural choice of items on our list. However, if we could rerun the universe with a slightly different initial condition would the items on the list be invariant? I think arbitrarily small perturbations will lead to different lists. An argument supporting this idea is that even among different world cultures we have slightly different lists. There are concepts in some languages that are not easily expressible in others. Hence, even if you think the list is imposed by the underlying laws of the physical world, in order to derive the list you would need to do a complete simulation of the universe making this task intractable.

This also makes me have to back track on my criticism of Montague's assertion that psychology can affect how we do physics. While I still believe that we have the capability to compute anything the universe can throw at us, our interpretation of what we see and do can depend on our priors.

Realistic versus abstract neural modeling

2008-08-16T11:20:00.013-04:00

There is a very interesting discourse running on the comp-neuro email list. I've only caught the past week but it seems to be a debate between the benefits of "abstract" versus biological "realistic" models. (Let me caveat that everything here is my interpretation of the two points of view). Jim Bower, who is a strong proponent of realistic modeling, argues that abstract models (an example is a network of point neurons) add biases that lead us astray. He thinks that only through realistic modeling can we set down all the necessary constraints to discover how the system works. In a side remark, he also said that he thought the most important problem to understand is what information a given neuron transmits to another and the rest is just clean up. Bower believes that biology is pre-Copernican and that abstract modeling is akin to Ptolemy adding epicycles to explain planetary motion and realistic modeling is closer to the spirit of Kepler and Newton.

I don't want to debate the history and philosophy of science here but I do want to make some remarks about these two approaches. There are actual several dichotomies at work. One of the things it seems that Bower believes is that a simulation of the brain is not the same as the brain. This is in line with John Searle's argument that you have to include all the details to get it right. In this point of view, there is no description of the brain that is smaller than the brain. I'll call this viewpoint Kolmogorov Complexity Complete (a term I just made up right now). On the other hand, Bower seems to be a strict reductionist in that he does believe that understanding how the parts work will entirely explain the whole, a view that Stuart Kauffman argued vehemently against in his SIAM lecture and new book Reinventing the Sacred. Finally, in an exchange between Bower and Randy O'Reilly, who is a computational cognitive scientist and connectionist, Bower rails against David Marr and the top down approach to understanding the brain. Marr gave an abstract theory of how the cerebellum worked in the late sixties and Bower feels that this has been leading the field astray for forty years.

I find this debate interesting and amusing on several fronts. When I was at Pitt, I remember that Bard Ermentrout used to complain about connectionism because he thought it was too abstract and top down whereas using Hodgkin-Huxley-like models for spiking neurons with biologically faithful synaptic dynamics was the bottom up approach. At the same time, I think Bard (and I use Bard to represent the set of mathematical neuroscientists that mostly focus on the dynamics of interacting spiking neurons; a group to which I belong) was skeptical that the fine detailed realistic modeling of single neurons that Bower was attempting would enlighten us on matters of how the brain worked at the multi-neuron scale. One man's bottom up is another man's top down!

I am now much more agnostic about modeling approaches. My current view is that there are effective theories at all scales and that depending on the question being asked there is a level of detail and class of models that are more useful to addressing that question. In my current research program, I'm trying to make the effective theory approach more systematic. So if you are interested in how a single synaptic event will influence the firing of a Purkinje cell then you would want to construct a multi-compartmental model of that cell that respected the spatial structure. On the other hand if you are interested in understanding how a million neurons can synchronize, then perhaps you would want to use point neurons.

One of the things that I do believe is that complexity at one scale may make things simpler at higher scales. I'll give two examples. Suppose a neuron wanted to do coincidence detection of its inputs, e.g. it would collect inputs and fire if the inputs arrived at the same time. Now for a spatially extended neuron, inputs arriving at different locations on the dendritic tree could take vastly different amounts of time to arrive at the soma where spiking is initiated. Hence simultaneity at the soma is not simultaneity of arrival. It thus seemed that coincidence detection was a hard problem for a neuron to do. Then it was discovered that dendrites have active ion channels so that signals are not just passively propagated, which is slow, but actively propagated quickly. In addition, the farther away you are the faster you go so that no matter where a synaptic event occurs, it takes about the same amount of time to reach the soma. The dendritic complexity turns a spatially extended neuron into a point neuron! Thus, if you just focused on understanding signal propagation in the dendrites, your model would be complicated but if you only cared about coincidence detection, your model could be simple. Another example is in how inputs affect neural firing. For a given amount of injected current a neuron will fire at a given frequency giving what is known as an F-I curve. Usually in slice preparations, the F-I curve of a neuron will be some nonlinear function. However, in these situations not all neuromodulators are present so some of the slower adaptive currents are not active. When everything is restored, it was found (both theoretically by Bard and experimentally) that the F-I curve actually becomes more linear. Again, complexity at one level makes it more simple at the next level.

Ultimately, this "Bower" versus "Bard" debate can never be settled because the priors (to use a Bayesian term) of the two are so different. Bower believes that the brain is Kolmogorov complexity complete (KCC) and Bard doesn't. In fact, I think that Bard believes that higher level behavior of networks of many neurons may be simpler to understand than sets of just a few neurons. That is why Bower is first trying to figure out how a single neuron works whereas Bard is more interested in explaining a high level cognitive phenomenon like hallucinations in terms of pattern formation in an integro-differential system of equations (i.e. Wilson-Cowan equations). I think most neuroscientists believe that there is a description of the brain (or some aspect of the brain) that is smaller than the brain itself. On the other hand, there seems to be a growing movement towards more realistic characterization and modeling of the brain at the genetic and neural circuit levels (in addition to the neuron level) as evidenced by the work at Janelia Farm and EPFL Lausanne, of which I'll blog about in the future.

New Paper on insulin's effect on free fatty acids

2008-08-15T12:08:00.005-04:00

A paper I've been trying to get published for two years will finally appear in the American Journal of Physiology - Regulatory, Integrative, and Comparative Physiology. The goal of this paper was to develop a quantitative model for how insulin suppresses free fatty acid levels in the blood. A little background for those unfamiliar with human metabolism. All of the body's cells burn fuel and for most cells this can be fat, carbohydrate or protein. The brain, however, can only burn glucose, which is a carbohydrate, and ketone bodies, which are made when the body is short of glucose. Why the brain can't burn fat is still a mystery. It is not because fat cannot cross the blood brain barrier as is sometimes claimed. Thus, the body has a reason to regulate glucose levels in the blood. It does this through hormones, the most well known of which is insulin.

Muscle cells cannot uptake glucose unless insulin is present. So when you eat a meal with carbohydrates, insulin is released by the pancreas and your body utilizes the glucose that is present. In between meals, muscle cells mostly burn fat in the form of free fatty acids that are released by fat cells (adipocytes) through a process called lipolysis. The glucose that is circulating is thus saved for the brain. When insulin is released, it also suppresses lipolysis. Basically, insulin flips a switch that causes muscle and other body cells to switch from burning fat to glucose and in addition switches off the fuel supply for fat.

If your pancreas cannot produce enough insulin then your glucose levels will be elevated and this is diabetes mellitus. Fifty years ago, diabetes was usually the result of an auto-immune disorder that destroyed pancreatic beta cells that produce insulin. This is known as Type I diabetes. However, recently the most prevalent form of diabetes, called Type II, arises from a drawn out process attributed to overweight or obesity. In Type II diabetes, people first go through a phase called insulin resistance where more insulin is required for glucose to be taken up by muscle cells. The theory is that after prolonged insulin resistance, the pancrease eventually wears out and this leads to diabetes. Insulin resistance is usually reversible by losing weight.

Thus, a means to measure how insulin resistant or sensitive you are is important. This is usually done through a glucose challenge test, where glucose is either ingested or injected and then the response of the body is measured. I don't want to get into all the methods used to assess insulin sensitivity but one of the methods uses what is known as the minimal model of glucose disposal, which was developed in the late seventies by Richard Bergman, Claudio Cobelli and colleagues. This is a system of 2 ordinary differential equations that model insulin's affect on blood glucose levels. The model is fit to the data and an insulin sensitivity index is one of the parameters. Dave Polidori, who is a research scientist at Johnson and Johnson, claims that this is the most used mathematical model in all of biology. I don't know if that is true but it does have great clinical importance.

The flip side to glucose is the control of free fatty acids (FFAs) in the blood and this aspect has not been as well quantified. Several groups have been trying to develop an analogous minimal model for insulin's action on FFA levels. However, none of these models have been validated or even tested against each other on a single data set. In this paper, we used a data set of 102 subjects and tested 23 different models that included previously proposed models and several new ones. The models have the form of an augmented minimal model with compartments for insulin, glucose and FFA. Using Bayesian model comparison methods and a Markov chain Monte Carlo algorithm, we calculated the Bayes factors for all the models. We found that a class of models distinguished themselves from the rest with one model performing the best. I've been using Bayesian methods quite a bit lately and I'll blog about it sometime in the future. If you're interested in the details of the model, I encourage you to read the paper.

SIAM Lifesciences '08

2008-08-09T11:32:00.005-04:00

I've just returned from the Society of Industrial and Applied Mathematics (SIAM) Lifesciences meeting in Montreal. I haven't traveled to a meeting since my baby was born so it was nice to catch up with old friends and the field. I thought that all of the plenary talks were excellent and I commend the organizing committee for doing a great job. Particularly interesting was a public lecture given by Stuart Kauffman on his new book "Reinventing the Sacred". That talk was full of many ideas that I've been directly interested in and I'll blog about them soon.

One of the things I took away from this meeting was that the field seems more diverse than when I organized it in 2004. In particular, I thought that the first two renditions of this meeting (2002, 2004) were more like an offshoot of the very well attended SIAM dynamical systems meeting held in Snowbird, Utah on odd numbered years. Now, I think that the participation base is more diverse and in particular there is much more overlap with the systems biology community. One of the unique things about this meeting is that people interested in systems neuroscience and systems biology both attend. These two communities generally don't mix even though some of the problems and methods have similarities and would benefit from interacting. Erik De Shutter wrote a nice article recently in PLoS Computational Biology exploring this topic. I thus particularly enjoyed the fact that there were sessions that included talks on both neural and genetic/biochemical networks. In addition, there were sessions on cardiac dynamics, metabolism, tissue growth, imaging, fluid dynamics, epidemiology and many other areas. Hence, I think that this meeting does play a useful and unique role bringing together mathematicians and modelers from all fields.

I gave a talk on my work on the dynamics of human body weight change. In addition to summarizing my PLoS Computational Biology paper, I also showed that because humans have such a long time constant to achieve energy balance when on a fixed rate of energy intake (i.e. a year or more), we can tolerate a wide amount of fluctuations in our energy intake rate and still have a small variance in our body weight. This answers the "paradox" that nutritionists seem to believe, namely that if a change of as small as 20 kcals/day (a cookie is ~150 kcal) can lead to a weight change of a kilogram then how do we maintain our body weights if we consume over a million kcals a year. Part of their confusion stems from conflating average with standard deviation. Given that we only eat finite amounts of food per day then no matter what you eat in a year you will have some average body weight. The question is why the standard deviation is so small; we generally don't fluctuate by more than a few kilos per year. The answer is simply that with a long time constant, we average over fluctuations. My back of the envelope calculation shows that the coefficient of variation (standard deviation divided by mean) of body weight suppresses by a factor of 15 or more the coefficient of variation in the food intake. This also points to correlations in food intake rate leading to weight gain, as was addressed in my paper with Vipul Periwal (Periwal and Chow, AJP-EM, 291:929 (2006)).

Karen Ong, from my group, also went and presented our work on steroid mediated gene expression. She won the poster competition for undergraduate students. I'll blog about this work in the future. While I was sitting in the sessions on computational neuroscience and gene regulation, I regretted not having more people in my lab attend and present our current ideas on these and other topics.

Penrose redux

2008-08-02T16:50:00.004-04:00

In 2006, I posted my thoughts on Roger Penrose's argument that human thought must be noncomputable. Penrose's argument follows from the fact that Godel's incompleteness theorem states that there exist true statements in a consistent formal system (e.g. arithmetic with integers) that cannot be proved within that system. The proof basically boils down to showing statements like "this statement cannot be proved" are true but cannot be proved because if they could be proved then there would be an inconsistency with the system. Turing later showed that this was equivalent to saying that there are problems, known as undecidable or uncomputable problems, that a computer could not solve. From these theorems, Penrose draws the conclusion that since we can recognize unprovable statements are true then we must not be a computer.

My original argument refuting Penrose's claim was that we didn't really know what formal system we were using or whether or not it remained fixed so we couldn't know if we were recognizing true statements that we can't prove. However, I now have a simpler argument, which is simply that no human has ever solved an uncomputable problem and hence has not shown they are more than a computer. The fact that they know about uncomputability is not an example. A machine could also have the same knowledge since Godel's and Turing's proofs (as are all proofs) are computable. Another way of staying this is that any proof or thing that can be written down in a finite number of symbols could also be done by a computer.

An example is the fact that you can infer the existence of real numbers using only integers. Thus, even though real numbers are uncountable and thus uncomputable, we can prove lots of properties about then just using integers. The Dedekind cut can be used to prove the completeness of real numbers without resorting to the axiom of choice. Humans and computers can reason about real numbers and physical theories based on real numbers without actually ever having to deal directly with real numbers. To paraphrase, reasoning about uncomputable problems is not the same as solving uncomputable problems. So until a human being can reliably tell me whether or not any Diophantine equation (polynomial equation with integer coefficients) has a solution in integers (i.e. Hilbert's tenth problem) or always know if any program will ever halt (i.e. the halting problem), I'll continue to believe that a computer can do whatever we can do.

Limits to thought and physics

2008-07-27T11:04:00.007-04:00

In a recent post, I commented on Reed Montague's proposal that it may be necessary to account for the limits of psychology when developing theories of physics. I disagreed with his thesis because if we accept that physics is computable and the brain is described by physics then any property of physics should be discernible by the brain. However, I should have been more careful in my statements. Given the theorems of Godel and Turing, we must also accept that there may be certain problems or questions that are not decidable or computable. The most famous example is that there is no algorithm to decide if an arbitrary computation will ever stop. In computer science this is known as the halting problem. (Godel's incompleteness theorems are direct consequences of the halting problem, although historically they came first). The implication is quite broad for it also implies that there is no sure fire way of knowing if a given computation will do a particular thing (i.e. print out a particular symbol). This is also why there is no certain way of ever knowing if a person is insane or if a criminal will commit another crime, as I claimed in my post on crime and neuroscience.

Hence, it may be possible that some theories in physics, like the ultimate super-duper theory of everything, may in fact be undecidable. However, this is not just a problem of psychology but also a problem of physics. Some, like British mathematical physicist Roger Penrose, would argue that the brain and physics are actually not computable. Penrose's arguments are outlined in two books - The Emperor's New Mind and Shadows of the Mind. It could be that the brain is not computable but I (and many others for various reasons) don't buy Penrose's argument for why it is not. I posted on this topic previously although I've refined my ideas considerably since that post.

However, even if the brain and physics were not computable there would still be a problem because we can only record and communicate ideas with a finite number of symbols and this is limited by the theorems of Turing and Godel. It could be possible that a single person could solve a problem or understand something that is undecidable but she would not be able to tell anyone else what it is or write about it. The best she could do is to teach someone else how to get into such a mental state to "see" it for themselves. One could argue that this is what religion and spiritual traditions are for. Buddhism in a crude sense is a recipe for attaining nirvana (although one of the precepts is that trying to attain nirvana is a surefire way of not attaining it!). So, it could be possible that throughout history there have been people that have attained a level of, dare I say, enlightenment but there is no way for them to tell us what that means exactly.

Neural Code

2008-07-18T11:15:00.013-04:00

An open question in neuroscience is: what is the neural code? By that it is meant, how is information represented and processed in the brain. I would say that the majority of neuroscientists, especially experimentalists, don't worry too much about this problem and implicitly assume what is called a rate code, which I will describe below. There is then a small but active group of experimentalists and theorists who are keenly interested in this question and there is a yearly conference, usually at a ski resort, devoted to the topic. I would venture to say that within this group - who use tools from statistics, Bayesian analysis, machine learning and information theory to analyze data obtained from in vivo multi-electrode recordings of neural activity in awake or sedated animals given various stimuli - there is a larger amount of skepticism towards a basic rate code than the general neural community.

For the beneficiary of the uninitiated, I will first give a very brief and elementary review of neural signaling. The brain consists of 10^11 or so neurons, which are intricately connected to one another. Each neuron has a body, called the soma, an output cable, called the axon, and input cables, called the dendrites. Axons "connect" to dendrites through synapses. Neurons signal each other with a hybrid electro-chemical scheme. The electrical part involves voltage pulses called action potentials or spikes. The spikes propagate down axons through the movement of ions across the cell membrane. When the spikes reach a synapse, they trigger a release of neurotransmitters, which diffuse across the synaptic cleft, bind to receptors on the receiving end of the synapses and induce either a depolarizing voltage pulse (excitatory signal) or a hyperpolarizing voltage pulse (inhibitory signal). In that way, spikes from a given neuron can either increase or decrease the probability of spikes in a connected neuron.

The neuroscience community is basically all in agreement that neural information is carried by the spikes. So the question of the neural code becomes: how is information coded into spikes? For example, if you look at an apple, something in the spiking pattern of the neurons in the brain is representing the apple. Does this change involve just a single neuron? This is called the grandmother cell code, from the joke that there is a single neuron in the brain that represents your grandmother. Or does it involve a population of neurons, known not surprisingly, as a population code. How did the spiking pattern change? Neurons have some background spiking rate, so do they simply spike faster when they are coding for something, or does the precise spiking pattern matter. If it is just a matter of spiking faster then this is called a rate code, since it is just the spiking rate of the neuron that contains information. If the pattern of the spikes matter then it is called a timing code.

The majority of neuroscientists, especially experimentalists, implicitly assume that the brain uses a population rate code. The main reason they believe this is because in most systems neuroscience experiments, an animal will be given a stimulus, and then neurons in some brain region are recorded to see if any respond to that particular stimulus. To measure the response they often count the number of spikes in some time window, say 500 ms, and see if it exceeds some background level. What seems to be true from almost all of these experiments is that no matter how complicated a stimulus you want to try, a group of neurons can usually be found that respond to that stimulus. So, the code must involve some population of neurons and the spiking rate must increase. What is not known is which and how many neurons are involved and whether or not the timing of the spikes matter.

My sense is that the neural code is a population rate code but the population and time window change and adapt depending on context. Thus understanding the neural code is no simpler than understanding how the brain computes. In molecular biology, deciphering the genetic code ultimately led to understanding the mechanisms behind gene transcription but I think in neuroscience it may be the other way around.

Crime and neuroscience

2008-07-08T17:04:00.003-04:00

The worry in the criminal justice system is that people will start trying to use neuroscience in their defense. For example, they will start to claim that their brain was faulty and it committed the crime. I think some have already tried this. I think the only way out of this predicament is to completely reframe how we administer justice. Currently, the intent of the perpetrator to commit the crime (negligence included as a crime) is required to establish criminal activity. This is why insanity is a viable defense. I think this notion will gradually become obsolete as the general public comes to accept the mechanistic explanation of mind. When the discontinuity between man and machine is finally accepted by most people, the question of intent is going to be problematic. That is why we need to start rethinking this whole enterprise now.

My solution is that we should no longer worry about intent or even treat justice as a form of punishment. What we should do in a criminal trial is to determine if the defendant a) actually participated in the crime and b) if they are dangerous to society. By this reasoning, putting a person in jail is only necessary if they are dangerous to society. For example, violent criminals would be locked up and they would only be released if it could be demonstrated that they were no longer dangerous. This schema would also eliminate the concept of punishment. The duration of a jail sentence should not be about punishment but only about benefit to society. I don't believe that people should be absolved of their crimes. For nondangerous criminals, some form of retribution in terms of garnished wages or public service could be imposed. Also, if some form of punishment can be shown to be a deterrent then that would be allowed as well. I'm sure there are many kinks to be worked out but what I am proposing is that a fully functional legal system could be established without requiring a moral system to support it.

This type of legal system will be necessary when machines become sentient. Due to the theorems of Godel and Turing, proving that a machine will not be defective in some way will be impossible. Thus, some machines may commit crimes. Given that they are sentient also means that we cannot simply go around and disconnect machines at will. Each machine deserves a fair hearing to establish guilt and sentencing. Given that there will not be any algorithmic way to establish with certainty whether or not the machine will repeat the crime, justice for machines will have to be administered in the same imperfect way it is administered for humans.

Oil Consumption

2008-07-04T18:10:00.003-04:00

I thought it would be interesting to convert the amount of oil consumed by the world each year into cubic kilometres to give a sense of scale of how much oil we use and how much could possibly be left. The world consumes about 80 million barrels of oil a day. Each barrel is 159 litres and a cubic kilometre corresponds to 10^12 litres, so the amount of oil the world consumes in a year is equivalent to 4.6 cubic kilometres. This would correspond to a cubic pit with sides that are about 1.45 km long. The US consumes a little over a quarter of the world's oil, which is about 1.2 cubic kilometres. The Arctic National Wildlife Refuge is estimated to have about 10 billion barrels of recoverable oil, about a half of year's supply.

Proven world reserves amount to about 1.3 trillion barrels of oil or about 200 cubic kilometres. If we continue at current rates of consumption, then we have about 40 years of oil left. If the rest of the world decided to use oil like American's then the world's yearly consumption could increase by a factor of 5, which would bring us down to 8 years worth of reserve. However, given that the surface of the earth is about 500 million square kilometres, it seems plausible that there is a lot more oil out there that hasn't been found, especially under the deep ocean. The main constraint is cost and greenhouse gas emissions. We may not run out of oil anytime soon but we may have run out of cheap oil already.

Physics and psychology

2008-06-17T18:23:00.005-04:00

Montague in the epiologue of his book, which I blogged about recently, argued that a marriage of psychology and physics is in order. His thesis is that our intuitions about the world are based on flawed systems that were only designed to survive and reproduce. There are several responses to his argument. The first is that while we do rely on intuition to do math and physics, the intuition is based on learned concepts more than our "primitive" intuitions. For example, logical inference itself is completely nonintuitive. Computer scientist Scott Aaronson gives a nice example. Consider cards with a letter on one side and a number on the other. You are given 2, K, 5, J and the statement every J has a 5 on the back. Which cards do you need to turn over to see if the statement is true? (Hint: If A implies B then the only conclusion you can draw is not B implies not A). Most college freshmen get this wrong. Quantum mechanics and thermodynamics are notoriously counter intuitive and difficult to understand. We were led to these theories only after doing careful experiments that could marginalize away our prior beliefs.

However, that is not to say that perhaps we're at a stumbling block over questions like what is dark matter or why do we remember the past and not the future because of some psychological impediment. A resolution to this issue could reside again on whether or not physics is computable. Montague doesn't think so but his examples do not constitute a proof. Now if physics is computable and the brain is governed by the natural laws of physics, then the brain is also computable. In fact, this is the simplest argument to refute all those that doubt that machines can ever think. If they believe that the brain is in the natural world and physics can be simulated then we can always simulate the brain and hence the brain is computable. Now if the brain is computable, then any phenomenon in physics can be understood by the brain or at least computed by the brain. In other words, if physics is computable then given any universal Turing machine, we can compute any physical phenomenon (given enough time and resources).

There is one catch to my argument and that is the fact that if we believe the brain is computable then we must also accept that it is finite and thus less powerful than a Turing machine. In that case, there could be computations in physics that we can't understand with our finite brains. However, we could augment our brains with extra memory (singularity anyone) to complete a computation if we ever hit our limit. The real question is again tractability. It could be possible that some questions about physics are intractable from a purely computational point of view. The only way to "understand" these things is to use some sort of meta-reasoning or some probabilistic algorithm. It may then be true that the design of our brains through evolution may impede our ability to understand concepts that are outside of the range it was designed for.

Personally, I feel that the brain is basically a universal Turing machine with a finite tape so that it can do all computations up to some scale. We can thus only understand things with a finite amount of complexity. The way we approach difficult problems is to invent a new language to represent complex objects and then manipulate the new primitives. Thus our day to day thinking uses about the same amount of processing but accumulated over time we can understand arbitrarily difficult concepts.

Why so slow?

2008-06-10T14:17:00.004-04:00

John Tierney of the New York times shows a figure from Ray Kurzweil of a log-log plot of the time between changes in history, such as the appearance of life multicellular organisms to new technologies like televisions and computers. His graph shows power law scaling with an exponent of negative one, which I obtained by eyeballing the curve. In other words, if dT is the time between the appearance of the next great change then it scales as 1/T where T is the time. I haven't read Kurzweil's book so maybe I'm misinterpreting the graph. The fact that there is scaling over such a long time is interesting but I want to discuss a different point. Let's take the latter part of the curve regarding technological innovation. Kurzweil's argument is that the pace of change is accelerating so we'll soon be enraptured in the Singularity (see previous post). However, the rate of appearance of new ideas seems to be only increasing linearly with T. So the number of new ideas are accumulating as T^2, which is far from exponential. Additionally, the population is increasing exponentially (at least in the last few hundred years). Hence the number of ideas per person is obeying t^2 Exp(-t). I'm not sure where we are on the curve but after an initial increase, the number of ideas per person actually decreases exponentially. I was proposing in the last post that the number of good ideas was scaling with the population but according to Kurzweil I was being super optimistic. Did I make a mistake somewhere?

The singularity

2008-06-06T14:05:00.005-04:00

Steve Hsu points to an IEEE Spectrum special report on The Singularity, which is variously described as the point in time when machine intelligence surpasses human intelligence so that machines can then go on to improve themselves without our help. The super optimists like Ray Kurzweil think this will happen in 15 years and when it happens machine intelligence will increase at doubling times that will be hours, minutes or shorter. The machines will then solve all of the world's problems instantly. He also thinks we can upload our minds into the machine world and be immortal.

Personally, I think that there is no question that humans are computable (in the Turing sense) so there is no reason that a machine that can think like we do will someday exist (we being an existence proof). I have no idea when this will happen. Having studied computational neuroscience for about 15 years now, I can say that we aren't that much closer to understanding how the brain works then we were back then. I have a personal pet theory, which I may expound on sometime in the future, but it's probably no better than anyone else's. I'm fairly certain machine intelligence won't happen in 15 years and may not in my lifetime.

The argument that the singularity enthusiasts use is a hyper-generalization of Moore's law of exponential growth in computational power. They apply the law to every thing and then extrapolate. For example, anything that doubles each year (which is a number Kurzweil sometimes uses) will improve by a factor of 2^15=32,000 in 15 years. To Kurzweil, we are just a factor of 2^15 away from singularity.

There are two quick responses to such a suggestion, the first is where did 2^15 come from and the second is nonlinear saturation. I'll deal with the second issue first. In almost every system I've ever dealt with there is usually some form of nonlinear saturation. For example, some bacteria can double every 20 minutes. If it weren't for the fact that they run out of food eventually and stop growing (i.e. nonlinear saturation) a single colony would cover the earth in less than a week. Right now components on integrated circuits are less than 100 nm in size. Thus in less than 10 doublings they will be smaller than atoms. Hence, Moore's law as we know it can only go on for another 20 years at most. To continue the pace beyond that, we will require a technological paradigm shift and there is no successor on the horizon. The singularists believe in the Lone Ranger hypothesis so something will come to the rescue. However, even if computers do get faster and faster, software is not improving at anything near the same pace. Arguably, Word is worst now then it was 10 years ago. My computer still takes a minute to turn on. The problem is that good ideas don't seem to be increasing exponentially. At best they only seem to scale linearly with the population.

That leads us to the first point. How far away are we from building a thinking machine? The answer is that we haven't a clue. We may just need a single idea or it might be several. Over the past 50 years or so we've really only had a handful of truly revolutionary ideas about neural functioning. We understand a lot about the machinery that the brain runs on but very little about how it all works together to create human intelligence. We are making progress but it's slow. However, nonlinearity could help us here because we may be near a bifurcation to take us to a new level of understanding. However, this is not predictable by exponential growth.

The other thing about the singularity is that the enthusiasts seem to think that intelligence is unlimited, so that thinking machines can instantly solve all of our problems. Well if physics is computable (see previous post), then no amount of intelligence can solve the Halting problem or Hilbert's tenth. If we believe that P is not equal to NP, then no hyper intelligence can solve intractable problems, unless the claim extends to the ability to compute infinitely fast. I would venture that no amount of intelligence will ever settle the argument of who was the greatest athlete of all time. Many of our problems are due to differences in prior beliefs and that can't be solved by more intelligence. We have enough wealth to feed everyone on the planet yet people still starve. Unless the singularity implies that machines control all aspects of our lives, there will be human problems that will not be solved by extra intelligence.

The example enthusiasts sometimes give of a previous singularity is the emergence of humans. However, from the point of view of the Dodo bird, humpback whale, buffalo, polar bear, American elm tree, and basically most other species, life got worse following the rise of the humans. We're basically causing the greatest extinction event since the fall of the dinosaurs. So who's to say that when the machine singularity strikes, we won't be left behind similarly. Machines may decide to transform the earth to suit their needs and basically ignore ours.

Are humans computable?

2008-05-30T22:12:00.010-04:00

I picked up Read Monatgue's book on decision making - Why Choose This Book?: How We Make Decisions, this evening at Barne's and Noble and read the epilogue with the provacative title: Are Humans Computable? In this chapter, Montague puts forward the argument that protein-protein interactions are not computable and hence neither is the brain or physics for that matter. He argues that proteins and worldly stuff have physical properties that are not computable but can be strung together algorithmically to achieve an end, such as the brain. As an example, he suggests that writing down the equations that govern a nuclear reactor do not suddenly create energy because the equations lack the "physical properties" necessary for a chain reaction. His final point is that our intuitions are based on a brain that is only designed to survive and reproduce and thus physics is ineluctably intertwined with psychology. He proposes that the next frontier is to incorporate the limitations of human perception into new physical theories.

I have been reading and thinking about the theory of computation lately. I have a host of incomplete and inchoate ideas on the topic that are not ready for prime time but after reading Montague's chapter I thought it would be useful to put some things down before I forget them. Montague has touched on some interesting and deep questions but I believe his particular point of view is flawed. He incorrectly conflates intractability with uncomputability and an algorithm with a computation. This post will only touch briefly on the very many issues related to this topic.

On protein-protein interactions, Montague writes that the totality of possible interactions is unimaginably large and thus could never fit on a realizable computer. He equates this with being uncomputable. Protein-protein interactions may not be computable but not because it can't fit on a computer. A problem is deemed uncomputable or undecidable if a computer (i.e. Turing machine) cannot solve (decide) it. This means that the problem cannot be solved by any algorithm. Perhaps Montague knows this but his editor forced him to tone down the technical details. The most famous undecidable problem is the halting problem, which says that there is no algorithm that can tell if a computation will ever stop. Hilbert's tenth problem on the solvability of diophantine equations with integers is also undecidable. It is not known if protein-protein interactions and by implication all of physics is decidable but almost everyone in physics and applied math believes it is so, whether they know this or not. One notable exception is Roger Penrose who believe that quantum gravity is uncomputable, but that is another story for another post.

The question is not as trivial as it sounds. Kurt Godel, Alonzo Church, Alan Turing, and many other twentieth century mathematicians established the criteria for computability and I hope to get to their ideas in more depth in future posts. However, for now in a nutshell, the question of computability comes down to the cardinality of the set you are trying to compute. If the set of possible outcomes of a problem is countable, then the problem is computable. If it is not countable, then it is not. Now to physics: If we believed that space-time were a true continuum (i.e. described by real numbers) then the set of all possible configurations of two proteins would be uncountable and Montague would be correct that the dynamics are uncomputable.

However, there are two very plausible responses to this problem. The first is that although real numbers are uncomputable, they can be approximated arbitrarily closely by countable rational numbers. This is the foundation of numerical analysis, which shows that any continuous dynamics can be approximated arbitrarily well with a discretized system. That's how we do numerical simulations for weather prediction, airflow over a an airplane wing, and even protein-protein interactions. The reason that numerical simulations work is that there is an underlying smoothness to the dynamics so we can approximate it by a finite set of points. In essence, we can predict what will happen next for short enough times and distances. Now, this need not be true. It could be that space-time is chaotic at small scales so that no discretization can approximate it. This is proposed in theories of quantum gravity. However, even if that were the case there is probably some averaging over larger scales that effectively smooths the underlying turbulence, (think the uncertainty principle), to make physics effectively computable and that is what we deal with on a day to day basis.

The second way to argue for the computability of the universe is that the entropy of the observable universe is finite. Entropy is basically the logarithm of the number of microstates. Thus if entropy is finite then the number of possible configurations of the universe is countable (actually finite) so again physics is computable. Why is the entropy of the universe finite? Again, think quantum mechanics. If space-time is quantized then there will be a smallest scale, namely the Planck length which is about 10^-35 metres.

However, Montague does have a point that a simulation of the interactions of two or more proteins could be intractable in that it would take an immense amount of memory or time to do a computation. The field of algorithmic complexity examines questions of intractability of which the most famous problem is whether or not P = NP. I don't have time to go into that one but I hope to post more on that later as well. So, while we may never be able to simulate the brain, that doesn't mean the brain is not computable, it's just intractable. However, intractability doesn't imply that we couldn't build an artificial brain.

I think I'll save my comments on Montague's other points in a future post.

Transients in dynamical systems

2008-04-09T15:20:00.003-04:00

I've just posted a paper to arXiv.org entitled "Competition between transients in the rate of approach to a fixed point", by Judy Day, Jonathan Rubin and Carson C. Chow. The paper examines how long it takes to approach a fixed point. The problem was motivated by a biological phenomenon known as tolerance, where the body's inflammatory response to a noxious stimulus is attenuated by a pre-exposure to that substance. We translated this problem into the question of given two orbits, under what conditions would one orbit "pass" another. We show that using general properties of the continuity of orbits and the concept of inhibition, a set of conditions for when tolerance can and cannot exist can be established. Transient dynamics have not been well studied and this paper represents an approach into the issue.

Multiple scale analysis

2008-04-01T16:34:00.002-04:00

I recently wrote a scholarpedia entry on multiple scale analysis. It is a useful tool of applied mathematics.

Just Published

2008-04-01T14:07:00.003-04:00

Our paper "The Dynamics of Human Body Weight Change" just appeared in PLoS Computational Biology.

The dynamics of human body weight change

2008-02-22T13:04:00.003-05:00

I've just posted a new paper to the quantitative biology archive: Carson C. Chow and Kevin D. Hall, The dynamics of human body weight change (arXiv:0802.3234). Understanding the dynamics of weight change has important consequences for conditions such as obesity, cancer, AIDS, anorexia and bulimia nervosa. While we know that changes of body weight result from imbalances between the energy derived from food and the energy expended to maintain life and perform physical work, quantifying this relationship has proved difficult. Part of the difficulty stems from the fact that the body is comprised of multiple components and we must quantify how weight change is reflected in terms of alterations of body composition (i.e. fat versus lean mass). In this paper, we show that a model of the flux balances of macronutrients, namely fat, protein and carbohydrates, can provide a general description of the way the body weight will change over time. Under general conditions, the model can be reduced to a two dimensional system of fat and lean body masses, which then can be analyzed in the phase plane. For a fixed food intake rate and physical activity level, the body weight and body composition will approach a steady state. However, the steady state can correspond to a unique body weight (fixed point) or a continuum of body weights (invariant manifold) depending on how fat oxidation depends on the body weight and composition changes. Interestingly, the existing experimental data on human body weight dynamics cannot presently distinguish between these two possibilities. However, this distinction is important for the efficacy of clinical interventions that alter body composition and mass.

Finite size neural networks

2007-12-05T14:07:00.000-05:00

A paper by Hedi Soula and myself on the dynamics of a finite-size neural network (Soula and Chow, Neural Comp, 19:3262 (2007)) is out this month. You can also download it from my homepage. As you may infer, I've been preoccupied with finite-size effects lately. In this paper we consider a very simple Markov model of neuronal firing. We presume that the number of neurons that are active in a given time epoch depends only on the number that were active in the previous epoch. This would be valid in describing a network with recurrent excitation for example. Given this model we can calculate the equilibrium probability density function for a network of size N directly and from that all statistical quantities of interest. We also show that the model can describe the dynamics of a network of more biophysically plausible neuron models. The nice thing about our simple model is that we can then compare the exact results to mean field theory, which is the classical way of studying the dynamics of large numbers of coupled neurons. We show that the mean activity is generally well described by mean field theory, except near criticality as expected, but the variance is not. We also show that the network activity can possess a very long correlation time although the firing statistics of a single neuron does not.

Kinetic Theory of Coupled Oscillators

2007-11-26T14:50:00.002-05:00

I have recently published two papers applying ideas from the kinetic theory of plasmas and nonequilibrium statistical mechanics to coupled oscillators. The first is Hildebrand, Buice and Chow, PRL 98:054101 and the second is Buice and Chow, PRE 76:031118. The main concern of both papers is understanding the dynamics of large but not infinite networks of oscillators. Generally, coupled oscillators are studied either in the small network limit where explicit calculations can be performed or in the infinite size "mean field" limit where fluctuations can be ignored. However, many networks are in between, i.e. large enough to be complicated but not so large that the effects of individual oscillators are not felt. This is the regime we were interested in and where the ideas of kinetic theory are useful.

In a nutshell, kinetic theory strives to explain macroscopic phenomenon of a many body system in terms of (the moments of the distribution function governing) the microscopic dynamics of the constituent particles (oscillators). In the coupled oscillator case, we actually have a macroscopic theory and what we want to understand is how the microscopic dynamics gave rise to that theory. For example, in the Kuramoto model of coupled oscillators, there is a phase transition from asynchrony to synchrony if the coupling strength is sufficiently strong. In the infinite oscillator limit where fluctuations can be ignored, an order parameter measuring the synchrony in the network can be shown to bifurcate from zero at a critical coupling strength. However, for a finite number of oscillators, the order parameter fluctuates and there is no longer a sharp transition from asynchrony to synchrony but rather a crossover. We show in the first paper that a moment expansion analogous to the BBGKY hierarchy can be derived for the coupled oscillator system and using a Leonard-Balescu-like approximation the variance of the order parameter can be computed explicitly.

The second paper shows that the moment hierarchy can be equivalently expressed in terms of a generating functional of the oscillator density (i.e. a density of the density if you like). Once expressed in this form, diagrammatic methods of field theory can be used to do perturbative expansions. In particular, we perform a one-loop expansion to show marginal modes in the mean field theory are stabilized by finite-size fluctuations. This problem of marginal modes had been a puzzle in the field for a number of years.

House of Cards

2007-03-22T17:55:00.000-04:00

The US Comptroller General, David Walker, is currently on a "Fiscal Wake-up Tour" to alert the population about the impending US fiscal crisis. I encourage everyone, including non-Americans, to go to one of these events or at least to the Government Accountability Office website at www.gao.gov. In essence, current US fiscal policy is unsustainable. Promised obligations such as Medicare, Medicaid and Social Security as well as servicing the interest of the debt will be 40% of GDP by 2040. Currently, revenue coming mostly form taxes is less than 20% GDP. Hence, either revenues must increase or spending must be cut. Whatever choice is made could lead to dire consequences.

Raising taxes is one solution but that could have adverse effects on the economy. I think the US could sustain some tax increases, especially on the rich, without bad effects but not enough to cover the deficit spending. Much of the current economy is geared towards providing services and goods to the well heeled. If disposable income starts to go down the first purchases to go will be luxury items like yachts, expensive restaurants, ski vacations, and so forth. People in these professions and industries will then lose their jobs putting more strain on social services. The very rich will be insulated from tax increases but the upper middle class, from which most of the tax revenue is extracted, would have to cut down on consumption. Additionally, with the real estate bubble of the past five years, many people are stuck with mortgages that they can barely sustain. Any increase in taxes could push them over the edge. So while taxes can be increased, it really can't be increased too much.

The other option is to cut spending. The retirement age can be raised to reduce the social security obligation. However, social security is actually in pretty good shape compared to medicare and medicaid. Something eventually will be done about these two programs. Hence, medical services and reimbursements to health care professionals will both likely be reduced. Basically, the US could end up being a nation where the poor and elderly will not receive first world medical care. This may be one other reason a complete overhaul of the health care system may be necessary.

Altruism and Tribalism

2006-12-28T17:15:00.000-05:00

There has always been a puzzle in evolutionary biology as to how altruism arose. On first flush, it would seem that sacrificing oneself for another would be detrimental to passing on genes that foster altruism. However, Darwin himself thought that altruism could arise if humans were organized into hostile tribes. From the Descent of Man he notes that the tribes that had more "courageous, sympathetic andfaithful members who were always ready to...aid and defend eachother... would spread and be victorious over other tribes.'' A recent paper in Science by Samuel Bowles presents a calculation that supports Darwin's hypothesis.

If this hypothesis is correct, then altruism required lethal hostility to flourish and survive. Our capacity for great acts of sacrifice and empathy may go hand in hand with our capacity for brutality and selfishness. It may be why a person can simultaneously be a racist and a humanist. It may also mean that the sectarian violence we are currently witnessing and have witnessed throughout history may be as part of being human as caring for an ailing neighbor or taking a bullet for a friend. Our propensity for kindness may go hand in hand with that of bigotry and violence. It may be that the more homogeneous we become, the less altruistic we may be. Perhaps there may be an important societal role for spectator sports. Cheering for the home team may give us that sense of tribalism and triumph that we need. Maybe, just maybe, hating that cross-town rival makes us kinder in the office and on the roads. What irony that would be.

The Hopfield Hypothesis

2006-12-06T16:05:00.000-05:00

In 2000, John Hopfield and Carlos Brody put out an interesting challenge to the neuroscience community. They came up with a neural network, constructed out of simple well known neural elements, that could do a simple speech recognition task. The network was robust to noise and the speed the sentences were spoken. They conducted some numerical experiments on the network and provided the "data" to anyone interested. People were encouraged to submit solutions for how the network worked and Jeff Hawkins of Palm Pilot fame kicked in a small prize for the best answer. The initial challenge with the mock data and the implementation details were published separately in PNAS. Our computational neuroscience journal club at Pitt worked on the problem for a few weeks. We came pretty close to getting the correct answer but missed one crucial element.

Hopfield wanted to present the model as a challenge to serve as an example that sometimes more data won't help you understand a problem. I've extrapolated this thought into the statement that perhaps we already know all the neurophysiology we need to understand the brain but just haven't put the pieces together in the right way yet. I call this the Hopfield Hypothesis. I think many neuroscientists believe that there are still many unknown physiological mechanisms that need to be discovered and so what we need are not more theories but more experiments and data. Even some theorists believe this notion. I personally know one very prominent computational neuroscientist who believes that there may be some mechanism that we have not yet discovered that is essential for understanding the brain.

Currently, I'm a proponent of the Hopfield Hypothesis. That is not to say I don't think there will be mechanisms, and important ones at that, yet to be discovered. I'm sure this is true but I do think that much of how the brain functions could be understood with what we already know, namely that the brain is composed of populations of excitatory and inhibitory neurons with connections that obey synaptic plasticity rules such as long-term potentiation and spike-time dependent plasticity with adaptation mechanisms such as synaptic facilitation, synaptic depression, and spike frequency adaptation that operate on multiple time scales. Thus far, using these mechanisms we can construct models of working memory, synchronous neural firing, perceptual rivalry, decision making, and so forth. However, we still don't have the big picture. My sense is that neural systems are highly scale dependent so as we begin to analyze and simulate larger and more complex networks, we will find new unexpected properties and get closer to figuring out the brain.

Wealth and Taxes

2006-11-16T07:57:00.000-05:00

Milton Freedman, the renowned Chicago economist, died yesterday at the age of 94. His monetary and free-market ideas have strongly influenced US and world economic policy for the past quarter century. However, the most recent US elections suggest that there may be mood shift taking place.

Currently, the US government spends about 25% of the gross domestic product (GDP) and obtains most of it through taxes and not an insignificant portion through borrowing. The current cost of Social Security is about 4.2% of GDP and is projected to rise to 6.3% by 2030. Medicare's annual cost currently represents 2.5% of GDP but is rising very rapidly and is projected to pass Social Security expenditures in 20 years and reach 11% by 2080. You can find all of this information and more at www.socialsecurity.gov. Thus unless the US starts to curtail benefits and spending or increases taxes it is headed for a budget crisis.

The argument by the free-marketeers is that we need to go to some form of personal savings accounts, so instead of contributing to the government's social security system, you save for your retirement yourself. The program would be modelled after the 401(K) tax deferred retirement plan, in which companies instead of providing a defined benefits pension plan would match contributions to the employee's own 401(K) plan. This then transfers the risk from the employer and government onto the individual.

The idea of Milton Friedman and his followers is that the government should reduce both spending and taxes and the result will be higher economic growth and prosperity for all. It is probably true that lowering taxes does help to increase the wealth of those already well off. However, there is a huge dissipative drag on wealth creation and unless you are above some threshold, extra income probably just goes into expenses or pays down some debt.

For most of the population, the largest expense is housing. The value of a house and rent is mostly just market value, so if the market is tight then any increase in income probably just gets manifested in higher real estate prices. There is an argument that one of the consequences of women entering the job force was that houses became more expensive. While everyone seems to think that real estate is a great investment (and maybe it would be if you bought rental property), you cannot realize any financial gain until you sell and unless you plan on downsizing or moving someplace where real estate value is lower, you won't see the returns as extra income for wealth generation.

The two other major and rapidly growing expenses for the average person are healthcare and college tuition. There is lots of talk about how to reduce costs but I think the increase in cost is real. Thirty years ago, there were limited medical tests and treatments. There were no MRI's, PET scans, costly medications especially for chronic conditions and so forth. For those who have access to good health care, the increased cost is probably worth it. Likewise, in the past universities needed little more than books, blackboards and chalk. Now there are computers, wireless internet, shuttle buses to drive students home, extra security for dormitories, 24 hour gyms, and so forth.

The commonality between healthcare and education is that they are necessarily collectively run institutions. The choice is whether to run these privately or publicly. If the choice is to go private, then public funding can be reduced and the cost savings can be returned to the tax payers who must then pay for these services themselves. However, the likely result is that you only get the service you can afford. A tax cut leading to an increase of 10% or 20% in income makes a huge difference if you have a large income but very little if you don't.

If we decide to fund these and other services publicly then we'll need to raise taxes. The problem is that there is a ratchet effect. With the real estate bubble of the past five years, half the population has bought houses they can barely afford and the other half has cashed out the increased value of their house in home equity loans and spent it. The result is that even a small increase in taxes could hurt a lot of families. So if there is a tax increase, the only viable way of doing so is to only tax those that can afford it.

War on Obesity

2006-10-31T12:02:00.000-05:00

There is a humourous and somewhat sad article in the New York Times last Sunday on the stigmatization of the obese. The article points out a recent research article that calculates that because of American's increasing girth, a billion extra gallons of gasoline (petrol for you Europeans) are burned each year. That means an extra 3.8 million tons of carbon dioxide. So, yes, obesity is now linked to climate change.

There is a lot of talk these days about the obesity epidemic and what to do about it. Many people still believe it is a lifestyle choice. The molecular biologists in the field believe that it is a genetic problem and can only be solved pharmaceutically. Not surprisingly, those most vocal about the magic pill fix also seem to have the most patents and biotech ventures on the side. While both of these points of view are probably true in some sense, they both kind of miss the point. I think that the main reason people are gaining weight is that for our current environment, it is the natural thing to do.

We live in a world where food is extremely cheap and plentiful and exercise is optional. The most logical thing to do it seems is to gain weight and plenty of it. The health consequences of this extra fat will likely not affect most people for many years. Although the incidence of insulin resistance and diabetes is increasing, it is still not clear if moderate weight gain is really all that bad. To quote Katherine Flegal of the Centers for disease Control and Prevention from the Times article: "Yes, obesity is to blame for all the evils of modern life, except somehow, weirdly, it is not killing people enough. In fact that's why there are all these fat people around. They just won't die.”

So what should we do about it? After, three years in this field, I've come to the conclusion that there really isn't much we can do about it on the individual level. Our metabolic systems are so geared to acquiring calories that I believe any pharmaceutical option will likely not be effective in the long run and/or have many side effects. From studies our lab has done on food records, it is quite clear that people generally have no idea how much they eat. I doubt people can will themselves to lose weight. I think the only thing that would work is a wholesale change of our society that would increase the cost or reduce the availability of food and motorized transportation. This is definitely not going to happen by choice or design. So barring a great depression or massive crop failure (which could happen), I think we're just going to have to live with all the extra weight.

Strong AI

2006-10-20T18:29:00.000-04:00

In the late eighties and early nineties, Roger Penrose in two books, presented an argument that the brain cannot be algorithmic and thus the AI program is doomed to failure. Unfortunately, he also proposed that a new theory of quantum gravity may be necessary to understand the brain and consciousness and so his ideas were largely ignored by the neuroscience community. However, I think his argument for the noncomputational aspect of brain function was actually well thought out and deserved more attention. I personally believe the argument is flawed but it does stir up some interesting questions.

Penrose's argument is essentially based on the theorems of Godel, Turing and Church. Godel showed that for any formal system, there will be statements that are true but not provable within that system. Hence, formal systems are incomplete in that there will always be undecidable statements. Turing then showed that for any computer (or any algorithmic system), there exist programs that we know and can prove will not stop but no computation on that computer can ever determine this fact. Penrose then argued that since we (at least Turing and Godel) can determine the truth of such undecidable statements, then we (they) could not be doing that computationally or algorithmically.

The implications are quite profound. It means more than just the futility of traditional AI. It also means the brain cannot even be simulated on a computer because any simulation on an algorithmic machine implies the outputs are also algorithmic. Pushing it further, if the brain is based on physical principles, then this implies that physics itself (or at least aspects of it) can't be simulated on a computer either. This is why Penrose was led to postulate that there must be some new physics out there that is beyond computation. The idea is really not that crazy if you think about it. However, it is definitely not air tight.

I think the hole in Penrose's argument is that he believes that we actually can circumvent Godel's theorem and decide undecidable problems. However, I don't think that this is necessarily true. We don't know what formal system our brain happens to be using so don't know which undecidable statements happen to be true but we can't prove. The ability to prove Godel's theorem and to decide truths for other formal systems that are not ours could be implemented computationally. So, the existence of Godel's and Turing's theorems does not necessarily imply that the brain is noncomputational.

Furthermore, it is doubtful that the formal system of our brains are even constant in time or conserved between individuals. More likely, our brain and hence formal system is constantly changing because of random environmental inputs. Thus, Penrose's argument for the futility of traditional AI may be correct. A truly human-like intelligent machine couldn't be built from a fixed formal system that is knowable. It may need to arise from a massively parallel learning system that constantly changes its axioms. Thus even if you could measure the formal system at some point in time, it would be changed before you could use this knowledge. This would be the equivalent of an uncertainty principle for the brain.

Penrose also rules out the role of randomness in breaking algorithmicity. He argues that randomness can be mimicked by an algorithmic pseudo-random number generator. I don't see why this is the case. Perhaps, true randomness is beyond computation. This then leads to the question of where randomness actually comes from. Perhaps it is a vestige of the initial conditions of the universe. And where did that come from? Well we may need a theory of quantum gravity to figure that one out. Hmm, maybe Penrose was right afterall:).

Fractional Reserve Banking and Inflation

2006-10-16T08:36:00.000-04:00

In a comment to a previous post, the question of why there is inflation arose. Being a complete neophyte in economics, I began to think about this question. Along the way, I discovered some very interesting things about how the monetary system works. I'm not sure if I can answer the question correctly but here is my unqualified answer.

The main mechanism behind inflation seems to be what is known as fractional reserve banking. Here by mechanism, I don't mean what economic factors drive inflation, but simply how does extra money get into the economy. When you get a bank loan, they don't dig into their vault and give you the money. Instead, they simply put those dollars into your bank account. The money is basically created out of thin air. All the bank is required to do is to make sure that they have enough reserves to cover some fraction of their loans. It's a complicated formula but it amounts to something like ten to fifteen percent. Each night, the banks must balance their books and they partially do this by borrowing money from the US Federal Reserve which lends at the Fed rate. In that way the Fed can influence the money supply in the economy. The amazing thing about this system is that in principle the money supply could be any size. When money is lent to you and you buy something from someone else, they can deposit that money back into the bank which can then be lent out again while only keeping ten percent in reserve.

So, when interest rates are low, the money supply expands and we get inflation or a bubble. When interest rates increase, the money supply can shrink and then we can have a slowdown in the economy, a recession or a bursting of a bubble (as we are experiencing now in real estate). If the money supply was completely static then if the economy grew we would experience deflation. (This is happening in some sectors like electronics and food where the cost of production is decreasing faster than inflation.) The problem with deflation is that people then tend to wait before they buy things and that can retard economic growth. So, the Fed tries to engineer a small amount of inflation to keep things going. When the economy is too heated then it raises the rate slightly to keep it in check, which is what the Fed has done for the past two years.

I think one of the reasons why inflation has been relatively benign these past few years even with low interest rates is that the extra cash has been used to fuel the internet bubble followed by the real estate bubble and also our savings rate is so low that banks don't have enough reserve to further inflate the money supply. However, just to make sure I end on a gloomy note, Nouriel Roubini is predicting a recession in 2007 triggered by the bursting of the housing bubble. So interest rates may actually be coming down again.

Rotten Eggs

2006-10-06T15:27:00.000-04:00

There is a terrifying article by Peter Ward in this month's Scientific American. There may be strong evidence that several of the last few great extinctions may be due in part to global warming. There is a clear geochemical signature that the most recent one 65 million years ago that wiped out the dinosaurs was caused by an asteroid strike in the Yucatan peninsula but the Great Dying at the end of the Permian 250 million years ago for example (see my previous post here) looks completely different.

As I wrote before, this extinction was marked by anoxia in the oceans. What I didn't write was that biomarkers such as certain lipids have been found in the ancient strata that indicate the presence of lots of photosynthetic green sulfur bacteria. For energy, they oxidize hydrogen sulfide (H2S) and convert it into sulfur. This means that the oceans were enriched with H2S at that time. If the oxygen level is sufficiently high then the H2S can be confined to the deep ocean by oxygen diffusing downwards. However, if the oxygen level drops enough the H2S will bubble to the surface. In addition to being foul smelling and poisonous, the H2S can also destroy the ozone layer and increasing UV radiation.

The circumstances that lead to this outcome could have been triggered by global warming from
massive volcanic activity that spewed tons of C02 into the atmosphere. This heated the oceans and made it harder to absorb oxygen. The extinction would begin in the ocean and then spread to land. A less intense version of this scenario may have taken place as recently as 54 million years ago at the end of the Paleocene era. During that time the concentration of C02 was about 1000 parts per million. We are currently at 385 and at current rates could reach 1000 by the end of the next century. So, if you start smelling rotten eggs on your stroll along the beach...

A new global fixed point

2006-09-28T11:16:00.000-04:00

James Lovelock's most recent book - The Ages of Gaia, argues that the earth is headed for a new fixed point at a much elevated temperature. He cites several mechanisms that are providing positive feedback to rising temperatures. One example is the emission of dimethylsulfide (DMS) by ocean phytoplankton into the atmosphere. DMS is what makes the ocean smell like well the ocean. DMS also contributes to cloud cover which increases the albedo of the earth. For small temperature increases and increases in UV radiation, phytoplankton upregulate DMS release and provide negative feedback However, recent evidence (I must admit that I haven't checked the primary source) suggests that for a very large increase in temperature, DMS release may actually decrease and lead to positive feedback. Lovelock also believes that the increased global temperatures will turn the Amazon rain forest into a savannah leading to even less carbon sequestration. Studies in the UK have found that as temperatures increase, CO2 is being released from the ground at a higher rate. Aquatic cyanobacteria (also known as blue-green algae), which are photosynthetic and may be the largest carbon sink we have, may also down regulate CO2 intake with increasing temperatures. The bottom line is that relying on Gaia to provide negative feedback to our fossil fuel use may not be a viable option. The earth may transition to a new fixed point where the temperature may be as much as 10 degrees warmer. This could turn much of the currently temperate zones into deserts. Lovelock believes it will end civilization as we know it. Even I think this is probably an overly bleak prediction but it is definitely something to think about.

Breast Milk

2006-06-22T17:26:00.000-04:00

Last week in the New York Times Science section, there was an article that reported on a national breast-feeding awareness program that is suggesting that not breast-feeding your children is tantamount to negligence. Now, I'm all for breast feeding, I think it probably is the best food for an infant. However, I think there is also lots of misinformation about why breast feeding is good.

One of the reasons I often hear is that breast milk confers extra immunity to the infant. I get this from everyone including pediatricians and scientists, and it also appeared in that Times article. The argument stems from the fact that breast milk contains immunoglobulins and lymphocytes. Well, let's think about this more clearly. Immunoglobulins or antibodies are proteins and digestion breaks proteins down into amino acids. That is why pharma always tries to develop small molecule drugs. Protein-based drugs must be injected. You can't take them orally. Now, in certain mammal species, neonates possess pathways to transport proteins in breast milk directly into the blood stream. The jury is still out on humans but it looks like we're not in that category. So, I'm sorry to say, your baby is probably not getting extra antibodies from breast milk.

So why should a baby be breast fed? One thing that has been found is that the microflora of breast-milk fed babies differ significantly from formula-fed babies. You're body has more bacterial cells than it's own cells so having the right mix of micro-organisms is very important. I think this is the main reason we should push breast milk. Establishing the correct microflora environment is probably crucial for digestion and fending off infections. Additionally, the contents of breast milk will vary depending on what the mother eats. Formula always tastes the same. Breast fed babies have been known to prefer what their mothers eat and to have more diversity in their food preferences in general. Cultural tastes may partially be propagated through breast feeding. Also, babies seem to control the amount they eat much better with breast feeding than with a bottle. That may partially be because parents encourage their babies to finish each bottle even if the baby is already full. The bottom line is that their are many benefits to breastfeeding but obtaining antibodies is not one of them.

Hybrid fallacy and car seats

2006-06-09T13:00:00.000-04:00

I think the Toyoto Prius is a great car. It's efficient and looks cool. However, hybrids are not the panacea for our energy problems they are made out to be. For one, the actual gains in fuel efficiency over a well designed gasoline car is not as great as presumed. In fact, if you mostly do highway driving, it may actually do worse because of the extra weight. A diesel powered car is likely to do better. A more insidious problem is that for the most part car companies are not producing hybrids so that they are more fuel efficient but because they can make for a more powerful engine. Ford and Lexus both have SUV hybrids that don't have much higher fuel efficiency than their conventional counterparts but do have a lot more horse power. Despite this shortcoming, these cars still get to go on the HOV lanes. Plug-in-hybrids have more promise if they are used around town but won't be of much help for long haul travel.

I don't fully blame the car companies because that public likes and wants big powerful cars. I think one reason is the infant car seat. We can barely fit our Britax Roundabout seat into our Honda CRV (yes, we own an SUV but it is mostly a tall Civic). We probably couldn't get two in and three is definitely out. In the old days, people would just stuff their kids into the back seat. Now, if you have three kids under eight you would need a minivan or equivalent. I'm pretty sure there could be ways to engineer a fuel efficient car that can safely transport three kids but it would require a collaboration between auto manufacturers and car seat companies. Gas would probably have to hit four or five dollars a gallon before such a thing would happen is my guess. Ultimately, we have to change the way we live and work. The demise of the personal car could be the best thing to happen to cities since the invention of the subterranean sewer.

Corn-ethanol

2006-05-25T10:57:00.000-04:00

Given the recent high price of gasoline, one of the proposed replacements is ethanol derived from corn. The problem with this idea is that it may take almost as much fossil fuel to make the stuff. Today in the New York Times, Michael Pollan, author of The Omnivore's dilemma: A Natural History of Four Meals, has an op-ed piece arguing against corn-based ethanol. Future Pundit also has a recent posting with links to economic and thermodynamic analyses of ethanol production. Currently, the federal government offers a tax break of 54 cents for every gallon of ethanol produced and levies a tariff of 54 cents a gallon on imported ethanol. We also subsidize the farming of corn, which takes a lot of fertilizer, pesticides and tractors, all of which use fossil fuels of some sort. Depending on how you do the estimate, it may even take more than a gallon of fossil fuel to produce one gallon of ethanol from corn. Ethanol could make sense if it is derived from a crop that is more efficient like switch grass or sugar cane. But with the strength of the corn lobby, those other options may never get a chance. Maybe it's time that we stop subsidizing the growth of all that corn. We produce way more than we can eat and high fructose corn syrup may be part of why we're getting fat and diabetic.

ILC or OWL?

2006-04-27T15:09:00.000-04:00

In today's New York Times, Dennis Overbye writes that a National Academy of Sciences panel recommends that the United States spends up to a half a billion dollars in the next five years to ensure that the International Linear Collider (ILC) is built on American soil. They're study says that American physics will lose its leadership in particle physics which "would erode the base of science and technology that has fueled innovation, provided intellectual and cultural inspiration and bolstered national security over the last century." I must confess that I became interested in physics primarily because of the exciting developments in particle physics in the early seventies. It lead me to a degree in physics which ultimately, through a very circuitous route, into theoretical biology. Although I had a lot of catching up to do in biology, I think my physics training was excellent preparation for what I do day to day.

That being said, in our current financial climate where the science budget is being cut in real terms, I doubt that if I had a half a billion dollars to spend, I would put it into particle physics. I think I would rather spend it on a technological push towards alternative energy sources, such as the International Test Fusion Reactor (ITER) or a really big telescope. The European Southern Observatory organization currently has a proposal for building a 100 m diameter Overwhelmingly Large Telescope (OWL). This telescope could detect images 1000 times fainter than the Hubble Space Telescope.

I'm all for promoting projects that may inspire the next generation of physics students but I'm also for spending money on something that I know will guarantee interesting results. My knowledge of the current status of high energy physics is admittedly low but I think the ILC will still be orders of magnitude away from testing string theory for example. Wasn't the scuttled Superconducting Super Collider dubbed the Desertron because it may not find anything at all? Maybe the ILC could shed light on the nature of dark matter and dark energy, which I believe is a pressing problem, but I think a better telescope has a higher chance of providing us with more insights in those areas. The pictures will also be a lot cooler!

Economics and Cosmology

2006-02-28T11:49:00.000-05:00

On Monday of this week, Paul Krugman wrote in the New York Times that the income disparity we currently see is not due to the increasing leverage of education as had been suggested by new Fed chair Ben Bernanke but rather a result of the recent rise of a narrow oligarchy. He points to data showing that only those with incomes in the 99th percentile were really reaping the benefits of increased productivity. College educated people in fact had lost real income in recent years. In an earlier post, I pointed out that the top 400 richest Americans make up of 1% of the entire U.S. GDP.

The prevailing supply side mantra is that "a rising tide floats all boats. " The theory is that increasing wealth at the top will lead to greater investment and higher productivity to which the entire nation will benefit. Unfortunately, this is thinking linearly and not exponentially. When I briefly flirted with Wall Street a decade ago, I was amused to learn that finance was similar to cosmology in that both fields involve stochastic fluctuations on an exponentially expanding manifold. The implication is that small differences will eventually lead to huge differences and this could be an explanation of both galaxy clustering and the growing income disparity.

Only small inhomogeneities in the initial conditions and growth rate can lead to wide disparities in wealth. This is especially true because we must compare all rates against the inflation rate. If you're growth rate is below inflation then you're wealth is essentially heading towards zero. Taxes can serve as a means to slow down the growth rate and nonlinearly saturate the growth for those with great wealth. Cutting taxes on income from capital gains and dividends, which mostly apply to those with disposable income, will only further accelerate the disparity between the rich and poor. The only solution is to try to keep the individual income growth rates as homogeneous as possible and above inflation. Trying to equalize initial conditions (i.e. current wealth) may be harder to achieve politically.

The Greenspan strategy was simply to try to keep inflation in check. However, I think we need to manipulate our current tax system to alleviate the problem. Perhaps we could have a floating tax rate that is calculated on the fly to partially homogenize everyone's growth rate. Another option we could explore is to tax wealth directly rather than income. The one thing we cannot do is to stay the current course.

Patent Law

2006-02-24T13:38:00.000-05:00

For all those addicted to blackberry wireless email, U.S. District Judge James Spencer ruled out an immediate injunction on the service. This case shows how dysfunctional the whole patent system has become. A Virginia-based patent holding company NTP Inc is claiming that Canadian-based RIM has infringed on five patents to operate the blackberry system. The judge has ruled that he accepts this claim although he's not quite ready to pull the plug today. A collective sigh of relief could be heard across the country from millions of "crackberry" users including many in U.S. federal departments like the CIA. RIM has taken a counter offensive and challenged the validity of the patents. The U.S. Patent and Trademark office has thrown out one NTP patent already and RIM claims it has now issued rulings invalidating the other four as well.

I can see how a patent is essential to allow a small fledgling company with a great new idea to get a foothold on the market before another bigger company can step in. However, it is another matter to take out a patent on an obvious idea and then sit on it so you can later sue some other company that takes the time and investment to make it commercially viable. The whole concept of a patent holding company is repulsive to me. Now, every software and biotech company is looking over its shoulder to see where it may be blindsided by some overly general patent issued years earlier. Instead of encouraging innovation and development, the current laws discourage it.

The entire U.S. patent system needs to be overhauled. It is plainly ridiculous to allow patents on DNA sequences or trivial ideas like 'one click' purchasing on a website. It doesn't take a genius to think of wireless email. However, it was RIM that actually got it to work and have it be universally adopted. I think software is intrinsically different from say mechanical devices in that the source code can be kept a secret. For example, it would be like someone patenting a lawn mower but not disclosing the mechanism. The concept of a lawn mower should not be patentable, only the implementation. But that is exactly what is happening for software. I think software patents could have a place but they must be held to a higher standard. A patent should only be protected if a company clearly has a head start using it over some other infringing company. If someone just sits on a patent, they should get no protection.

Free Speech

2006-02-09T16:16:00.000-05:00

This current uproar over the Danish cartoons has gotten me to think about what is free speech. It certainly doesn't mean you can say anything you want. Clearly, laws against slander and libel do not violate the Constitution. Hate speech also does not receive First Amendment protection. The 1942 Supreme Court decision Chaplinsky v. New Hampshire decided that "fighting words", which incite an immediate fighting response "are no essential part of any exposition of ideas, and are of such slight social value as a step to the truth that any benefit that may be derived from them is clearly outweighed by the social interest in order and morality."

If free speech doesn't mean you can say anything you want with impunity then what is it exactly. My personal view is that free speech is not a free pass to express any thought but a safeguard to protect those that lack power to criticize those in power. So a governement scientist should be able to complain that science is being distorted for political ends without the fear of losing his job or a newspaper should be able to claim that a certain politician is corrupt without the fear of being shutdown.

Should European newspapers be allowed to print cartoons that are inflammatory towards Muslims? Having not seen the cartoons, my assumption is that the intention was to criticize certain elements of the Muslim world for using religion to incite violence. However, given that images of the Prophet Mohammed are deemed sacrilegious, I think this could have been done in an editorial essay. The Muslim community is clearly marginalized in European society so this is not the same as say burning the US flag. Also, given that there are laws against denying the Holocaust in many European countries, I think banning images considered sacrilegious to Muslims would not be inconsistent. I am a proponent of free speech but I do believe there can be limits.

Balls and Brains

2006-01-24T15:49:00.000-05:00

Biology is all about trade-offs and it turns out that at least in bats, there is a conflict between the size of sexual organs and the brain. In the current issue of the Proceedings of the Royal Society B, a paper reports on the effects of sexual selection on neocortex and testes sizes in bats. The authors find that the testes of males are much larger in species where the females are promiscuous, than those in species where the females exhibit fidelity. The inverse relationship held for brains. The theory is that both organs are expensive metabolically (especially since bats fly). If females have many partners, the crucial competition is between the sperm and the bat with the most wins. On the other hand, if females are selective about their partners for a given breeding cycle then the quantity of sperm produced by a given male is not so important.

It's not so clear why female faithfulness should promote a larger brain in males. One argument is that there may be a genetic constraint in that genes for both organs are co-expressed. In an earlier post, I wrote about the hypothesis that if females select for a trait in males, then any genes for that trait residing on the X chromosome would be very effectively selected for since males only carry one X. However, this genetic constraint may be a product of other selection pressures. It could simply be that selective females prefer more intelligent mates.

Stardust

2006-01-13T10:02:00.000-05:00

This coming Sunday, if all goes well, the Stardust probe will land in the Utah desert carrying microscopic dust gathered from Comet Wild 2 and interstellar space. This was a seven-year, 2.88 billion mile, 200 million dollar mission. You can follow the exciting progress at the official NASA link. The comet and interstellar dust is captured in an aerogel array mounted on the spacecraft.

It is expected that a few thousand cometary dust grains and just 45 interstellar submicroscopic grains will be captured by the collector. The grains will be embedded at high speed into the gel and create tracks like subatomic particles in a bubble chamber. Digital images will be taken and must be analyzed by hand. It is expected that 30,000 person hours will be required to examine 1.5 million images. NASA is asking for volunteers to take part through a Seti@home-like project called Stardust@home. Each volunteer must first pass a test where they must find a few tracks on a sample image. If two of four volunteers for a given image finds a track it will then be subjected to the scrutiny of 100 more volunteers. If it still passes muster it will then be examined by a crack team of Berkeley undergraduates. The dust grains will then be extracted by a specially designed microtweeser which wasn't even developed until after Stardust was launched. I'm dying to find out what sorts of things they'll find. Who knows, maybe they'll be some organic molecules and life on earth really was seeded from space.

Humans In Space

2006-01-04T16:18:00.000-05:00

In 2004, President Bush presented a new agenda for NASA that included human missions to the moon and Mars. The report is in the President's Commission on Moon, Mars and Beyond. I grew up during the space age and one of the most inspirational moments of my life was witnessing Neil Armstrong walking on the moon in 1969. It certainly was a factor in my decision to pursue a career in science. However, I now firmly believe that manned space flight is intrinsically flawed and should not be supported by the government.

The reasons are two fold. The first is that there is no scientific purpose that requires humans and the second is that humans are very badly adapted to space. Other than bringing back moon rocks (which could now be done with robots) and fixing the Hubble telescope (which could soon be done by robots), there has been no contribution to space science from human exploration. The only science that has been done is the study of the effects of space on humans and the conclusion is that we don't belong there. The weightlessness causes severe muscle atrophy and bone loss. Even more problematic is the high levels of ionizing radiation present in space. The extra cost required for human over robotic missions is astronomical.

The only reason we should go to Mars is the same reason we should climb Mt. Everest. I think this is perfectly fine and honourable but I don't think we should support such a junket with federal dollars. Let some maverick billionaire fund the operation. By the time we actually have the technology to be able to colonize other planets or even go beyond the solar system our knowledge of biology and artificial intelligence will have also greatly advanced. Instead of sending people we could send human embryos or better yet just the genetic codes. Once we arrive at our destination we can simply grow our colonizers from available organic molecules. Robots could raise and educate the first generation. This seems far more sensible than sending people in a lead lined cabin or putting them in suspended animation (at least until we get that Star Trek warp drive, force field and artifical gravity generator working).

Scientific Fraud

2005-12-29T11:51:00.000-05:00

The New York Times is now reporting that the human cloning result of Hwang Woo Suk was completely fabricated. His two Science papers in 2004 and 2005 rocked the scientific community and put South Korea at the forefront of stem cell research. He had planned to open cloning centers around the world to provide research labs with various embryonic stem cell lines. Now, he has been completely disgraced and the field has been set back somewhat.

I think this is an example of how giving unlimited resources to a single individual can lead to bad results. The Korean government gave Hwang almost 40 million dollars since 1998 in hopes of garnering the nation's first Noble prize. He must have felt tremendous pressure to succeed.

I can envision a scenario that led to the fraud. His lab probably had preliminary results that seemed to work but they then couldn't reproduce it. Hwang likely felt frustrated but confident that the method would eventually work so he decided to proceed with publication so they wouldn't get scooped. Maybe he rationalized that there would be little harm to embellish some data to get the news out earlier. In the meantime he would get it to work reliably. He may even have gotten away with it if the papers didn't have as large an impact which led to greater scrutiny of the work.

Sometimes throwing money at a problem does pan out. Examples include the Manhattan project and the Apollo moon mission. In these cases, there was a talented and motivated team focused on the task. There was a sense of urgency but there was also an imperative to be correct. People were checking other people's work because making an error had dire consequences. The participants weren't thinking about future riches or fame. While it is true that Oppenheimer became a household name, he certainly didn't put pressure on the team to succeed so he could become rich and famous. What he did do was assemble the greatest minds of the time.

Will a Manhattan type effort work in biology? I'm not so sure because there is still a lot of basic science to discover. We don't really know what must be done to cure diabetes, malaria or cancer. I think the best thing to do now is to have many labs pursue many different paths. We may even want to divide the money out more evenly than it is now. Someone should do a study to see if it is more cost effective to fund a few big labs or many small labs. I'm betting on the latter.

Life Will Go On

2005-12-20T14:55:00.000-05:00

At this moment, we are undergoing a massive loss of species on par with the six great extinctions of history such as the most recent one 65 million years ago and the Great Dying of 250 million years ago. Being a good existentialist, I agree that we should do all we can to prevent loss of biodiversity even if the effort is likely to be futile. However, a part of me is secure that life will go on regardless of what happens now. Each great extinction is always followed by a flowering of new life. It was only because of the demise of the dinosaurs were mammals able to rise. I agree that it is a major cause of concern as to whether we will survive this extinction but on the long time scale all species eventually go extinct.

Ironically, one reason for my optimism is the large amount of garbage we dump into the environment. The toxicity of a waste product implies that it is bioreactive and hence could be exploited. Remember that oxygen was once an environmental toxin that forced major changes to life on earth. The extra methane and carbon dioxide we currently spew into the atmosphere could likewise be utilized. I'm sure new species of life will be found in garbage dumps sometime in the future.

There are already some signs of adaptation to the modern world even by large animals. For example, deer are now doing so well in cities and suburbs that some communities need to cull them. New York City has several pairs of thriving peregrine falcons. Some fear that beautiful species like orchids, parrots, and sea otters will disappear leaving behind only drab scavengers like cockroaches, coyotes, crows and rats. I think we need to give life more credit. I don't think beauty is necessarily correlated with fragility. Also, our standards of beauty have been shaped by our environment. As the environment changes, so too will our brains and hence what we consider beautiful. Barring a catastrophic event like a nearby supernova or a complete loss of atmosphere, I think we can be fairly certain that life will continue to flourish on earth in spite of what we do.

Observations of a Dad

2005-12-03T15:18:00.000-05:00

When a baby is born, she immediately has a set of reflexes - like rooting for a nipple, crying, grasping, and sleeping. These reflexes are hardwired into the brain and get triggered by certain sensory cues. Hunger and discomfort trigger crying, stroking of the face sets off rooting, and putting something in her hand initiates grasping.

What makes a baby fussy or easy going is coded in the thresholds that trigger the reflexes and shuts them off. Some babies can tolerate a large amount of discomfort before they cry and transition from an active state into sleep quickly. These are easy babies. In fussy babies, the discomfort thresholds may be very low and small perturbations can trigger crying. Most babies are somewhere in between - fussy in some aspects, impervious in others.

The multiple thresholds are set at birth and in essence define the initial personality of a baby. There is probably a genetic component but I bet most are set entirely randomly. After the baby is born, neural plasticity can shift these settings. So depending on how the parents react to the baby, thresholds could be moved up or down.

An entire book industry has sprouted in an attempt to educate parents on how to train their baby to be an easy going one. However, I doubt there will ever be a surefire method. The different sensory and reflex modalities probably interact in a highly nonlinear fashion. So trying to make a baby less sensitive to one thing could make them more sensitive to something else. Personally, I think we should just enjoy our babies the way they are. But then again, I think my baby is pretty easy, even if she stays up all night.

Life with Baby

2005-11-21T17:57:00.000-05:00

Time stands still, time rushes forward.

A day is gone and what has happened?

Change her diaper, feed her, make a trip to the doctor, go get food, do the laundry, do some more laundry, make another trip to the doctor.

Did I miss a day? When did I last sleep?

Yet, when she opens her bright wide eyes and sneaks a peak, my heart melts.

When she cracks a wry smile, I just want to hold her close.

What if she could be like this forever, so tiny in my hands, so beautiful?

She waves her arms about and alights one gently on her cheek.

She is my one joy, my being, my life.

She is my baby.

Why vote?

2005-11-09T20:14:00.000-05:00

In last Sunday's New York Times magazine, the economists that brought us Freakonomics argued that a rational individual should abstain from voting. According to them, virtually no election is decided by a single vote so the cost for voting is never compensated by any payoff. Their conclusion of why we (or some of us) actually do vote is because of the social esteem gained by being seen voting by our peers. They predict that internet voting may actually reduce voter turnout because we would no longer get this social payoff.

It seems to me that using social prestige as the basis for selecting a leader is a shaky way to maintain a democracy. The real intent of an election is to determine which candidate is favoured by the majority of the populace. For the most part, our current method accomplishes this task (insert your favourite Florida 2000 joke here) although it has two main problems.

The first is that making everyone vote to determine who is preferred is wasteful. In statistics, this is known as an overpowered experimental design. We only need to sample a fraction of the population to obtain an estimate of the election result. The error on the estimate scales with the square root of the sample size. In an election, when enough people have voted so that the error in the estimated result is less then the eventual margin, additional polling won't give you any new information. This is why a single vote doesn't matter.

In a practical sense, this is already what we do because only a fraction of the population votes. However, the fraction that votes is not guaranteed to be a representative sample of the population. Any bias in how the sample is selected will bias the estimate. This leads to the second problem with elections. In a close election, who shows up to vote could skew the results. One facet of election strategy is to enhance the turnout of your voters and suppress that of your opponent's. This never seemed very democratic to me.

If we really wanted to elect leaders based on what the true majority wishes then an election is not the optimal method. What we really should be doing is to scientifically select a sample of the population to vote. Of course deciding on how to choose this sample will never be perfect. There will also be some misrepresentation (like underrepresentation of homeless people) but I think we can certainly do better than what we have now. If we wanted to be really efficient we could even use a bootstrap method to estimate the error compared to the estimated margin of victory on the fly. Now, I'd like to see some brave politician suggest this scheme.

Dark Genome

2005-11-04T18:05:00.000-05:00

Cosmologists are very troubled by the fact that they can't account for (depending on whom you ask) 90% to 99% of the mass and energy of the universe. The nature of this "Dark Matter" is the most pressing problem of their field. However, biologists don't seem nearly as perturbed by the fact that the purpose of a similar fraction of the mammalian genome is completely unknown. They are so unconcerned that only a small fraction of the genome is in the genes that code for proteins that much of the non-coding region is simply called junk DNA.

It has always perplexed me why most of our DNA would be junk. I can't believe that 90% of the DNA has no use whatsoever. It would seem much more likely that this so-called junk DNA is necessary for genetic regulation. After all, the main reason I am different from another person is not in the differences in the proteins I carry but in how and when they are expressed. Darwin himself recognized that much of the variation in nature must be due to regulation.

A very nice paper by Peter Andolfatto in the October 20 issue of Nature shows that in the fruit fly between 40% to 70% of the DNA nucleotides situated between genes are under selection pressure by evolution. He showed this in a very clever way. He analyzed the DNA of two species of Drosophila - D. melanogaster and D. simulans and looked at the level of polymorphism (differences within a species) and divergence (differences between species) in the genome. As a control he looked at synonymous sites (region in the coding region of DNA where a change in the nucleotide does not change the amino acid it codes for because of redundancies in the nucleotide triplet code).

Andolfatto found that the rate of mutation in non-coding regions is slightly lower than in synonymous sites indicating these sites have undergone negative selection pressure. Additionally, he found that the divergence rate in selected sites was increased relative to the polymorphism indicating that they also experience positive selection pressure. In other words, most mutations in these regions are deleterious and thus are selected against but every once in a while a nucleotide substitution confers some advantage and this is selected for. The bottom line is that these non-coding regions are crucial for the survival of the organism.

What these non-coding regions are for is unknown. The current dogma says that gene expression is controlled by sets of transcription factors that act on various promoter regions. According to Alex Kondrashov in the accompanying News and Views piece, current estimates of the fraction of functionally important segments of mammalian non-coding DNA is less than 15%. Although, an equivalent study still needs to be done in mammals, I'm betting that a significant portion of what is thought of as junk DNA is used for regulation and in a completely novel way.

The Make Work Society

2005-10-26T18:25:00.000-04:00

Recently, I've been thinking a lot about economics, the dismal science. In particular, I've been thinking about the creation and distribution of wealth. Of course this question has been analyzed countless times before, most notably by Marx, but I'll add my take anyway. Let's define the wealth of our society as the sum of everything we create. More precisely, we create wealth at a given rate and it dissipates at another rate. Some things like a restaurant meal or a pop tune, dissipate rather quickly, and others like a gold figurine can last a long time. For the most part, we only get a share of this wealth if we work.

As time goes by, we get more efficient and increase the wealth creation rate (or decrease the dissipation rate). While that seems to be a good thing it does generate some problems. One of them is quite simple. If you get more efficient at making something and you work the same amount of time, then you'll make more of that thing. However, unless someone wants to buy more of your thing, the extra stuff you've created will just sit around and not be of much use. They way around this problem is to either convince people they need more of the same stuff (i.e. marketing) or to produce other stuff. That is why thirty years ago you had a pair of dress shoes and a pair of athletic shoes but now you must have running shoes, basketball shoes, tennis shoes, squash shoes, mountain biking shoes, climbing shoes, light hiking boots, heavy hiking boots, and so forth.

However, even the best ad agency can only convince us to consume so much of one thing. So, once a market is saturated, increased efficiency means fewer people are required to make that product. These people must then find new ways to work to get a share of the wealth. Some can either make new stuff for you to buy like iPods, digital cameras, gourmet food, and storage crates to put all this stuff in or create new services like personal chefs, dog walkers, and fitness instructors. Most likely, they'll work at a low paying jobs like fast food server.

More efficiency means we will be inundated by more stuff and services we don't really need or want. Many of the new jobs will not be well compensated so the gap between the rich and poor will grow. In some sense, our solution to distributing wealth is to create a "make work" state where most of the people support themselves with artificial or menial jobs. I don't really see this as being much better than a welfare state.

One solution is that we could choose to keep wealth creation per capita fixed so as our efficiency increased we would simply work less for the same pay. So if a factory can make the same shoe in half the time then people could just work half as long. Cutting the work week even just a little bit could solve our unemployment problem. The US has twice the wealth of most western European countries. As I wrote before, we are already more than rich enough. What we now need is the time to enjoy some of this wealth.

Your Ecological Footprint

2005-10-20T21:37:00.000-04:00

The ecological footprint gives an estimate of how much productive land and water is necessary to support what you use and discard. This includes all the land you need for the food you eat, the fossil fuels you burn to sustain your lifestyle, the amount of forest required to absorb the green house gases you produce, and the waste you create. There are many websites estimating the ecological footprint. I recently tried www.myfootprint.org. You fill out a questionnaire and then it gives you the result. According to this website (and I didn't check their algorithm) the average American has a footprint of 24 acres. Worldwide there only exists 4.5 biologically productive acres per person. I answered the quiz twice. The first time I gave what I thought was the upper bound for my current lifestyle and I got a result of 24 acres - the American average. I then tried it again using the lowest bound, ignoring my now long commute and it still came up with 8 acres. It is almost impossible to live in the US and get below this value. Our whole infrastructure is based on an inexhaustible supply of natural resources. I am simply not paying for the true cost of having a New Zealand grown apple in February, working in a climate controlled office year round, and being able to jet over to Europe when I need to. If this is true then we're going run into serious trouble when the rest of the world starts to consume like we do.

Stem Cell Loopholes

2005-10-16T21:30:00.000-04:00

When gas mileage regulations were tightened after the energy crisis in the late seventies, the American car companies hired lawyers to challenge the regulations while the Japanese companies hired engineers to find ways to comply. Given that Toyota is on the verge of surpassing GM as the world's largest car company, it seems pretty clear which was the better strategy.

Now, a similar situation exists for stem cell research. In an earlier post, I wrote about some of the ethical questions surrounding the issue. Current US regulations only allow federal funding for human stem cell research on embryonic stem (ES) cells derived from a fixed set of cell lines established prior to August 2001. However, many of these lines have been contaminated and researches would like to create more lines. So while some people are striving for a political solution to obtaining more stem cell lines, another group is looking for a technological fix to ease the ethical and religious concerns.

Two papers appearing in today's Nature advanced online publications, demonstrate possible methods in mice to obtain stem cells without technically destroying an embryo. The first by a group from Advanced Cell Technology, extracts a single cell from a developing blastocyst when it is comprised of 8 cells. They then show that they can create an ES line from this single blastomere that remains viable even after 50 divisions. The second paper from MIT, uses a method called altered nuclear transfer (ANT). This is a means of doing therapeutic cloning without destroying an embryo. Instead of implanting adult cells directly into a donated egg that can become a viable embryo to harvest stem cells, certain developmental genes are first deactivated in the adult cell so that the ensuing blastocyst can never implant in a uterus and hence fully develop.

Both approaches have been embraced and criticized and the New York Times has a synopsis of the reactions. I find both approaches to be rather unsatisfactory and expose the hypocrisy of the whole issue. Evidently, fertility clinics already do single-cell embryo biopsy for genetic screening prior to implantation. Supposedly, it is safe and there have been many successful births from embryos that have had a cell extracted. But I wonder who were the first parents to have had this procedure done on their baby? It sounds like rolling the dice on a life to me. The child may seem normal now but we don't know if there are any long term consequences. ANT seems completely contrived. The altered blastocyst is completely identical to a normal one when the stem cell is extracted. The only difference is that if both happen to be in a uterus, then one can implant and the other can't. However, unless you actually implant it you'll never know. So, if we were to say that we fully intend to alter a nucleus prior to implantation for therapeutic cloning, would that be good enough?

If one wants to have a fully consistent position on stem cells then one can either be for them or against them. There is no middle ground. There are no semantic loopholes. Each cell contains the entire genome so it has all the information to create life. Destroying a cell is thus equivalent to destroying an embryo. If one is against stem cell research then one must be against all genetic and cellular manipulations in humans (and perhaps all animals). That means no in vitro fertilization, no gene therapy, no ultrasound imaging of the fetus and certainly no amniocentesis. One could possibly take it further and say that any biological research that destroys cells should not be performed. People must start accepting that it's the software and not the hardware that defines a human life.

Forbes 400

2005-10-06T18:08:00.000-04:00

Every Fall, the Nobel Prizes are awarded and Forbes Magazine publishes a list of the 400 richest Americans. The bottom line is that the rich are definitely getting richer. Bill Gates again tops the list with 51 billion dollars although the five members of the Walton family (numbers 6 through 10) if combined would exceed his fortune with close to 80 billion. Rounding out the top 11 are Warren Buffet at number 2, Paul Allen at 3, Michael Dell at 4, Larry Ellison at 5, and Steve Balmer at 11.

The combined wealth of the Forbes 400 is 1.13 trillion dollars. To put this in perspective, according the CIA world factbook, the US GDP in 2004 was 11.8 trillion dollars and the world GDP was 55.5 trillion. If we assume an income of 10% per year, the Forbes 400 makes up 1% of the US GDP and 0.2% of the world GDP.

The US per capita GDP works out to be about $40,000 a year. Thus, if income were distributed evenly, every family of four would take home $160,000 per year. You wouldn't know it after seeing the effects of Katrina but the US is wealthy enough such that every family could be upper middle class. The US is by far the richest nation on the planet. In comparison, the other group of 7 nations all have per capita incomes below $30,000.

So what is the solution to poverty? I think the argument that it can be eliminated with economic growth seems to be proven wrong. We already are more than rich enough to ensure that every person could have a comfortable life. Obviously, some form of reallocation is necessary although I'm sure there are those who would argue that any attempt to redistribute wealth would only decrease US productivity. However, this imbalance cannot be sustained forever. It took a great depression and two world wars to lessen the income disparity from the robber baron era of a century ago. Do we need to go through something catastrophic again to repair our current inequities?

Grass Power

2005-09-28T12:31:00.000-04:00

A possible promising crop for biofuel may be a form of Elephant Grass (Miscanthus x giganteus). This tall grass hybrid could yield up to 60 tonnes of biomass per acre. It is currently being tested in Europe. It thrives in northern climates and requires very little fertilizer. It has a low water content so it can be easily harvested at the end of the growing season and simply burned for power. Stephen Long of University of Illinois calculates that if 8% of all the land in Illinois was dedicated to this crop, it could generate half of the electrical needs of the state. It also has little effect on the carbon balance because it simply releases the CO2 when burned that was sequestered when it was growing. The hybrid does not produce seeds so it can't spread. Further bioengineering could make it even more efficient. Who says I can't say something positive?

Programmed for Failure

2005-09-22T18:20:00.000-04:00

Every time I feel kind of optimistic about the future, I think back to the Roman Empire and realize that it could all end pretty quickly. It may be no accident that civilizations tend to have finite lives and our brains may be responsible. Jared Diamond(in his book Collapse) posits a framework for a society's demise but he basically believes it is some combination of bad decision making and management that leads to failure. I'm proposing that it may actually be embedded in how our brains work and how it reacts to success. What allows us to build great civilizations may ultimately be responsible for our undoing.

As has been written in countless columns and blogs, manufacturing, software development, clerical work and so forth is being or will soon be outsourced to an offshore location where labour costs are so much lower. Many have argued that the US can retain world dominance by remaining a source of innovation and ideas. However, Thomas Friedman and others have been screaming lately that the US is losing it's lead in technology and science and American students are falling behind the rest of the world in technical subjects.

The reason is not just that we've become lazy or stupid. The Flynn effect shows that average IQ's have actually been rising every generation and in the recent book Everything Bad is Good For You: How Today's Popular Culture Is Actually Making Us Smarter, Steven Johnson argues that video games and popular culture are actually making us smarter. So why is it that we are becoming less intellectual even though we are getting smarter?

I think it is related to the fact that it takes effort to concentrate on something. This effort is not because we're using more energy. Although it may seem that thinking hard burns more calories, there is in fact little evidence for this. So if there is no metabolic cost then why is it so difficult to think? The reason may be that the brain is a novelty machine that constantly seeks new stimuli. Advertising and marketing people know that they need to change a scene every 10 or 15 seconds in a commercial or people's attention will be lost. Our brains are designed to wander and seek new stimuli. This constant novelty seeking probably helps in the early stages of a civilization where things need to be built and everyone sees open opportunities for growth.

As a civilization matures, it takes longer and longer for the citizens to acquire and digest the accumulated knowledge required just to keep it running much less advance it. Years of training is necessary before anyone can make a contribution. Given our current comfortable circumstances, there is little incentive to undertake such an ordeal when there are so many other distractions to occupy us. In the past, scholastic learning might have been the most cognitively stimulating thing one could engage in. Now, our lives are filled with leisure activities that are much more interesting and entertaining than what we learn in school. For every high school kid with his nose stuck in an analysis textbook, there are hundreds or thousands of other kids who are playing video games, surfing the web, reading a Harry Potter novel or solving a Sudoku puzzle.

Is there a way out? I'm pessimistic. While it is true that those on the cutting edge are doing very interesting and stimulating things, the journey to get there is so long and arduous that fewer and fewer are likely to take it. No matter how appealing you may make calculus or organic chemistry, they just will never be able to compete with the endless variety of distractions in modern society. There will still be an educated elite but there won't be enough of them to keep the engine going.

The decline of the US could be very rapid. Even now, much of science and technology is being driven by foreigners. However, as the balance of power starts to shift overseas and the US remains xenophobic, that spigot could be shut off quickly. The incentive to come here will diminish and people may return to their native countries as things decline here accelerating the process.

It may be that the only hope for humanity is to maintain uneven economic development. If the entire world became comfortable simultaneously, it might completely collapse all at once. However, if the decline of the US is accompanied by the rise of China and India then at least some order in the world could be maintained. After a century or so, the US could rise again in a perpetual cycle of localized growth and decay.

Peak Oil Production

2005-09-18T14:29:00.000-04:00

We all know that the supply of oil is finite so the big question is how much do we have left. Geophysicist M. King Hubbert created a model of known oil reserves in 1956 and proposed that American oil production would peak between 1967 and 1972. US oil production peaked in 1971 and it's been downhill ever since. Hubbert died in 1989 but other geologists have applied his theory to global production and predict a peak between 2000 and 2010.

This month both American Scientist and Technology Review have book reviews of James Howard Kunstler's book 'THE LONG EMERGENCY: Surviving the Converging Catastrophes of the Twenty-First Century. Kunstler's thesis is that the 20th century defied Malthus because we have been living on cheap oil but when it does run out we will be in big trouble. He argues that all alternative sources of energy like solar, wind, nuclear, biofuels, hydrogen are a pipe dream that won't even come close to replacing oil. He believes that the depletion of oil will lead to social unrest and upheaval of the likes we've never seen before. The 14th century with the black death and all was pretty bad so this is really saying something.

The events of the past few weeks has sent me into a "The World is Going to End" kind of mood so I'm rather susceptible to this message. However, I think that we still have a chance to save ourselves. Conservation measures could prolong the supply of oil for say another century. This would buy us time to bring all alternative energy sources online. We will probably have to depend on nuclear power for much of it. If we're really lucky, we might get fusion to work in 50 years but that will also bring it's own set of problems. We will have to use bio-derived fuels for plastic and to power airplanes. However, our current unstainable American standard of living will decline. How far it drops is up to us.

Math and Biology

2005-09-14T14:27:00.000-04:00

There's a nice article on the synergy between my two favourite subjects in the December 2004 issue of PLoS Biology.

Pitch perception and beyond

2005-09-08T21:52:00.000-04:00

American symphony orchestras tune their instruments to A 440 Hz (in Europe they tend to go a little sharper with A 444Hz). However, no instrument produces a single frequency. Instead, they tune their instruments so that it sounds like an A. Humans can do this quite easily but if you were to look at the spectrum you would see a mess of frequencies. If you removed the fundamental frequency and just listened to the harmonics of a sound you would still identify the pitch you hear as that of the fundamental. It has always been quite puzzling as to how the brain does it. In this week's issue of Nature, a group from Johns Hopkins reports that it has found neurons in the auditory cortex of the marmoset (Callithrix jacchus) that respond to pitch, the way a human responds. These neurons will fire when the animal is presented with the pure tone fundamental or a set of harmonics without the fundamental.

This is without a doubt a very nice and illuminating piece of work but I wouldn't say it was surprising. Ever since Hubel and Wiesel discovered orientation selective neurons in the visual cortex in the late fifties and early sixties, much of systems neuroscience has been driven to providing more and more exotic stimuli and looking for neurons that respond to them. Given the data from the past forty years, I would venture that for anything which we can sense, perceive or ideate, there exists some neuron who's activity is directly correlated to that thing. That is not to say there is just one neuron that responds to some given concept. The other lesson we learned from Hubel and Wiesel is that neurons are broadly tuned over some category. Thus, for any type of perception there will be a population of active neurons.

By any type of perception, I mean all aspects of a thought. So if you are viewing a Cezanne landscape for example, there will be neurons responding to primitive elements like shapes, lines, colours and so forth. Simultaneously, there will be other neurons that respond only to more specific things like a coloured square or to a pair of adjacent coloured squares. Then there will be neurons that respond just to trees or houses and if you know enough about art just to Cezanne paintings. What we don't know is what sorts of neural architectures and learning rules can give rise to such behaviour and how all this cacophony of activity gets sorted out. While there are some candidate ideas floating around, the jury is definitely still out.

Aquarium of the Americas

2005-09-07T17:05:00.000-04:00

My favourite aquarium was the one in New Orleans. It had the most spectacular jelly fish display I have ever seen. I use the past tense because CNN reports that most of the sea creatures have died because of the power loss which shut down the water oxygenation systems. A few of the animals like the 250 pound sea turtle, the white alligator, birds, sea dragons and sea otters have survived. Another loss in this major tragedy.

Agriculture's downside

2005-09-02T16:45:00.000-04:00

Brad De Long has posted an opinion piece from Jared Diamond arguing that agriculture, which displaced hunting and gathering as the means for sustenance, was actually humanity's greatest mistake. The article contends that agriculture promotes a much larger population with a lower quality of life for most people except for a dominating elite. In a recent post, I argued that welfare could be considered compensation for eliminating the right to forage. According to Diamond, that would hardly be a fair deal.

Fragility of civilization

2005-09-02T11:29:00.000-04:00

The disaster unfolding before our eyes is even more troubling because it could have been mitigated in so many ways. I don't need to add anything more to the disbelief and anger spreading across the nation but it is hard not to. Among the many lessons to be drawn is that we are just a few days away from a complete breakdown of civil society. If ever there was an argument that government serves an essential role then this is it. It is quite clear who was able to get out and who was not. To blame the victims for their predicament is beyond reproach. If a city gives a mandatory evacuation order it must also provide a means for evacuation and resources for the evacuees. I can only hope that this tragedy will make us reevaluate what a just and fair civilization really means. Given that "values" was an issue in the last election I will quote directly from the bible:

The righteous is concerned for the rights of the poor; the wicked does not understand such concern. Proverbs 29:7

The flooding of New Orleans

2005-08-31T11:56:00.000-04:00

It seems that the aftermath of Hurricane Katrina will be of much greater concern than the storm itself. Eighty percent of New Orleans is currently under water. Two levees holding back Lake Pontchartrain and the Mississippi have burst and water is pouring in (Here is a map from the NY Times). Since most of New Orleans is below sea level, this water cannot drain out. It will have to be pumped out. As of now, workers are still trying to repair the levees. Science writer Mark Frischetti wrote about such a possible disaster in the October 2001 issue of Scientific American. The only good news is that most of the inhabitants evacuated before the storm and water did not overwhelm the levees initially.

The nightmare scenario is that a severe storm surge will flow over the levee walls and flood the city quickly. The levees that were then designed to keep water out will now keep water in. In such a scenario, all of New Orleans would be under water up to 10 or more metres. Right now, the parts of the city above sea level like the French Quarter may be spared. I know I'm a doom and gloom kind of guy but it could take months just to make New Orleans habitable again. Half a million people could be displaced for a long time. I know the sentiment will be to rebuild the city but this will not be the last time a major hurricane will pummel the city. Wetlands that used to protect New Orleans to the south and east are diminishing at a pace of an acre very 24 minutes. The city sits directly in the path of where the Mississippi really wants to go and to top it all off it is slowly sinking. Should we seriously consider if it is worth maintaining New Orleans?

The Flying Spaghetti Monster

2005-08-30T10:22:00.000-04:00

I've always been partial to linguini but it turns out I've been worshipping a false prophet. We must now give praise to the Flying Spaghetti Monster, who came to Bobby Henderson in a divine vision. You can learn about the Church of FSM at venganza.org. Henderson has written an open letter to the Kansas State Board of Education arguing that if intelligent design is to be taught in schools then all forms of it including those involving the FSM must also be taught. He is threatening legal action if they don't comply. He has started a whole movement of followers including splinter groups. There was a humourous column about it in the New York Times yesterday. In that article the author wonders if anyone has every converted parody into a religion. Does the Church of Scientology qualify? Personally, I'm all for "teaching the controversy". After all, less than 50% of the population believes or even understands evolution anyway (see a previous post). It may even stir up some interest in science.

Mind enhancing drugs

2005-08-24T23:22:00.000-04:00

An article appearing yesterday in PloS Biology reports that an Ampakine known as CX717 can alleviate cognitive impairment due to sleep deprivation. The study was done in monkeys although the company that produced the drug - Cortex Pharmaceuticals, has announced in a press release that there is evidence that it works in humans as well.

The monkeys were first tested on a simple cognitive task (delayed-match-to-sample). They performed significantly better when administered CX717. After sleep deprivation of 30 to 36 hours, the same monkeys showed a markedly decrease in ability to perform the task. However, when given CX717 afterwards their performance improved dramatically, even exceeding normal levels.

The brains of the monkeys were imaged with a PET scan for glucose use during these tests. The researchers found that the medial temporal lobe (MTL) and dorsal prefrontal cortex (DPFC) showed enhanced activity during the task. With CX717, there was a slight increase in activity in these two regions and a greater increase of activity in the precuneate cortex. Sleep deprivation caused an increase in activity in MTL and precuneus but a decrease in the DPFC. With CX717, the brain activity in the sleep deprived animals approached normal vehicle levels.

Ampakines are positive modulators of the glutamate AMPA receptor. Glutamate is the main excitatory neurotransmitter in the brain. The AMPA receptor when activated produces a short (few ms) excitatory post-synaptic voltage pulse. Ampakines can make this pulse stronger and last longer. The interesting thing is that the addition of CX717 to the sleep deprived animal increased activity in some areas but decreased it in others. This goes to show you that jazzing up excitation in the brain does not necessarily lead to increased activity. Unfortunately, PET scans can only tell us about changes in glucose usage and not the actual neural activity.

It will only be a matter of time when these drugs hit the streets and college campuses. Students already take speed to stay up and Ritalin to enhance concentration. NMDA and CREB enhancers to boost memory will also soon be on the market. The CB1 blocker rimonobant will soon be approved as an obesity drug. Personally, I wouldn't go near any of this stuff. We have no idea what long term effects these drugs will have. The brain is probably pretty optimized so any enhancement will have some trade-off. I think I would like to know what that cost will be before I decide to mess with my brain.

Tenure and Welfare

2005-08-21T11:05:00.000-04:00

One commonly held notion is that the university tenure system is instituted mainly to protect those in the social sciences who are critical of the current establishment or hold controversial views. This is of course highly valuable for a free society. Recently there has been some suggestion that tenure should be disbanded in the sciences and be replaced with 5 or 10 year renewable contracts. The main reason for this argument is that every department has unproductive faculty and there is no way to replace them.

However, it could be argued that academic freedom is just as important for the sciences. Faculty need the ability to pursue risky ideas that may have no return. If Andrew Wiles was on a five year contract, he may never have had the peace of mind to hole away in an attic for seven years to prove Fermat's Last Theorem. Progress is not possible without failure. The tenure process should be decided carefully but once conferred, not producing another piece of research for the rest of one's career must be considered to be an acceptable outcome.

The argument could be expanded for justifying unemployment as an acceptable career choice. A just and free society should provide a minimal standard of living for all citizens. There are both practical and moral reasons for this stance. Even in our current society, people do accept that some people either through misfortune or bad judgment fall through the cracks and need temporary help to get back on their feet. One could also argue, although I bet this gets less support, that welfare could serve as a stipend for segments of the society wishing to pursue interests in areas that have no current commercial value such as writing a novel or painting. This is akin to academic freedom on the society scale.

But the most compelling argument is a moral one. A person born into a structured society does not have the freedom to live in their natural state. They must follow the conventions imposed upon them. They have no choice to opt out. They cannot choose to become a hunter-gatherer. In fact, few of us have absolute choice over what we do to support ourselves. We have some choice in the area we are trained but the market ultimately decides where we are hired. Thus, one could argue that compensation for eliminating this freedom could be a welfare system. Now, obviously, if everyone chose to take welfare the system would collapse. So to make it viable, any form of work should have a higher compensation than welfare. Did I hear someone say minimum wage?

Fruitless

2005-08-16T20:10:00.000-04:00

I've always found so-called innate behaviour to be much more puzzling than learned behaviour. I can somewhat fathom how a neural network might learn a complex task through training but how does a gene do it? My research program over the past ten years has basically been to show that just knowing the connections of a neural network is not enough to specify what it does. Details like time scales and synaptic strengths matter crucially. Although the neural circuit of the worm C. elegans has been mapped out for quite some time, we still don't know how the creature functions. Since most of the animal world functions just fine on genetically programmed traits, we must be missing something.

An advanced online paper in Nature (Manoli et al., June 15, 2005) has found that the fruitless (fru) gene in Drosophila is responsible for male courtship behaviour. This gene encodes a set of male-specific transcription factors that are expressed in about 2% of neurons in the central nervous system. Inactivating this gene completely wipes out all male courtship behaviour but seemingly preserves all other functions. Expressing the gene in females induces courtship behaviour. Yes, females with fru will attempt to mate with other females even though they lack the apparatus to do so.

Most interestingly is when the gene is selectively inhibited in specific systems like olfaction. Naive males will typically court all drosophila they encounter, male or female, but they quickly learn to not try to mate with other males. However, when fruitless is inhibited in the olfactory receptor neurons, the flies will persist in courting males. This fact has been played up in the press as evidence of a gay gene but it actually indicates an inability to distinguish between males and females. What is amazing to me is that a single gene (although it does encode for a number of proteins) has a nonmonotonic action. Knock it out everywhere and the fly won't mate; knock it out in a specific location and the fly won't stop mating.

We still have no idea what the gene does but this result seems to imply that male courtship behaviour is pre-programmed into the neural circuitry and is activated by priming a subset of neurons or neural connections. Now we need to do the electrophysiology on the neurons expressing fru and try to untangle this mystery.

Podcasts

2005-08-14T22:16:00.000-04:00

I've been occupying my time during my two hour train rides to work by listening to podcasts on my Apple iPod. My listening selections include The Science Show and All in the Mind, both produced by the Australian Broadcasting Corporation and Quirks and Quarks produced by the Canadian Broadcasting Corporation. I used to listen to Quarks on CBC radio when I was a kid in Toronto. It was hosted by David Suzuki in those days.

I'm amazed at how good the Australian shows are. They focus on a topic each week and interview several experts from all over the world. The hosts of the shows are highly knowledgeable. These shows are so much better than NPR's Talk of the Nation Science Fridays. It is rather sad that the United States does not have a decent science show on the radio. It is no coincidence that Canada and Australia are both Commonwealth countries that followed the tradition of a national network in the vein of the BBC.

Stem cells, cloning and beginning of life

2005-08-06T16:39:00.000-04:00

The stem cell debate and cloning was again in the news this past week. A South Korean team announced that they had successfully cloned a dog (a very cute Afghan named Snuppy). Dogs had been notoriously difficult to clone and the team only obtained one success in one thousand tries. This work shows that it is just a matter of time before primates including humans will be cloned.

In terms of therapeutic promise, this research implies that someday we will be able to extract genetic material from a person and create a blastocyst from which embryonic stem cells could be harvested. These cells would be pluripotent and potentially be able to replace or repair any tissue in the body. Additionally, they would be a complete genetic match to the donor eliminating the chance of rejection and the need for immunosuppressive drugs which have many side effects.

In order to make this work we first need to understand how to manipulate stem cells to create desired cell types. Right now we have very little understanding of what causes differentiation in cells. Implanted stem cells could possible replace damaged neurons but they could also become tumor cells. Currently, federal funding is restricted to research only on established embryonic stem cell lines. Unfortunately, many of these lines may be contaminated with other genetic material or damaged from repeated replication. While the rest of the world is pushing forward the US is beginning to lag behind.

However, the tide may be turning. In May, the US House voted overwhelmingly to repeal the ban on creating new stem cell lines. This past week, US Senate majority leader Bill Frist, shifted his position and is now supporting a bill to expand federal funding of stem cell research although the president is threatening to veto the measure.

The argument against the use of embryonic stem cells and cloning is the same as that against abortion and that is the destruction of an embryo is tantamount to taking a human life - the premise being that life begins at conception. The curious thing is that those that support this position don't seem to have a problem with in vitro fertilization where many eggs are extracted and fertilized to create an embryo but only a few ever make it to term. The rest are either frozen, donated or discarded.

That aside, the notion of a well defined moment where life begins is not so clear cut. Is it the moment that the sperm fuses with the egg or the moment that the formed zygote implants in the uterus? If it's the former, then why not make it the moment the sperm collides with the egg or even the moment the sperm will inevitably collide with the egg. Given that we now know any cell in the body can become a new life form, should we prohibit the destruction of any cell? Should we go further and prohibit the destruction of human genetic material of any form including the sequence itself?

Someday, the only thing we'll have left of monarch butterflies, giant pandas or blue whales will be the sequence. Currently, we can build a virus starting from just the genetic map but eventually we will be able to reconstruct any life form. What protection should the genetic code have when it's erasure implies the extinction of an entire species? Perhaps in the distant (or not so distant) future, we will reproduce entirely algorithmically. A computer could combine the sequences of two people and generate the genetic material for their child. Suppose there were only one copy of that sequence and it were destroyed. Would that be murder? When biology fully merges with computer science, how do we define life then?

Bursting of the low carb bubble

2005-08-01T22:58:00.000-04:00

The backlash against the Atkins diet is definitely on. Atkins Nutritionals, the company founded by the namesake of the diet, filed for bankruptcy today. It is in debt for 300 million dollars. The company greatly expanded over the past few years when the hype over low carb diets was at it's frenzied highest. Earlier this year I predicted that this diet fad was in its waning moments. I also wrote about some of the theory behind the diet. Basically, all diets will fail within a couple of years. It is much more difficult to keep off weight than to lose it. Maybe Krispy Kreme donuts will now make a comeback.

Echinacea

2005-07-28T18:07:00.000-04:00

The herbal supplement industry is a multi-billion dollar juggernaut. It's nice to see some of their outlandish claims finally get put to the test. The New England Journal of Medicine today reports that echinacea ( E. angustifolia root) has no effect for curing the common cold. There is a nice story in the New York Times. I'm sure adherents that swear by it (like my parents) will continue to take it but perhaps this study will put a little dent into sales.

The study took 437 volunteers, challenged them with the cold virus and randomly assigned them with pretreatment, treatment or placebo. The result was that there was no evidence that any form of treatment with echinacea had any significant effect in combating the cold or alleviating its symptoms.

Phosphorus and nitrogen

2005-07-24T10:26:00.000-04:00

The six most important elements for life are carbon, oxygen, hydrogen, nitrogen, sulfur and phosphorus. The abundances of these elements in biomass approximately mirror what is found in the earth's crust except for phosphorus which is about 6 times more abundant in biomass. However, the abundance of phosphorus in the ocean is the same as that found in biomass. This is no accident. Oceanographer Alfred Redfield found 70 years ago that the nitrogen to phosphorus ratio was 16:1 in both sea plankton and the ocean. He noted that this was not a coincidence but that the plankton was setting the ratio of the ocean. There is no intrinsic need for this ratio as plankton grown in laboratory conditions can exhibit a wide range.

The phosphorus in the ocean comes from the weathering of rocks on land. It is sequestered by oceanic life forms like plankton and then precipitates to the ocean floor when these organisms die. The availability of phosphorus sets the limit to how much life can be sustained by the ocean. This then sets the balance between oxygen and carbon dioxide in the oceans which in turn affects the balance of carbon dioxide in the atmosphere.

Now, some have argued that the increase of carbon dioxide in the atmosphere will lead to more vegetation which will counter the growth in CO2. However, more vegetation can grow only if it can acquire enough phosphorus and nitrogen. Although the atmosphere is 78% nitrogen, plants, other than legumes, cannot utilize it. They must obtain their nitrogen from the soil which mostly comes from animal waste or decaying biomass. Land animals that eat fish will transfer some nitrogen from the ocean back to the earth. The bottom line is that life on earth is precariously balanced and we really have no idea what will happen when we begin to perturb the system.

Modern Living

2005-07-19T18:20:00.000-04:00

The June issue of Smithsonian Magazine has some interesting numbers:

The List: 1970 Price Index

Gasoline $.36/gallon
Median Income $8,734/year
Median Rent $108/month
Median Home $17,000
Bacon $.97/pound
Eggs $.51/dozen
Bread $.24/loaf
Vitamin D Milk $1.14/gallon
First-Class Postage Stamp $.06
Harvard College Tuition $2,600/year

Compared to 2005, the cost of food has changed surprisingly little. The price for eggs and milk have only doubled over the last 35 years while Harvard's tuition and the cost of housing has gone up by more than a factor of ten. You only have to walk down the streets of any large American city to realize that getting enough food is no longer a major problem. The problem these days is finding affordable housing and putting your kids through college.

According to a February article in Amber Waves (a USDA publication):

Between 1952 and 2003, the ratio of food prices to the price of all other goods has fallen by 12 percent. The drop is even more dramatic if you factor in `quality improvements'—the reduced time cost of acquiring and preparing food (convenience), greater variety, and omnipresent restaurants and vending machines.Foods that once were available only seasonally are now available year-round. Advances in food processing and packaging have introduced a multitude of ready-to-eat foods, available virtually anywhere and at any time.

Harvard University's David Cutler, Edward Glaser, and Jesse Shapiro have suggested that the increase in food consumption prompted by the falling time cost of food is the major cause behind the surge in obesity since 1980. They note: "Technological innovations—including vacuum packing, improved preservatives, deep freezing, artificial flavors, and microwaves—have enabled food manufacturers to cook food centrally and ship it to consumers for rapid consumption. In 1965, a married woman who didn't work spent over two hours per day cooking and cleaning up from meals. In 1995, the same tasks took less than half the time."

Face Cells

2005-07-17T18:39:00.000-04:00

The June 23, 2005 issue of Nature reports work by Christof Koch and colleagues on the existence of "face recognition" cells in the hippocampus of the brain. This paper got a lot of play in the popular press because some of the cells only responded to famous people such as Halle Berry. The group found that the cells were highly selective to various views of a given person but not to another person. I think this work confirms some current theories of memory (see for example McClelland et al. Psychological Review, 102:419 (1995)). It's also more proof that there isn't much difference between humans and other mammals.

It is known that cells in the hippocampus in the rat code for spatial location in the same way. A given cell will only fire when a rat runs through a given spatial location in a given environment. When the environment changes, that same cell will then code for a completely different location. Location is important to a rat, just as the recognition of people is important to humans.

When the hippocampus is removed, humans can no longer form long term memories. They can remember things as long as they pay attention to it but once they lose their train of thought, the memory is completely gone. It is thus thought that the hippocampus is a form of mid-term memory that stores lots of information that is then slowly uploaded to the cortex for longer term storage.

It's useful to have different memory systems for different time scales because every time you remember something new you run the risk of erasing something old. One way out of this conundrum is to separate long term memory from short term memory. Simplistically, your hippocampus would store whatever information comes in and indiscriminately overwrite old information. Then slowly over time, the hippocampus would upload information to the temporal cortex (perhaps during dreams) which would update its synapses in a controlled fashion making sure not to erase important old memories.

What this paper shows is completely consistent with this idea. From theoretical work on associative memory, we know that the capacity of any neural network is limited by how correlated the stored patterns are with each other. The more correlated the patterns, the more likely they are to interfere. Thus, one way to make sure you don't overwrite old memories is to make sure the input patterns are orthogonal. The hippocampus may serve this purpose. A very sparse code, where only a few neurons encode a given concept (like Halle Berry), automatically orthogonalizes the patterns representing given memories presented to the higher cortical areas.

A sparse code is not robust because if you knock out that particular neuron you lose the memory it coded. A more robust code would be a population code where a large group of neurons encodes a given concept. The problem with this type of memory is that it's hard to train a network. So the way to overcome the trade-off between robustness and speed is to have a fast but fragile system (hippocampus) feed inputs to a slow but robust system (temporal cortex).

It is known that inferotemporal (IT) cortex of monkeys also respond to faces among many other percepts and that a given cell in IT will respond to a wide variety of images. So, if they ever get a chance to implant electrodes in the temporal cortex of humans, I'm sure they'll find similarly behaving cells.

Value

2005-05-25T18:04:00.000-04:00

I am finally starting to recover enough from my recent wedding and honeymoon to start posting again. The June 2005 issue of Technology Review covers the topic of Intellectual Property. There is a debate between Lawrence Lessig who believes that new laws governing copyright known as "digital rights management" (DRM), will undermine creativity and culture and Richard Epstein who thinks we need strong copyright laws to protect creators who add value to society.

I think this issue revolves on how we should value things in our society. The music and film industry is very worried about digital piracy. CD sales have gone down since illegal downloading of music became prevalent. I think that it will be impossible to fully stop it. Will this be the end of music? No, but I think it will be the end of powerful record companies who restrict the supply of recording artists so the prices can remain high. The argument is that making a record is expensive and they prescreen for quality. However, with inexpensive recording software like Apple's GarageBand and the internet, anyone can make a record now. The old model will become obsolete.

I can see two models for how artists might support themselves in the future. One is that they distribute their music for free or for a low fee and make money on concerts. The Grateful Dead has used this model for years. Classical and jazz musicians basically make their money from performances. The second is that we could re-establish a system of patronage for the arts. Artists would be paid a salary and their music would be given away for free. This last model is essentially the model for science. Scientists have jobs in research labs or universities and their work is given away for free.

In either model, the millionaire recording artist will become extinct. However, this won't change the lives of that many people. In the current system, only a handful of lucky people "make it" and become incredibly rich while everyone else waits on tables. What separates the people often has little to do with talent. It always seemed rather unfair to me that the world's greatest harmonica player is probably just getting by while a well marketed, lip synching pop star with little talent makes millions.

The motion picture industry does have an argument that films cost a lot to make and thus they need to be compensated. However, I think people will always want to see movies on the big screen. VCR's and DVD's have not reduced ticket sales. What will likely happen here is that more and more pictures will become digital and movie stars may become obsolete. Actors may only be necessary for live theatre.

Even in this world, I'm sure there will still be those that rise to be top and become stars. They may not be as well compensated as they are today but they will be as popular and probably a lot happier.

Suspended animation

2005-04-27T22:26:00.000-04:00

In the April 22 issue of Science, a group from the Fred Hutchinson Cancer Center reports that a state of suspended animation can be achieved in (nonhibernating) mice when administered hydrogen sulfide (H2S). H2S is a reversible inhibitor of oxidative phosporylation. It is known that this induces hibernation in some animals.

When the mice were exposed to 80 ppm of H2S, their oxygen consumption dropped by 50% in the first 5 minutes. After 6 hours, their metabolic rate dropped by 90% and the core body temperature reached 15 degrees Celsius where the ambient temperature was 13 degrees. When the mice were returned to room air and temperature, their metabolic rate and body temperature returned to normal.

If this works in humans, we may now have a means of reducing metabolic demand after traumatic injury or surgery. H2S may become a standard part of the repertoire of paramedics. I won't bother to dwell on the space travel implications.

Hydrogen sulfide is the gas emitted by volcanos and geysers responsible for the rotten egg smell. It is usually considered toxic but now you know that if you see someone looking lifeless at the edge of a volcano, make sure to pull them out because they may note be dead but just be in a state of suspended animation.

Mathematical Biology

2005-04-12T19:36:00.000-04:00

I am the first to admit that I'm not sure how much mathematics and theory has contributed to biology. Certainly the buzz is there. With the reams of data generated by the human genome project, biologists are begining to realize that they could use some help to understand all of this new data. The result has been a surge of available grant money and a flood of physicists, computer scientists and mathematicians into the field. (For the record, I made the jump over ten years ago when it was less fashionable.) I think it's safe to say that the jury is still out on whether or not the hype has been justified.

However, there has been one instance where mathematics has made a major difference and that is in the development of the triple cocktail treatment for HIV-AIDS. HIV is a particularly insidious virus because it attacks CD4 T cells of the immune system. However, it is rather slow acting. So often, when a person is infected with HIV, their virus load will remain low and CD4 counts will remain relatively high for a long period of time. Then, suddenly, the CD4 count will plummet and they will lapse into fully developed AIDS. It was first assumed that the virus replicated slowly and then accelerated at some point.

In the early 90's, David Ho and his group were testing treatments for HIV infection and decided that mathematically modeling the virus dynamics may give clues as to what was really happening. So they called in Los Alamos biological physicists Alan Perelson and Avidan Neumann (who is currently visiting our lab at NIH) to see if anything could be inferred about the virus. They used simple models of just a few ordinary differential equations to fit to the virus load during perturbation experiments where a potent protease inhibitor was administered.

Their simple model showed that the virus was far more active than previously believed. During the quiet phase where virus loads were low, the virus was actually replicating very rapidly but the immune system was running at high speed to compensate. Full blown AIDS developed when the immune system wore out and could no longer keep up with the virus. The implication was that any anti-viral treatment that targeted a single specific mechanism would fail because the virus would quickly evolve a defense. Thus the triple cocktail was invented. The virus would then need to evolve three separate defenses and this was difficult enough to keep it at bay. The results were published in two deservedly celebrated papers - the first in Nature in 1995 and the second in Science in 1996. I think their achievement represents the best example of how theoretical ideas can be useful in biology.

A new new world order

2005-04-06T19:11:00.000-04:00

An excerpt from Thomas Friedman's book "The World Is Flat: A Brief History of the Twenty-First Century" appeared in last Sunday's New York Times magazine. The premise is that the information age has truly arrived and now people all over the world can compete on equal terms. Steve Hsu summarizes the idea in his blog. The tone of Friedman's excerpt and Hsu's posts is that the Chinese, the Indians, and the Russians are coming and we better get ready for the new competition. America's dominance over the world is beginning to decline and if we don't recognize it now our standard of living will fall with it as our wealth starts to diffuse across the globe.

I certainly believe this is happening and it is unavoidable. Improving our educational system or motivating our citizens won't solve the real problem and that is the US only constitutes five percent of the world's population and if life were fair, it should have only five percent of the wealth. Even given that life is not fair, having only five percent of the world's population means that we only have five percent of the brightest, most innovative, and most energetic people to create the wealth for the future. Eventually, things will equalize. It's a battle we just can't win.

So what should we do? One solution is to bring the rest of the world up to our economic level. Unfortunately, I don't think a world where everyone lives like an American is sustainable. See my estimate of energy use from a previous post. A possible scenario is that as the world catches up and begins to compete for scarcer and scarcer resources, the pecking order will be sorted out through military means. I truly hope we are sane enough to avoid that fate. My deluded, quasi-utopian vision, is that we all scale back and share. The pessimistic nihilist in me says that won't happen in my lifetime.

The North Pole

2005-04-05T21:37:00.000-04:00

The earth's magnetic field is generated by the motion of molten iron in the earth's core. The combination of convection and coriolis forces generates the right set of currents to establish a dipole field with the north and south poles approximately in line with the rotational north and south poles. We partially owe our existence to this magnetic field because it provides a shield against charged particles from the solar wind. Without it, we would be subject to ionizing radiation and may also lose our atmosphere.

Neither Mars nor Venus has a significant magnetic field and we're not fully sure why. Mars once had a thick atmosphere of CO2 that may have been blown away by the solar wind. It could be that the molten iron has solidified or that the pattern of flow no longer supports a magnetic field. In any case, I think the lack of a field on Mars should make us less secure that we'll always have ours.

The earth's dipole flips every 250,000 years on average. We're not exactly sure why but some recent magnetohydrodynamic simulations of the geodynamo have shown examples of field reversals due to instabilities in the turbulent flow. It's been 780,000 years since the last reversal so we may be due for another one soon. It will take approximately 4,000 to 10,000 years for a reversal to take place. During the transition, the magnetic field may lose its intensity as well as its dipole structure which may have implications for life on the earth.

Another consequence of the turbulent geodynamic flow is that the north pole is in constant motion. It usually moves about ten kilometres a year but recently it has been moving forty! If it keeps moving at this rate, in about fifty years it will leave Canada and reach Siberia. This increased speed of drift of the north pole may be nothing more than natural random variations but it definitely makes me worry just a little.

Engineers in Government

2005-04-01T17:10:00.000-05:00

Ever wonder why science and engineering policy doesn't make much sense. Here's an article from this month's Technology Review that may give a reason why.

Engineers and Political Power
by Ed Tenner, April 2005

In the united states, engineers don’t rule. According to a Congressional Quarterly survey of the 109th Congress, there are just four engineers in the House and one in the Senate. When the engineering specialties in the 2004–2005 Statistical Abstract of the United States are combined, there are 2.12 million engineers in the U.S. versus 952,000 lawyers and 819,000 doctors; yet 10 physicians now sit in the House and two in the Senate, and CQ lists 160 representatives and 58 senators with legal backgrounds.

One explanation for those discrepancies is that rapid technological change makes it hard for engineers to return from political office to professional life. In a 1992 interview with Technology Review, John H. Sununu, President George H. W. Bush’s chief of staff, acknowledged that as a consulting mechanical engineer, he was lagging ten years behind the field. Physicians, however, face equally great problems keeping up with the latest research, and by entering public service, they often forgo even greater potential income.

Another theory is that engineers are self-selected for social distance. Sylvia Kraemer is an intellectual historian who became a senior NASA official and interviewed 51 colleagues for her insightful study NASA Engineers and the Age of Apollo. She found that lab engineers and those promoted into management endorsed the reputation of awkwardness. A manager declared that most engineers "wouldn’t recognize an emotion if it hit them in the face." One rocket engineer flatly acknowledged, "I related to things."

This is an old American stereotype. In The Engineers and the Price System, the maverick economist Thorstein Veblen, championing what was later called technocracy, wrote that the public considered engineers a "somewhat fantastic brotherhood of overspecialized cranks, not to be trusted out of sight except under the restraining hand of safe and sane businessmen." He added, "Nor are the technicians themselves in the habit of taking a greatly different view of their own case."

But in many other cultures, especially in Eastern Europe, Asia, and the Middle East, engineers have been in the thick of power. The’ve been prominent in Marxist movements, such as the brief Hungarian Communist revolution of 1919. They became influential enough in the early Soviet Union that Stalin directed one of his first purges against them. Later, scientists and engineers were put to work in the gulags’ special research prisons, the sharashkas. After Stalin’s death, engineering degrees became desirable credentials for the politically ambitious. As the historian Kendall Bailes wrote in 1974, "What lawyers and businessmen are in the American political system—the major professional groups from which most politicians and policymakers are recruited—men with engineering backgrounds have become to a large extent in the Soviet Union."

In 2004, almost all two dozen members of China’s ruling Politburo had engineering degrees, including all nine members of the Politburo’s Standing Committee. In the Middle East, prominent engineers fill the political spectrum, from former president Süleyman Demirel of Turkey to the members of the Society of Muslim Engineers, pillars of the ayatollahs’ Iran, to the late secular nationalist Yasser Arafat. In many countries, engineering appeals -to the civic minded. On the other hand, disaffected young men recruited in European engineering schools were prominent among the September 11 hijackers. As R. Scott Appleby and Martin E. Marty observe in Foreign Affairs, "fundamentalists tend to read scriptures [as] engineers read blueprints—as a prosaic set of instructions and specifications." Civil engineer Osama bin Laden surely did.

Globally, then, the unpolitical Anglo-American nerd is the exception. The argument that gained credence in 19th-century France and was echoed in other regimes is that a state must be guided by a scientific and technological elite. Two forces kept that notion from taking hold in the United States. The first was American suspicion of central government. The second was industry’s appetite for engineers; at the turn of the 20th century, U.S. companies fearing manpower shortages resisted attempts to make elite postgraduate degrees the norm for engineers, as they were becoming for lawyers, doctors, and executives. So engineers in this country continue to design and implement everything but our laws.

Krakatoa and the Tsunami of 2004

2005-03-30T20:24:00.000-05:00

The southern coast of Indonesia is one of the most active tectonic zones in the world because the Indian plate is subducting beneath the Burma microplate. This was borne out tragically in the Aceh-Andaman earthquake and Tsunami of 2004. In this week's Nature, Stein and Okal argue that the magnitude of that earthquake was 2.5 times greater than previously thought and should have been a 9.3 instead of a 9.0. Just this past Monday, another earthquake of 8.7 magnitude struck southern Sumatra. Fortunately, this one was in shallower water so much less water was displaced and thus no Tsunami was generated.

It was over a century ago, on August 27, 1883, that the volcano Krakatoa exploded nearby. The volcano was part of an island consisting of three main cones in the straight between Sumatra and Java. It stood nearly 800 metres high and had a diameter of 15 kilometres. Two of the cones were completely blown apart in the eruption. The Tsunami generated was estimated to be 40 metres high although it was of short wavelength so it didn't travel very far. It did kill 36,000 people in southern Sumatra and western Java. It carried a steamship 2 miles inland. The explosion was so loud it was heard as far away as Perth Australia and the atmospheric shock wave travelled around the world seven times. There were brilliant red sunsets for five years afterwards due to the 21 cubic kilometres of material thrown up.

The eruption was extremely well documented by the scientists of the day. The field of volcanology was invented to understand why the eruption was so violent. The current theory is that Krakatoa had been dormant for 200 years before the 1883 eruption and this allowed the magma to cool inside the volcano. About four months before the August eruption, Krakatoa became active and this cool magma escaped in a series of minor eruptions. It was then replaced by hot magma from below. The entering hot new magma heated up the remaining cool old magma releasing gases and built up tremendous pressure which was released in a massive explosion equivalent to 21,000 megatons of TNT. The amazing thing is that a new volcano has almost entirely replaced the old one. In just a hundred years, Anak Krakatau (child of Krakatoa) has risen to a height of 300 metres and is still growing. It is not expected to explode like it's parent in the near future but given recent events one never knows.

Failure

2005-03-28T19:30:00.000-05:00

Scott Sandage, an associate professor in history at Carnegie Mellon University, has recently penned a book entitled - Born Losers - A History of Failure in America. An interesting fact that Sandage points out is that the concept of being a failure as a human being did not exist until the late nineteenth century. Prior to that, failure was only used to imply bankruptcy, a circumstance that did not necessarily imply any personal deficiency. Unlike today where a lack of ambition and vision is synonymous with being a loser, it was perfectly fine to be satisfied with one's lot. Social climbers, like Becky Sharp in Vanity Fair, were considered crass.

Sandage finds that the Civil War may have been a turning point. Prior to the war, America divided itself into citizens and slaves. However, after the war, the division changed to winners and losers. It was also a time where the robber baron class arose. These were usually self-made men such as Carnegie, Mellon and Rockefeller who rose to prominence in a burgeoning industries such as steel, banking and oil. They also defined success in American as earned (plundered?) wealth. This attitude is probably even more apparent today. It is only in America (until recently perhaps) that being called aggressive was a compliment. I'm not suggesting that we all return to a society without ambition but perhaps defining ourselves exclusively in terms of our achievements and finances may not be the most satisfying way to live.

Jurassic Park

2005-03-25T17:30:00.000-05:00

The possibility of resurrecting extinct animals some day in the future may not be as far fetched as previously thought. A paper in this week's Science reports that soft tissue from a 70 million year old Tyrannosaurus Rex was found in Montana. The specimen contained blood vessels and possibly cells. Finding pieces of DNA is a long shot but remains a remote possibility.

In fact, extracting DNA from extinct animals is becomng more common A recent article in PLOS Biology reviews the many ways that ancient DNA has been used to ask questions about population genetics in the past. It is probably not surprising that there are signs that our current descriptions of the evolutionary process based on the fossil record and modern DNA is far too simple. Since DNA is a fragile molecule, most studies are limited to DNA less than 60,000 years old. However, that's still old enough to get some interesting information. The hardest thing to prove is that the sample is not contaminated with modern DNA.

The X chromosome

2005-03-17T20:45:00.000-05:00

This week's Nature has a picture of the X chromosome on the cover. The issue has two articles on the topic. The first contains an analysis of the recently completed sequence. As I wrote before on the Y chromosome, women have two copies of the X. Previously, it had been thought that one was suppressed and the other is expressed. In the second paper, it is shown that about 15% of genes are still expressed on the inactive chromosome. The same genes on the active chromosome are also expressed so this could mean that more of the protein is produced. This could provide a basis for the differences between individual women and for this month's hot button topic - the differences between women and men.

Ironically, the X chromosome may actually be more important for men than for women. Since men only have one X, any advantageous mutation for men will we be selected for strongly even if it is somewhat deleterious for women. Conversely, a deleterious mutation will be strongly selected against. It has been proposed by Horst Hameisterat of the University of Ulm in Germany that females may be responsible for intelligence in humans. His group proposes that early females had a preference for intelligence and if the genes for superior intelligence and the preference for intelligence are found near each other then they would be selected together and this could lead to an instability because both lead to enhanced survival. It had been known for some time that many of the genes related to mental retardation are found on the X and now it is confirmed that there are many crucial genes on the X for neural development. Hence, the X could be what makes humans human.

Steroids

2005-03-15T17:54:00.000-05:00

Congress has subpoenaed seven baseball players to testify about steroid use in major league baseball. This is a result of the recent BALCO scandal that implicated sprinter Marion Jones, Barry Bonds, Jason Giambi among others and the recent publication of Jose Canseco's book alleging steroid use by several stars including Mark McGwire. Given the current debates on our fiscal crisis, the Iraqi war, social security, the Arctic National Wildlife Refuge, North Korea, outsourcing and so on, aren't there more important things for Congress to do? I love sports and baseball as much as the other guy but come on it's just a game.

I believe that international sporting bodies must accept that medical enhancement will only increase in the future and there is virtually no chance to completely stop it. If they ever get gene therapy to work we'll be in real trouble. I think testing for cheating is the wrong strategy. What we will eventually need to do is to set physiological standards that athletes cannot exceed. So everyone is allowed up to some level of testosterone, human growth hormone, myostatin, red blood cells and so forth. You can do whatever you want to get to these standards but you can't go beyond. I suppose those that exceed the standards "naturally" would be able to petition the governing body to obtain an exemption. I agree that this would take away something from sport but in some sense it also levels the playing field. After all, isn't being born with better genes an advantage?

Dyson's take on evolution

2005-03-13T20:51:00.000-05:00

Freeman Dyson always has something interesting to say on most topics. In the March issue of Technology Review he summarizes Carl Woese's idea about the end of Darwinian evolution due to bioengineering. I don't think we've stopped evolving since we're still battling microbes but the thought is intriguing and terrifying.

Carl Woese published a provocative and illuminating article, A New Biology for a New Century, in the June 2004 issue of Microbiology and Molecular Biology Reviews. His main theme is the obsolescence of reductionist biology as it has been practiced for the last hundred years, and the need for a new biology based on communities and ecosystems rather than on genes and molecules. He also raises another profoundly important question: when did Darwinian evolution begin? By Darwinian evolution he means evolution as Darwin himself understood it, based on the intense competition for survival among noninterbreeding species. He presents evidence that Darwinian evolution did not go back to the beginning of life. In early times, the process that he calls horizontal gene transfer, the sharing of genes between unrelated species, was prevalent. It becomes more prevalent the further back you go in time. Carl Woese is the worlds greatest expert in the field of microbial taxonomy. Whatever he writes, even in a speculative vein, is to be taken seriously.

Woese is postulating a golden age of pre-Darwinian life, during which horizontal gene transfer was universal and separate species did not exist. Life was then a community of cells of various kinds, sharing their genetic information so that clever chemical tricks and catalytic processes invented by one creature could be inherited by all of them. Evolution was a communal affair, the whole community advancing in metabolic and reproductive efficiency as the genes of the most efficient cells were shared. But then, one evil day, a cell resembling a primitive bacterium happened to find itself one jump ahead of its neighbors in efficiency. That cell separated itself from the community and refused to share. Its offspring became the first species. With its superior efficiency, it continued to prosper and to evolve separately. Some millions of years later, another cell separated itself from the community and became another species. And so it went on, until all life was divided into species.

The basic biochemical machinery of life evolved rapidly during the few hundred million years that preceded the Darwinian era and changed very little in the following two billion years of microbial evolution. Darwinian evolution is slow because individual species, once established, evolve very little. Darwinian evolution requires species to become extinct so that new species can replace them. Three innovations helped to speed up the pace of evolution in the later stages of the Darwinian era. The first was sex, which is a form of horizontal gene transfer within species. The second innovation was multicellular organization, which opened up a whole new world of form and function. The third was brains, which opened a new world of cordinated sensation and action, culminating in the evolution of eyes and hands. All through the Darwinian era, occasional mass extinctions helped to open opportunities for new evolutionary ventures.

Now, after some three billion years, the Darwinian era is over. The epoch of species competition came to an end about 10 thousand years ago when a single species, Homo sapiens, began to dominate and reorganize the biosphere. Since that time, cultural evolution has replaced biological evolution as the driving force of change. Cultural evolution is not Darwinian. Cultures spread by horizontal transfer of ideas more than by genetic inheritance. Cultural evolution is running a thousand times faster than Darwinian evolution, taking us into a new era of cultural interdependence that we call globalization. And now, in the last 30 years, Homo sapiens has revived the ancient pre-Darwinian practice of horizontal gene transfer, moving genes easily from microbes to plants and animals, blurring the boundaries between species. We are moving rapidly into the post-Darwinian era, when species will no longer exist, and the evolution of life will again be communal.

In the post-Darwinian era, biotechnology will be domesticated. There will be do-it-yourself kits for gardeners, who will use gene transfer to breed new varieties of roses and orchids. Also, biotech games for children, played with real eggs and seeds rather than with images on a screen. Genetic engineering, once it gets into the hands of the general public, will give us an explosion of biodiversity. Designing genomes will be a new art form, as creative as painting or sculpture. Few of the new creations will be masterpieces, but all will bring joy to their creators and diversity to our fauna and flora

Cold Spring

2005-03-10T12:02:00.001-05:00

The weather has not been kind for us on the east coast these past few weeks. After a reasonably mild winter, late winter/early spring has been filled with snow, cold, and more snow. There may be a silver lining to this though. Here is my completely unsubstantiated account of North American weather patterns.

Anyone who watches the weather channel knows that when it is cold on the east it's because a cold front has moved in from Canada. I heard it being called an Alberta Clipper this morning. The atmosphere gets warmed or cooled primarily through contact with the ground since solar radiation mostly passes through the air and is absorbed by the ground. In the winter and spring, the air over north western Canada is cold while the air over the Gulf of Mexico and the southern US is mostly warm. Thus a pressure gradient can arise with high pressure in Canada and low pressure in the southern US. This causes air to flow from north to south.

Now all that air coming down must go somewhere so it makes it's way up the Atlantic coast and returns to the arctic. Along the way, it picks up warmth and moisture which often ends up as snow in New England. Hence, the more cold air there is coming from the north, the more warm air there will be returning to the arctic. This should then lessen the temperature and pressure gradient between the north and the south. So, perhaps a cold late winter may actually lead to a milder late spring.

Hans Bethe

2005-03-07T18:09:00.000-05:00

Yesterday marked the passing of Hans Bethe, the last of the great physicists from the Manhattan Project. I had the privilege of seeing Bethe speak twice when I was a graduate student at MIT in the late 1980's. Bethe was in his early eighties at the time and was still sharp as a tack. One talk was on neutrino oscillations and the other was on disarmament policy. Both talks were well thought out and exhibited the clear thinking that marked Bethe's career.

The amazing thing about Bethe was that he remained relevant for much of the twentieth century. Bethe was held in the highest regard by both his peers and the political establishment throughout his life. He was Sommerfeld's graduate student. His Nobel Prize work on energy production in stars was done in 1938. He played an important role in the development of QED and was Richard Feynman's mentor. He was instrumental in pushing for arms limitation treaties and derailing the Star Wars space-based anti-missile defense during the Reagan administration. He and John Bahcall wrote a landmark paper outlining how the solar neutrino problem was finally solved in 1990. He was a true giant among giants.

The Y chromosome

2005-03-04T19:33:00.000-05:00

DNA is packaged into discrete units called chromosomes. The chromosomes come in pairs with one from the mother and one from the father. There does not seem to be any rhyme or reason to the number of chromosomes a species may have. For example, humans have 23 pairs, the other three great apes have 24 pairs, mice have 21 and horses have 32. It appears that during the course of evolution chromosomes can undergo fission and fusion or be lost altogether.

It is not necessary for two species to have the same number of chromosomes to produce fertile offspring. For example, Przewalski's wild horse has 33 pairs yet when crossed with a domesticated horse with 32, the offspring are fertile and can mate with other domesticated horses (Chandley AC, Short RV, Allen WR, J Reprod Fertil Suppl 23:356-70, 1975). What happens is that two of the chromosomal pairs from the wild horse fuse in a process called Robertsonian translocation. This can happen in humans as well and some people are walking around with 45 chromosomes.

The X and Y chromosomes determine sex. Females have two X chromosomes and males have an X and a Y. While female offspring will obtain an X chromosome from each of their parents, male offspring will always inherit their father's Y chromosome. So mutations in the Y chromosome will always get passed on and thus can accumulate. Jennifer Graves of the Australian National University, believes that the Y chromosome is rapidly shrinking and may disappear altogether. The basis of her claim is from comparing the genomes of various mammals. For example, she has found that kangaroos diverged from humans 180 million years ago and the platypus 210 million years ago. These mammals also have analogues to human sex chromosomes. In the last 300 million years the Y has lost most of its original genes. She estimates that the remaining genes have about 10 million more years to go. Hence to avoid extinction, humans may need to evolve into a new hominid species.

There is some dissent though. David Page of the Whitehead Institute has discovered 78 genes on the Y (about twice as much as previously believed) and there could be more. He also found palindromic sequences so there could be some redundency. The Y might be able to repair errors by taking hairpin configurations. So are we men on our last legs? I guess we'll have to wait to see.

Out on the tail

2005-03-02T17:59:00.000-05:00

As the world's population gets larger, should we expect more Newtons, Mozarts and Tiger Woods or should we expect fewer? By this, I mean should the gap between the best in a field and the second best, be larger than the gap between the second best and the rest? If you presume that the aptitude level of people for some enterprise is normally distributed then the answer is no. The probability to be between n standard deviations and n+1 standard deviations above the mean decreases exponentially. As you go out onto the tail, the probability of someone being a standard deviation better than you is decreasing very quickly. Thus we should see a lot of people bunched near the top.

So why is it that we have Tiger Woods? Tiger is dominating golf like no one since perhaps Bobby Jones in the twenties and Ben Hogan in the early fifties. He was more dominant over a stretch of five years than Jack Nicklaus ever was although his dominance declined slightly last year. There are many more golfers now then there were fifty years ago so if anything we should see a much more competitive environment. Except for Woods and perhaps Vijay Singh, this is true as there is very little separating the rest. Dominance is not just limited to golf. Ten years ago we had Michael Jordan in basketball and Wayne Gretzky in hockey. Right now we have Barry Bonds in baseball and Lance Armstrong in cycling. These people are dominating or have dominated their sports as much or more than anyone else before.

Part of the reason is surely that a normal distribution is a bad approximation for the tail. Anyone in finance certainly knows this. The central limit theorem does not converge uniformly over the entire range, so even though the bulk is well described by a normal distribution, the tail can still experience low number statistics. Thus as the population increases, more of the tail gets sampled and that could actually increase the probability of rare events not decrease it as would be expected. So my prediction is that we will see more dominant athletes in the future.

Now what about physics and music? By this argument we should see more anomalies in these areas as well. However, it is not clear who is the Newton of our day. Ed Witten maybe? I think the reason is that there are fewer objective measures of mastery in these areas as compared to sports. A lot of what leads to great impact in physics is the choice of problem. In this case, there is a greater element of luck. One could be way out on the tail in terms of sheer brilliance (if that can even be measured) but not have a great impact if they worked in a dead end field. In this case, physics may suffer from low number statistics in the bulk and the tail so there will be no correlation of dominance with population size.

Music has a similar problem. Who in the 20th century compared to Mozart? Diatonic Western classical music reached its acme with Mahler. After that, composers either became atonal or "popular". The 20th century was the century of performers. Perhaps if Rubenstein, Heifetz, or Yo Yo Ma was born in the 18th century they would have been great composers. The playing field changed in the 20th century so the question of dominance became more difficult to answer. In 200 years, we may look back at the 20th century and consider a Jazz or Rock musician to be the Mozart of the day.