Feng Li's blog

Sweden and the euro: Out and happy

2011-12-12T12:55:00.001+01:00

Sent to you by Feng via Google Reader:

Sweden and the euro: Out and happy

WHEN Swedes voted in 2003 on whether or not to join the euro, most political and business leaders were strongly in favour. Today even the euro's supporters are grateful to the 56% of voters who said no. As worried investors push up yields on government bonds right across the euro zone, yields on Swedish ten-year bonds have fallen to 1.7%, more than half a point below German Bunds.Anders Borg, the finance minister, still thinks that in the long run Sweden should join the euro. But he seems happy to be out for now. Fears that Sweden, a small export-based economy, might suffer if it kept the krona were a strong pro-euro argument in 2003. Yet Mr Borg says that Sweden has gained something from standing aside. "Being an outsider, you must make sure your competitiveness and public finances are in order. We have had to impose on ourselves a self-discipline that euro countries did not feel they needed. If you know the winter will be very cold, you have to ensure the house has been built well. Otherwise you will freeze."In Sweden this translates into tight fiscal policy, a budget surplus of some 0.1% of GDP and a...

Things you can do from here:

Subscribe to Economist using Google Reader
Get started using Google Reader to easily keep up with all your favorite sites

Terence’s Stuff: Knowing

2011-11-17T19:16:00.001+01:00

As a Chinese, I really like the Confucius' saying in the end.

Sent to you by Feng via Google Reader:

Terence’s Stuff: Knowing

via IMS Bulletin by Editor on 11/17/11

On and off through my career, I've encountered people—usually students, occasionally faculty members—who didn't know what they were talking about. Of course that was just how it seemed to me at the time, and I may have been wrong. However, I think I'm able to work out whether someone talking the talk has also walked the walk, at least in the areas with which I am moderately familiar. As with Lieutenant Columbo, a few "dumb" questions are usually enough.

When I was a student, there was always a very great deal to learn, much more than I could ever cope with. I accepted the fact that I would never be on top of everything, and aimed for a more or less adequate grasp of all, rather than a really thorough understanding of part of the syllabus. Perhaps exams encourage this approach, and no great harm was done, but it now seems as though I was sacrificing quality for quantity.
When I started doing research, with no syllabus saying I needed to learn this and I didn't need to learn that, and no exams, and I alone decided what I learned, I needed a different approach. I had met the concept of arguing "from first principles" in high school, that is, developing a line of reasoning from the axioms to the conclusion, assuming nothing. This became my benchmark for knowing what I knew, and it still is. As a result, my research has been limited to things I can explain from scratch. I will insert a term into a discussion only if I can define and illustrate it; I'll use a theorem in an explanation only if I can prove it on demand. I try to read all the papers I cite in my own papers, but even here I must hedge a little: what does it mean to say, "I've read that paper"? It's exactly the same issue: can I explain the contents of that paper from first principles? With that requirement, I have actually read relatively few papers, far fewer than I've cited. I am reluctant to use quotes unless I have read them 'in situ,' and ideally all the surrounding text as well. (I find it hard to adhere to this rule, as there are so many wonderful quotes out there, met as quotes, not in context.)
While I was working on my PhD I encountered a new phenomenon: material that I could never know, regardless of the amount of time I spent on it. Mathematics is cumulative: to know this, you need to know that, to know that, you need to know the other, and while this doesn't go on ad infinitum, the chain can be very long indeed. I became aware of many areas of mathematics that would remain beyond me forever. It is hard to put this assertion to the test, but after trying, I accepted it, even if as nothing more than a statement of my priorities. To paraphrase Niebuhr, "God grant me the serenity to accept the things I cannot know; courage to master the things I can; and wisdom to know the difference."
All that was about my knowing. As a teacher I have spent a lot of time helping others figure out what they know and don't know, and perhaps will never know. An approach that I often take when a student asks me a question is to go back to something more basic related to their question which they say they understand, and quiz them about that. Not infrequently, there's still a problem, and so we go further back, until we are on solid ground, usually not ad infinitum, but sometimes to a tabula rasa. Not long ago I asked a student to figure out a certain paper. At our next meeting I asked, "Have you understood it?" and the reply was, "Yes." I passed over a felt-tipped pen, pointed to my whiteboard and said, "Please explain how to get from line two to line three on page three." The student demurred, muttering something about thinking under pressure, and so I handed over a piece of paper and said, "Write out the explanation and give it to me later." I never did get that piece of paper back. It's important to be able to say, "I don't understand."
I realize that I'm sounding as though everything is black and white, although I do know it's not always clear. But often it is.
As Confucius says: "When you know a thing, to recognize that you know it, and when you do not know a thing, to recognize that you do not know it. That is knowledge." (Arthur Waley's translation; original text: 知之为知之，不知为不知，是知也。)

Things you can do from here:

Subscribe to IMS Bulletin using Google Reader
Get started using Google Reader to easily keep up with all your favorite sites

"Education For All"

2011-10-15T00:23:00.000+02:00

I will try to tell you some true stories that I have heard recently.

A friend of my who was the hopeless boy during his undergraduate studies. All most all of his courses were ranked as E (the lowest grade to pass the exam). Suddenly he found interesting in math and ended up with a Ph.D. degree this summer.
Another story I have heard is that he was a taxi driver for some ten years and decided to study again. He got his Ph.D. in statistics and now he is a university teacher.
A third person who is now at his retiring age. He is taking a master program in statistics seriously. As far as I know he has passed almost all the exams.

There are some more similar stories I knew about. All of those drive me to think out of loud: what does "all" mean in the phrase of "Education For All"? It probably should be all purposes, all-time and all possibilities.

A History of Bayes' Theorem

2011-09-04T19:55:00.001+02:00

Mattias pointed this to me in his twitter.

Sent to you by Feng via Google Reader:

A History of Bayes' Theorem

via Articles Tagged ‘bayes’ - Less Wrong on 8/29/11

Submitted by lukeprog 72 comments

Sometime during the 1740s, the Reverend Thomas Bayes made the ingenious discovery that bears his name but then mysteriously abandoned it. It was rediscovered independently by a different and far more renowned man, Pierre Simon Laplace, who gave it its modern mathematical form and scientific application — and then moved on to other methods. Although Bayes' rule drew the attention of the greatest statisticians of the twentieth century, some of them vilified both the method and its adherents, crushed it, and declared it dead. Yet at the same time, it solved practical questions that were unanswerable by any other means: the defenders of Captain Dreyfus used it to demonstrate his innocence; insurance actuaries used it to set rates; Alan Turing used it to decode the German Enigma cipher and arguably save the Allies from losing the Second World War; the U.S. Navy used it to search for a missing H-bomb and to locate Soviet subs; RAND Corporation used it to assess the likelihood of a nuclear accident; and Harvard and Chicago researchers used it to verify the authorship of the Federalist Papers. In discovering its value for science, many supporters underwent a near-religious conversion yet had to conceal their use of Bayes' rule and pretend they employed something else. It was not until the twenty-first century that the method lost its stigma and was widely and enthusiastically embraced.

So begins Sharon McGrayne's fun new book, The Theory That Would Not Die, a popular history of Bayes' Theorem. Instead of reviewing the book, I'll summarize some of its content below. I skip the details and many great stories from the book, for example the (Bayesian) search for a lost submarine that inspired Hunt for Red October. Also see McGrayne's Google Talk here. She will be speaking at the upcoming Singularity Summit, too, which you can register for here (price goes up after August 31st).

Origins

In the 1700s, when probability theory was just a whiff in the air, the English Reverend Thomas Bayes wanted to know how to infer causes from effects. He set up his working problem like this: How could he learn the probability of a future event occurring if he only knew how many times it had occurred or not occurred in the past?
He needed a number, and it was hard to decide which number to choose. In the end, his solution was to just guess and then improve his guess later as he gathered more information.
He used a thought experiment to illustrate the process. Imagine that Bayes has his back turned to a table, and he asks his assistant to drop a ball on the table. The table is such that the ball has just as much chance of landing at any one place on the table as anywhere else. Now Bayes has to figure out where the ball is, without looking.
He asks his assistant to throw another ball on the table and report whether it is to the left or the right of the first ball. If the new ball landed to the left of the first ball, then the first ball is more likely to be on the right side of the table than the left side. He asks his assistant to throw the second ball again. If it again lands to the left of the first ball, then the first ball is even more likely than before to be on the right side of the table. And so on.
Throw after throw, Bayes is able to narrow down the area in which the first ball probably sits. Each new piece of information constrains the area where the first ball probably is.
Bayes' system was: Initial Belief + New Data -> Improved Belief.
Or, as the terms came to be called: Prior + Likelihood of your new observation given competing hypotheses -> Posterior.
In each new round of belief updating, the most recent posterior becomes the prior for the new calculation.
There were two enduring criticisms to Bayes' system. First, mathematicians were horrified to see something as whimsical as a guess play a role in rigorous mathematics. Second, Bayes said that if he didn't know what guess to make, he'd just assign all possibilities equal probability to start. For most mathematicians, this problem of priors was insurmountable.
Bayes never published his discovery, but his friend Richard Price found it among his notes after Bayes' death in 1761, re-edited it, and published it. Unfortunately, virtually no one seems to have read the paper, and Bayes' method lay cold until the arrival of Laplace.

Laplace

By the late 18th century, Europe was awash in scientific data. Astronomers had observations made by the Chinese in 1100 BC, by the Greeks in 200 BC, by the Romans in AD 100, and by the Arabs in AD 1000. The data were not of equal reliability. How could scientists process all their observations and choose the best? Many astronomers simply averaged their three 'best' observations, but this was ad-hoc. The world needed a better way to handle all these data.
Pierre-Simon Laplace, a brilliant young mathematician, came to believe that probability theory held the key, and he independently rediscovered Bayes' mechanism and published it in 1774. Laplace stated the principle not with an equation, but in words: the probability of a cause (given an event) is proportional to the probability of the event (given its cause). And for the next 40 years, Laplace used, extended, clarified, and proved his new principle.
In 1781, Richard Price visited Paris, and word of Bayes' earlier discovery eventually reached Laplace. Laplace was now all the more confident that he was on the right track.
He needed to test his principle, so he turned to the largest data set available: birth records. A few people had noticed that slightly more boys than girls were born, and Laplace wanted to know if this was an anomalous or constant phenomenon. He began by applying equal probability to his hunches, and then updated his belief as he examined data sets from Paris, from London, from Naples, from St. Petersburg, and from rural areas in France. Later he even asked friends for birth data from Egypt and Central America. Finally, by 1812, he was almost certain that the birth of more boys than girls was "a general law for the human race."
Laplace's friend Bouvard used his method to calculate the masses of Jupiter and Saturn from a wide variety of observations. Laplace was so impressed that he offered his readers a famous bet: 11,000 to 1 odds that Bouvard's results for Saturn were within 1% of the correct answer, and a million to one odds for Jupiter. Nobody seems to have taken Laplace's bet, but today's technology confirms that Laplace should have won both bets.
Laplace used his principle on the issue of testimony, both in court and in the Bible, and made famous progress in astronomy. When asked by Napoleon who authored the heavens, Laplace replied that natural law could explain the behavior of the heavens. Napoleon asked why Laplace had failed to mention God in his book on the subject. Laplace replied: "Sire, I have no need of that hypothesis."
The answer became a symbol of the new science: the search for natural laws that produced phenomena without the need to call upon magic in the explanation.
And then, Laplace invented the central limit theorem, which let him handle almost any kind of data. He soon realized that where large amounts of data were available, both the Bayesian and the frequentist approaches (judging an event's probability by how frequently it occurs among many observations) to probability tended to produce the same results. (Only much later did scientists discover how wildly the two approaches can diverge even given lots of data.)
And so at age 62, Laplace — the world's first Bayesian — converted to frequentism, which he used for the remaining 16 years of his life.
...though he did finally realize what the general theorem for Bayes' method had to be:

P(C|E) = [ P(E|C)P_prior(C) ] / [ΣP(E|C')P_prior(C')

Which says that the probability of a hypothesis C given some evidence E equals our initial estimate of the probability times the probability of the evidence given the hypothesis C divided by the sum of the probabilities of the data in all possible hypotheses.
Basically, Laplace did all the hard work, and he deserves most of the honor for what we call Bayes' Theorem. But historical accidents happen, and the method is named after Bayes.

The Decline of Bayes' Theorem

Empowered by Laplace's central limit theorem, government officials were expected to collect statistics on all sorts of things: cholera victims, the chest sizes of soldiers, the number of Prussian officers killed by kicking horses, and so on. But the idea that probability quantifies our ignorance was gone, replaced by the idea that the new science could not allow for anything 'subjective'. John Stuart Mill denounced probability as "ignorance... coined into science."
By 1891, the Scottish mathematician George Chrystal urged: "[Laplace's principle] being dead, [it] should be decently buried out of sight, and not embalmed in text-books and examination papers... The indiscretions of great men should be quietly allowed to be forgotten."
And thus, Bayes' Theorem fell yet again in disuse... at least among theoreticians. A smattering of practitioners continued to find it useful.
Joseph Bertrand was convinced that Bayes' Theorem was the only way for artillery officers to correctly deal with a host of uncertainties about the enemies' location, air density, wind direction, and more. From 1890-1935, French and Russian artillery officers used Bertrand's Bayesian textbook to fire their weapons.
When the French Jew Alfred Dreyfus was falsely accused of having sold a letter to German military expert, France's famous mathematician Henri Poincaré was called to the stand. Poincaré was a frequentist, but when asked whether Dreyfus had written the letter, Poincaré invoked Bayes' Theorem as the only sensible way for a court of law to update a hypothesis with new evidence, and proclaimed that the prosecution's discussion of probability was nonsense. Dreyfus was still convicted, though his sentence was reduced, but the public was outraged and the president issued a pardon two weeks later.
Statisticians used Bayes' Theorem to set up a functioning Bell phone system, set of up the United States' first working social insurance system, and solve other problems.
Meanwhile, the biologist R.A. Fisher was pioneering new randomization methods, sampling theory, tests of significant, analyses of variance, and a variety of experimental designs. In 1925 he published his revolutionary manual, Statistical Methods of Research Workers. The success of the book enshrined frequentism and the standard statistical method.

Jeffreys

Even during its decline, a few people made progress on Bayesian theory. At about the same time, three men in three countries — Émile Borel, Frank Ramsey, and Bruno de Finetti — independently happened upon the same idea: knowledge is subjective, and we can quantify it with a bet. The amount we wager shows how strongly we believe something.
And then, the geologist Harold Jeffreys made Bayes' Theorem useful for scientists, proposing it as an alternative to Fisher's 'p-values' and 'significance tests', which depended on "imaginary repetitions." In contrast, Bayesianism considered data as fixed evidence. Moreover, the p-value is a statement about data, but Jeffreys wanted to know about his hypothesis given the data. He published the monumental Theory of Probability in 1939, which remained for many years the only explanation of how to use Bayes to do science.
For decades, Fisher and Jeffreys were the world's two greatest statisticians, though both were practicing scientists instead of theoreticians. They traded blows over probability theory in scientific journals and in public. Fisher was louder and bolder, and frequentism was easier to use than Bayesianism.

Bayes at War

In 1941, German U-Boats were devastating allied naval forces. Britain was cut off from its sources of food, and couldn't grow enough on its own soil to feed its citizens. Winston Churchill said the U-boat problem was the scariest part of the war for him.
The German codes, produced by Enigma machines with customizable wheel positions that allowed the codes to be changed rapidly, were considered unbreakable, so nobody was working on them. This attracted Alan Turing to the problem, because he liked solitude. He built a machine that could test different code possibilities, but it was slow. The machine might need four days to test all 336 wheel positions on a particular Enigma code. Until more machines could be built, Turing had to find a way for reducing the burden on the machine.
He used a Bayesian system to guess the letters in an Enigma message, and add more clues as they arrived with new data. With this method he could reduce the number of wheel settings to be tested by his machine from 336 to as few as 18. But soon, Turing realized that he couldn't compare the probabilities of his hunches without a standard unit of measurement. So, he invented the 'ban', defined as "about the smallest change in weight of evidence that is directly perceptible to human intuition." This unit turned out to be very similar to the bit, the measure of information discovered using Bayes' Theorem while working for Bell Telephone.
Now that he had a unit of measurement, he could target the amount of evidence he needed for a particular hunch and then stop the process when he had that much evidence.
While Turing was cracking the Enigma codes in Britain, Andrey Kolmogorov was fleeing the German artillery bombardment of Moscow. In 1933 he had showed that probability theory can be derived from basic mathematical axioms, and now Russia's generals were asking him about how best to fire back at the Germans. Though a frequentist, Kolmogorov recommended they used Bertrand's Bayesian firing system in a crisis like this.
Shortly after this, the British learned that the Germans were now using stronger, faster encryption machines: Lorenz machines. The British team used Turing's Bayesian scoring system and tried a variety of priors to crack the codes.
Turing visited America and spent time with Claude Shannon, whose brilliant insights about information theory came a bit later. He realized that the purpose of information is to reduce uncertainty and the purpose of encryption is to increase it. He was using Bayes for both. Basically, if the posterior in a Bayesian equation is very different from the prior, then much has been learned, but if the posterior is roughly the same as the prior, then the information content is low. Shannon's unit for information was the 'bit'.
Meanwhile, Allied patrol planes needed to narrow their search for German U-boats. If 7 different listening posts intercepted the same message from the same U-boat, it could be located to somewhere in a circle 236 miles across. That's a lot of uncertainty, and mathematician Bernard Koopman was assigned to solve the problem. He wasn't bashful about Bayes at all. He said: "Every operation involved in search is beset with uncertainties; it can be understood quantitatively only in terms of... probability. This may now be regarded as a truism, but it seems to have taken the developments in operational research of the Second World War to drive home its practical implications."
Koopman started by assigning 50% probability that a U-boat was inside the 236-mile circle, and then update his probability as more data came in, apportioning plane flyover hours according to the probabilities of U-boat locations.
And then, a few day's after Germany's surrender, Churchill ordered the destruction of all evidence that decoding has helped win the war, apparently because the British didn't want the Soviets to know they could decrypt Lorenz codes. It wasn't until 1973 that the story of Turing and Bayes began to emerge.

Revival

Its wartime successes classified, Bayes' Theorem remained mostly in the dark after the Second World War. Textbooks self-righteously dismissed Bayes. During the McCarthyism of the 1950s, one government statistician half-jokingly called a colleague "un-American because [he] was a Bayesian, ...undermining the United States Government."
In 1950, an economist preparing a report asked statistician David Blackwell (not yet a Bayesian) to estimate the probability of another world war in the next five years. Blackwell answered: "Oh, that question just doesn't make sense. Probability applies to a long sequence of repeatable events, and this is clearly a unique situation. The probability is either 0 or 1, but we won't know for five years." The economist replied, "I was afraid you were going to say that. I've spoken to several other statisticians, and they all told me the same thing."
Still, there were flickers of life. For decades after the war, one of Turing's American colleagues taught Bayes to NSA cryptographers. I.J. Good, one of Turing's statistics assistant, developed Bayesian methods and theory, writing about 900 articles about Bayes.
And then there was the Bible-quoting business executive Arthur Bailey.
Bailey was trained in statistics, and when he joined an insurance company he was horrified to see them using Bayesian techniques developed in 1918. They asked not "What should the new rates be?" but instead "How much should the present rates be changed?" But after a year of trying different things, he realized that the Bayesian actuarial methods worked better than frequentist methods. Bailey "realized that the hard-shelled underwriters were recognizing certain facts of life neglected by the statistical theorists." For example, Fisher's method of maximum likelihood assigned a zero probability to nonevents. But since many businesses don't file insurance claims, Fisher's method produced premiums that were too low to cover future costs.
Bailey began writing a paper about his change in attitude about Bayes. By 1950 he was vice president of a large insurance company in Chicago. On May 22 he read his famous paper at a black-tie banquet for an actuarial society. The title: 'Credibility Procedures: Laplace's Generalization of Bayes' Rule and the Combination of [Prior] Knowledge with Observed Data.'
Bailey praised his colleagues for standing mostly alone against the statistics establishment. Then he announced that their beloved Credibility formula was actually Bayes Theorem, and in fact that the person who had published Bayes' work, Richard Price, would today be considered an actuary. He used Bayes' ball-and-table thought experiment to attack Fisher and his methods, and ended with a rousing call to put prior knowledge back into probability theory. His speech occupied theorists for years, and actuaries often see Bailey as taking their profession out of its dark ages.
That same year, I.J. Good published Probability and the Weighing of Evidence, which helped to found Bayes' Theorem into a logical, coherent methodology. Good was smart, quick, and by now perhaps the world's expert on codes. He introduced by holding out his hand and saying "I am Good." When the British finally declassified his cryptanalysis work, allowing him to reveal Bayes' success during WWII, he bought a vanity licensed plate reading 007 IJG.
In the 1950s, Dennis Lindley and Jimmie Savage worked to turn the statistician's hodgepodge of tools into a "respectable branch of mathematics," as Kolmogorov had done for probability in in general in the 1930s. They found some success at putting statistics on a rigorous mathematical footing, and didn't realize at the time that they couldn't get from their theorems to the ad hoc methods of frequentism. Lindley said later, "We were both fools because we failed completely to recognize the consequences of what we were doing."
In 1954, Savage published Foundations of Statistics, which built on Frank Ramsey's earlier attempts to use Bayes' Theorem not just for making inferences but for making decisions, too. His response to a classic objection to Bayesianism is worth remembering. He was asked, "If prior opinions can differ from one researcher to the next, what happens to scientific objectivity in data analysis?" Savage explained that as we gain data, subjectivists move into agreement, just as scientists come to consensus as evidence accumulates about, say, cigarettes causing lung cancer. When they have little data, scientists are subjectivists. When they have tons of data, they agree and become objectivists.
Savage became a Messianic advocate of Bayesianism, but died suddenly of a heart attack in 1971. I.J. Good was active but working at a small university and was poor at public speaking. David Lindley, however, moved to Britain and almost single-handedly created 10 Bayesian departments in the U.K. — professorship by professorship, battle by battle, he got Bayesians hired again and again. By 1977 he was exhausted and retired early.

Medicine

In 1951, history major Jerome Cornfield used Bayes' Theorem to solve a puzzle about the chances of a person getting lung cancer. His paper helped epidemiologists to see how patients' histories could help measure the link between a disease and its possible cause. Moreover, he had begun to establish the link between smoking and lung cancer. Later efforts in England and the U.S. confirmed Cornfield's results.
Fisher and Neyman, the world's two leading anti-Bayesians, didn't accept the research showing that cigarettes caused lung cancer. Fisher, especially, published many papers. He even developed the hypothesis that, somehow, lung cancer might cause smoking. But in 1959, Cornfield published a paper that systematically addressed every one of Fisher's arguments, and Fisher ended up looking ridiculous.
Cornfield went on to be involved in most of the major public health battles involving scientific data and statistics, and in 1974 was elected president of the American Statistical Association despite never having gotten any degree in statistics. He had developed a congenial spirit and infectious laugh, which came in handy when enduring long, bitter battles over health issues.
In 1979 he was diagnosed with pancreatic cancer, but his humor remained. A friend told him, "I'm so glad to see you." Smiling, Cornfield replied, "That's nothing compared to how happy I am to be able to see you." As he lay dying, he called to his two daughters and told them: "You spend your whole life practicing humor for the times when you really need it."

Practical Use

Frequentist methods worked for repetitive, standardized phenomena like crops, genetics, gambling, and insurance. But business executives needed to make decisions under conditions of uncertainty, without sample data. And frequentism didn't address that problem.
At Harvard Business School, Robert Schlaifer thought about the problem. He realized that starting with prior information about demand for a product was better than nothing. From there, he realized that he could update his prior with new evidence, and independently arrived at Bayes' Theorem. Unaware of the literature, he reinvented Bayesian decision theory from scratch and began to teach it confidently. He did not think of it as 'an' approach. It was the approach, and everybody else was wrong, and he could show everybody else why they were wrong.
Later, he recruited Howard Raiffa to come work with him, because he needed another Bayesian to teach him more math. Together, the two invented the field of Decision-making Under Uncertainty (DUU). Schlaifer wrote the first practical textbook written entirely from a Bayesian perspective: Probability and Statistics for Business Decisions (1959). They introduced useful tools like decision trees, 'tree-flipping', and conjugate priors. They co-authored what would become the standard textbook of Bayesian statistics for two decades: Applied Statistical Decision Theory. Today, Bayesian methods dominate the business decision-making literature but frequentists still have some hold on statistics departments.
Meanwhile, Frederick Mosteller spent a decade using early computers and hundreds of volunteers to painstakingly perform a Bayesian analysis of the disputed Federalist Papers, and concluded with high probability that they were all written by Madison, not Hamilton. The work impressed many statisticians, even frequentists.
Bayes had another chance at fame during the 1960 presidential race between Nixon and Kennedy. The race was too close to call, but the three major TV networks all wanted to be the first to make the correct call. NBC went looking for someone to help them predict the winner, and they found Princeton statistics professor John Tukey. Tukey analyzed huge amounts of voting data, and by 2:30am during the election Tukey and his colleagues were ready to call Kennedy as the winner. The pressure was too much for NBC to make the call, though, so they locked Tukey and his team in a room until 8am when it was clear Kennedy was indeed the winner. NBC immediately asked him to come back for the 1962 election, and Tukey worked with NBC for 18 years.
But Tukey publicly denied Bayesianism. When working on the NBC projects, he said he wasn't using Bayes, instead he was "borrowing strength." He didn't allow anybody on his team to talk about their methods, either, saying it was proprietary information.
In 1980 NBC soon switched to exit polling to predict elections. Exit polling was more visual, chatty, and fun than equations. It would be 28 years before someone used Bayes to predict presidential election results. When Nate Silver of FiveThirtyEight.com used Bayes to predict results of the November 2008 race, he correctly predicted the winner in 49 states, an unmatched record among pollsters.
When the U.S. Atomic Energy Commission ordered a safety study of nuclear power plants, they hired Norman Rasmussen. At the time, there had never been a nuclear power plant accident. He couldn't use frequentist methods to estimate the probability of something that had never happened. So he looked to two sources: equipment failure rates, and expert opinion. But how could he combine those two types of evidence?
Bayes' Theorem, of course. But Rasmussen knew that Bayes was so out of favor that his results would be dismissed by the statistics community if he used the word 'Bayes'. So he used Raiffa's decision trees, instead. They were grounded in Bayes, but this way he didn't have to use the word 'Bayes.'
Alas, the report's subjectivist approach to statistics was roundly damned, and the U.S. Nuclear Regulatory Commission withdrew its support for the study five years later. And two months after they did so, the Three Mile Island accident occurred.
Previous experts had said the odds of severe core damage were extremely low, but the effects would be catastrophic. Instead, the Rasmussen report had concluded that the probability of core damage was higher than anticipated, but the consequences wouldn't be catastrophic. The report also identified two important sources of the problem: human error and radioactivity outside the building. In the eyes of many, the report had been vindicated.
Finally, in 1983 the US Air Force sponsored a review of NASA's estimates of the probability of shuttle failure. NASA's estimate was 1 in 100,000. The contractor used Bayes and estimated the odds of rocket booster failure at 1 in 35. In 1986, Challenger exploded.

Victory

Adrian Raftery examined a set of statistics about coal-dust explosions in 19th-century British mines. Frequentist techniques had shown the coal mining accident rates changed over time gradually. Our of curiosity, Raftery experimented with Bayes' Theorem, and discovered that accident rates had plummeted suddenly in the early 1890s. A historian suggested why: in 1889, the miners had formed a safety coalition.
Frequentist statistics worked okay when one hypothesis was a special case of another, but when hypotheses were competing and abrupt changes were in the data, frequentism didn't work. Many sociologists were ready to give up on p-values already, and Raftery's short 1986 paper on his success with Bayes led many sociologists to jump ship to Bayesianism. Raftery's paper is now one of the most cited in sociology.
One challenge had always been that Bayesian statistical operations were harder to calculate, and computers were still quite slow. This changed in the 90s, when computers became much faster and cheaper than before, and especially with the invention of the Markov Chain Monte Carlo method, which suddenly allowed Bayesians to do a lot more than frequentists can. The BUGS program also helped.
These advances launched the 'Bayesian revolution' in a long list of fields: medical diagnosis, ecology, geology, computer science, artificial intelligence, machine learning, genetics, astrophysics, archaeology, psychometrics, education performance, sports modeling, and more. This is only partly because Bayes' Theorem shows us the mathematically correct response to new evidence. It is also because Bayes' Theorem works.

Things you can do from here:

Subscribe to Articles Tagged ‘bayes’ - Less Wrong using Google Reader
Get started using Google Reader to easily keep up with all your favorite sites

I am doing good

2011-09-02T16:45:00.001+02:00

It was long ago since my previous post. I am doing good.

Chinese input method setup under an English locale (Debian)

2010-02-16T13:52:00.002+01:00

:~# apt-get install im-switch scim scim-pinyin scim-bridge-agent scim-bridge client-gtk scim-bridge-client-qt scim-bridge-client-qt4
:~$ im-switch -s scim -z all_ALL

你好，2010！

2010-01-01T10:35:00.004+01:00

一眨眼我们都站在新千年第一个十年的尾巴上了。这十年我都干了些什么呢？

2000--2003 我还在读高中，
2003--2007 我考入中国人民大学，开始我的大学生涯，
2007--今我来到瑞典，开始我真正的学习生涯。

读书成了我这十年生活的主旋律。

过去一年我都干了些什么呢？

有些事情完成了，

夏天完成第一篇博士论文，秋天被杂志正式接收。
荷兰和丹麦旅游。
坚持早睡。

哦，中间一个小插曲是买了一个域名是我的全名，http://feng.li/。我得儿意的笑。

然而有些却可耻的失败鸟，

坚持早起。
坚持锻炼身体。
坚持学习英语。
坚持不发脾气。

其实每年都有一个新年（学期）计划，草草写下2010年的计划，

完成第二篇博士论文，学习如何 RESEARCH 而不是 RE－SEARCH。
夏天去一趟法国和西班牙。
重启2009年失败的计划。
学摄影。
坚持手写汉字。
......以及其他若干小事

记得本科时候有个老师让大家写下每个人的人生规划，我写道 "......人生无常，计划赶不上变化......", 结果被老师狠狠当堂匿名批评了。其实有个大概的计划（当然不用细到每天几点起床，几点上厕所，几点喝咖啡）是挺好的事情，每个一段时间回来看看，反省一下，以免出现这样的情节

路人甲：最近怎么样？
路人乙：忙！
路人甲：忙什么呢？
路人乙：瞎忙。

过去的2009有许多事情值得回忆，

2009年我感受最深的一句话是2003年开学吴喜之老师说的 "直到读到博士，我才明白什么是统计"，虽然至今我还不懂什么是统计。
2009年我最感动的事件是中国三名矿工在井下被困602个小时顽强生存下来。
2009年我最喜欢的歌曲是 "I dreamed a dream" by Susan Boyle。
......以及其他很多......

那么，开始吧， 2010。

Replace ^M in Emacs

2009-08-22T13:49:00.002+02:00

Have you seen a ^M at the end of each line while you are using your favorite emacs to open a file which was created under Windows? Very annoying? This is because in the Dos/Windows world text files end each line with a return and a new line character \r\n. Linux and UNIX on the other hand just end the line with a new line character \n.

I didn't realize this is going be a problem until one day I opened my friend's big latex file which was created under Windows. At the first time, I just replaced those ^M with space but later I realized this would be a disaster for me when I get the second and third... Thanks to the wonderful macro mechanism in emacs, so we can make things much easier. You can also take this as one application of emacs macro. The basic procedure is quite simple,

1. Open one file with emacs and record a macro.

C-x (              # start recording
M-<      # go to the beginning of the document
M-% C-q C-m RET <space> RET !         # query ^M and replace it with space
M-<      # finish replacing and go to top of the document (optional)
C-x ) # finish recording macro
M-x name-last-kbd-macro RET my-replace-m # give a name of the macro

2. Don't quit emacs and open your .emacs .

M-x insert-kbd-macro RET my-replace-m RET # The macro will be written to .emacs at curent your cursor place.

You may find two lines like those in your .emacs

(fset 'my-replace-m
[escape ?< escape ?% ?\C-q ?\C-m return ? return ?! escape ?<])

You can add a short cut <M-9>for this macro after the preceding

(global-set-key "\M-9" 'my-replace-m)

3. Ready to go.

M-x my-replace-m RET # or just
M-9

Manually Install readline library and R under Linux

2009-08-15T02:22:00.006+02:00

Recently, I found that the R program in my Linux server (x86_ 64) is too old to use. I decide to install the newest version under my home folder that means no root needed. Here is a note for installation.

1. Build readline from source and install it to my home folder. If we don't install readline library, we have to add --with-readline=no in step 3 but the drawback is that we are not able to edit command lines as we are in the shell environment.

./configure --prefix=/where/to/install/
make
make install

2. edit file config.site to tell R where to find readline library and header for compiling, like this

LDFLAGS='-L/where/to/install/lib'

CPPFLAGS='-I/where/to/install/lib/include'

3. Build R but disable X11, since my server does not support X11.

./configure --prefix=/where/to/install --with-x=no

make
make install

4. Now I am ready to start R by typing.

/where/to/install/bin/R

How to type Ä, ö, Å etc on Blackberry

2009-08-12T00:30:00.004+02:00

For example, to get Ä, hold A and move the trackball to select. Same goes for õ and other characters. It works just fine though I feel a bit slow while typing.

Customize AUCTeX menu

2009-08-04T17:41:00.006+02:00

I am a heavy AUCTEX user and I found the default menu is not enough if I want generate div file and then ps to get the pdf file in the end. So I add a one-click command in the menu by put the few lines to .emacs file

(eval-after-load "tex" '(add-to-list 'TeX-command-list '("dvi->ps->pdf->Adobe" "dvips \"%s.dvi\" -o \"%s.ps\"; ps2pdf \"%s.ps\"; acroread \"%s.pdf\"" TeX-run-command nil t) nil) )

which is very convenience for me.

再回人大

2009-07-28T07:57:00.001+02:00

时间：2009.7.13 -- 7.15

距毕业那年已经整整两年了，终于有机会再次踏上这片土地，这次专门住在人大招待所，可以去集天吃早点，可以去小南门买麻辣烫，可以去西门吃西瓜，可以去东门吃黄记隍，可以去北边吃权金成，还可以在校园溜达。已经是半夜了，拿起那个许久之前的电话本，随手翻着顺便拨出去，那头传出熟悉略带调侃的声音，"你小子真会选时间啊，不知道我在上班啊"，哦，曾经我们半夜跑出去吃烧烤，也不管第二天有没有课。我都忘记了，你们都工作了。看看表，都快12点了，可还是忍不住又拨了了几个，"我来北京了，抽空聚聚？"，"好啊。..."

电视里放着陈亦迅的好久不见，
"... 熟悉的那一条街
只是没了你的画面
我们回不到那天
你会不会忽然的出现
在街角的咖啡店
我会带着笑脸
回首寒暄
和你坐着聊聊天
我多么想和你见一面
看看你最近改变
不再去说从前
只是寒暄
对你说一句
只是说一句
好久不见 ..."

第二天，人大西门，九头鹰，四楼渴望，大家喝着啤酒，聊着天，

后乐说把一百万变成一千万要比把一块变成十块容易。

李晶说我们单位那个什么补助给一辈子，许杰说没有床单？

刘冬养了个蛐蛐手机。

我想吃你们那的豆皮，抱歉这个没有，这个可以有，这个真没有。

钟桢一如继往的迟到了，罚酒是免不了的。 ...

说着说着就晚了，于是结帐，请出示学生证，可以打八八折，摇摇头，我们曾经是来着。

于是散去，我们来年再见。

最美

2009-02-22T00:11:00.001+01:00

baby 为了这次约会
昨夜我无法安然入睡
准备了十二朵玫瑰
每一朵都像你那样美
你的美无声无息
不知不觉让我追随
baby 这次动了情
彷徨失措我不后悔
你在我眼中是最美
每一个微笑都让我沉醉
你的怀你的好
你发脾气时翘起的嘴
你在我心中是最美
只有相爱的人最能体会
你明了我明了
这种美妙的滋味
baby 记得那次约会
那夜我想你想得无法入睡
送你的十二朵玫瑰
是否还留有爱的香味
你的美无声无息
不知不觉让我追随
baby 这次动了情
彷徨失措我不后悔
你在我眼中是最美
每一个微笑都让我沉醉
你的怀你的好
你发脾气时翘起的嘴
你在我心中是最美
只有相爱的人最能体会
你明了我明了
这种美妙的滋味
走在街中人们都在看我
羡慕我身旁有你依偎
陷入爱中的我不知疲惫
为了伴你左右与你相随
你在我眼中是最美
每一个微笑都让我沉醉
你的怀你的好
你发脾气时翘起的嘴
你在我心中是最美
只有相爱的人最能体会
你明了我明了
这种美妙的滋味

Paul Krugman

2008-12-08T19:18:00.001+01:00

Listen to the gentiles
Question the question
Dare to be silly
Simplify, simplify...

--Paul Krugman

Import matlab file to R

2008-11-21T17:06:00.002+01:00

If you need to read a .mat file came from Matlab, it can be a pain when you don't have Matlab installed on your machine. So, buy a license? No. Let's do it in the easy way with R. I assume you are running a Linux machine. You can simply start an R session and type the following lines

install.packages('R.matlab')
require('R.matlab')
install.packages("Rcompression", repos = "http://www.omegahat.org/R")

Probably you also need libz (a.ka. zlib) and libbz2 (a.k.a bzip2) when you build from the source. Check http://www.omegahat.org/Rcompression/ for a detailed installation guide for Rcompression package. Now you can read and write mat file within R.

require('R.matlab')
readMat() # read .mat file to R
writeMat() # write R data set into .mat file

Note: For a windows machine, you may encounter such warning,

Warning message:

In getDependencies(pkgs, dependencies, available, lib) :
package ‘Rcompression’ is not available

This happens because you have a newer version R and there is lack of compiled zip file for current R version. You can either compile it yourself (requires Rtools) or you can go to the repository http://www.omegahat.org/R/bin/windows/contrib/ and find a old version to use.

new start 2

2008-09-06T12:58:00.001+02:00

我记得去年大概这个时候我写过一个叫做 "new start", 那时候我才刚来瑞典，转眼就整整一年过去了，这次姑且叫做 "new start 2" 吧。

今年搬到斯德哥尔摩了，换了新环境，新面孔。我所在的这个系很传统，几乎没有外国学生，我大概是这里第一个亚洲学生了吧。本国学生年纪都很大了，有的小孩都好几个了还在读书。很赞人家生活的方式和心态。

系里给安排了办公室和电脑，１９寸的液晶显示器让我这个成天守在电脑前的人来说是最好不过了。大家对我很好，我不怎么会瑞典语，大家就没事的时候教我一两句。话说我还是长进很快的，在一个月后我终于学会了用瑞典语问早上好: God morgon!

这里经常会组织聚会，九月三号我们所有的人下班后乘船去了一个叫做 Fjäderholmarna 的离斯德哥尔摩不远小岛，那里有一家酒店有很传统的熏三文鱼。这还是我第一次吃传统的瑞典正餐。我和我的两个导师还有一个师兄坐一起，每人前面有两副刀叉和一个勺，由于不知道礼节，怕出洋相不敢乱动什么，偷偷瞟一眼人家怎么做，自己赶紧跟着。第一道菜是一篮子手工面包和黄油，大家每人拿了一片抹点黄油吃了。接下来服务生问喝什么酒，他们都要了白酒（白兰地之类，我也不大清楚），好大一杯，大概有半瓶的样子，把我给吓坏了，我说我要啤酒吧，人家说给我换个杯子，结果给我换一个扎啤大小的杯子，满满的一杯。接下来是第二道菜，一个大盘子上铺一片黑面包，上面盖一团奶烙和水果拌好的虾，旁边是一小碟黑色的酱，尝了一下没什么味道（用圆的刀叉）。第三道菜是熏三文鱼和烤小土豆，味道不错（用尖的刀叉），正餐就这样结束了，大家酒喝的也差不多了，于是说话的多起来了，像在中国式的，人们到处乱窜，大声说笑，有吃多喝多的去上厕所等等。最后一道是每人一个小点心和咖啡。这时服务员又过来问要不要酒，有人点了，不过这回是自己付帐，现金结算，有人连续要三四次。之后就是大家聊天一直到半夜才坐船返回来。有两个女博士觉得喝的不过隐，半路上又去了另一个酒吧。她是真的喝多了，第二天还醉熏熏的问我，昨天你去哪了，怎么没见到你? 我明明就坐在她对面，还和她聊了一路。

生活虽然丰富，还是学习重要。这学期主要以看书为主，自己看书，有问题问老师，最后作报告给导师，导师给成绩。国外就是书贵，一本书动辄好几百，工资还没有下来，没钱买，只能借图书馆的看。主要是系里没有特别合适我的课程，倒是在乌普萨拉有一些，但是得等到明年春季。

这里大牛很多，每个人对统计都有独到的见解，系里安排了每天下午两点的咖啡时间，不管新来的还是退休的，都聚在一起聊聊。对我来说也是一个锻炼的机会。有时候一些无聊的人也会想出一些很奇怪的问题来，比如这个问题:

任意给定两个正整数 a 和 b, 比如 a = 99, b = 18. 那么 a 乘以 b 等于 1782。现在作如下运算，每次给 a 除以 2 并且向下取整命名为新的 a，同时给 b 乘以 2 并且命名为新的 b，直到 a = 1，如下所示:

a               b
------------------------
99            18*
49            36*
24            72
12            144
6           288
3             576*
1             1152*

然后找到所有奇数 a 对应的 b （标*号）并将其求和 18+36+576+1152 = 1782。正好等于一开始给的a 乘以 b 等于 1782。怎么证明呢?

闲言少叙，总之，新的起点新的开始。希望大家也是。

农历七月初七

2008-08-06T20:39:00.001+02:00

去年这个时候我好像正在飞机上，

一晃眼一年过去了，

大家都好好的吧，呵呵。

今天吃了一颗糖和一只梨

2008-05-06T17:58:00.002+02:00

糖是阿公送的，从爱沙尼亚带回来的，沙甜沙甜的；梨是我从Hemköp买的，水不多，放几天就好了。

点名游戏

2007-11-19T21:08:00.002+01:00

点名了，嘿嘿，我按规矩回答问题，大家都忙就不点名了，大家以后若还记得我，一定要默念八荣八耻+南无阿弥陀佛祝福。
1. 从谁那里接到的题目？
Saiyo mm
2. 2007年最大的心愿是什么？
2007年都快过去了，都这个年纪了，没什么大的心愿了，随便说两个小小的吧，能弄到奥运梦票
，回国看场比赛；身体倍儿棒，吃嘛嘛香（皮萨除外）。
3. 你喜欢暧昧的感觉吗？
不知道是啥感觉，谁发明的这个词，看了半天，怎么那么像暖味。
4. 刻骨铭心的事。
很多吧，譬如毕业，譬如天津，譬如我来到了瑞典

5. 在你最难过时，你会首先想到谁？
至今似乎没有经历过特别难过的事，但愿不会。
6. 你最喜欢什么颜色？
喜欢墨绿色，大概和我的性格有关系吧。
7. 你最害怕什么？
失去自己所爱的人和爱自己的人（严重同意Saiyo mm）。
还有补充一个要命的，从此无笑点。
8. 对传题人印象最深的一件事是什么？
牛mm
9. 不开心的时候你会做什么？
睡觉，什么也不想。
10. 如果是自己关心的人不开心了呢？你会为他（她）做些什么？
安静的陪着。
11. 你的梦想是什么?
曾经梦想有很多，现在只想

12. 做答卷的10min前在做什么？
在厨房吃牛肉面，外加一个鸡蛋，哈喇子~
13. 为什么选我做朋友？
明明不是我选择的啊，哈哈，况且有朋自远方来，不亦乐乎。
14. 用一个词形容我们之间的关系。（请被点名的朋友回答此问题：最满意自己什么方面？）
同学+朋友? 最满意自己能吃能睡
15. 如果得到自己想要的幸福会不会与朋友一起分享？
当然会，有福同享。

16. 你认为我最大的弱点是什么？
还没找到，容我慢慢找找，待续
17. 觉得自己孤独么？
在北欧这个下午三点天黑的小镇居然没有孤独到抑郁，谢天谢地。
18. 哪一首歌能代表你最近的心情？
找一首老歌来听吧，任贤齐"春天花会开"

19. 觉得未来是怎么样的？
地球变暖了，北欧冬天可以穿短裤了
20. 如果我们呆在一起不说话你会不舒服么？
好久不见的老朋友怎么会不说话呢？如果真的不说话，只有一个可能，吃撑了。

白色的冬天

2007-11-07T22:38:00.001+01:00

又下雪了，在这个远离喧嚣的地方，下雪了我却没有感到出行任何的不便。一大早走在松软的雪地上，软的也是暖的，留下一串脚印，偶尔会有一两片倏的钻进脖子，心想当今世间竟还有有点如此佳境。此时要是有一盘熟牛肉，烫一壶热酒，即使落草也愿意。

...把花枪挑了酒葫芦，将火炭盖了，取毡笠子戴上，拿了钥匙出来，把草厅门拽上；出到大门首，把两扇草场门反拽上锁了，带了钥匙，信步投东，雪地里踏着碎琼乱玉，迤逦背着北风而行...——林教头风雪山神庙陆虞候火烧草料场

...武松入到里面坐下，把哨棒倚了，叫道："主人家，快把酒来吃。"只见店主人把三只碗，一双箸，一碟热菜，放在武松面前，满满筛一碗酒来。武松拿起碗一饮而尽，叫道："这酒好生有气力！主人家，有饱肚的，买些吃酒。"酒家道："只有熟牛肉。"武松道："好的切二三斤来吃酒。"...—— 横海郡柴进留宾景阳冈武松打虎

其实我一直认为向往自由的人去做一个山大王是一个不错的选择。早上英文课一个瑞典老兄说，It's a hard life if you don't weaken, and it's expensive being rich.其实他们不知道—— I really don't want to be rich, because I want to save money.

雪过了，天晴了，出去时还是一片蜡象，回来就变了。不知道古人造出好景不长这个成语是啥居心？以后能不能白天下雪，晚上化掉，反正瑞典下午三点就天黑了。

A TeX Study Note

2007-09-24T01:50:00.002+02:00

OS: Microsoft Windows XP sp2

Tools: MiKTeX 2.6

Winedt 5.5 or Emacs 22.1

Components: GSview 4.8

Ghostscript 8.53

Plug-ins & Add-ons:

TeXaide 4

Tablas 3.0

Graphics Interface

Bibliography: The Not So Short Introduction to LaTeX2e(lshort)

LaTeX Tutorials -- A Primer

LaTeX math and graphics

LaTeX2e for authors

---------------------------------

Frankly speaking, I really dislike the interface of Winedt: tool bar in two lines, menue with logs and useless wrap. I prefer Emacs in either Windows or Linux which is hard for beginner to handle but easy to use. I am new here. I mean Tex. But it doesn't matter. I can use google as well as Ctrl+ F.arha~ That's enough for me to write a thesis with TeX without many problems.

No news is good news

2007-09-14T13:28:00.002+02:00

As the title, no news is good news. But still some may interest you.

I really want to improve my poor English from pronunciation to writing systematically. It is really a huge project for me. Luckily, I have found a good teacher yet, exactly speaking, yes,excellent. Hope that will be a new start.

Learn from others. Change the ugly habits and never indulge myself. It is quite right that one always indulge himself is the one without any aim in his life. ---by noname.

Don't lost my way. Never.

That's all.

New start

2007-08-24T00:18:00.002+02:00

I am fine in Sweden.

New start.

要走了,和大家告个别

2007-07-20T12:36:00.001+02:00

多日没写，来除除草，顺便和大家告个别。

签证已经下来了，大部分事情也已经办妥。我一两天就回内蒙古，之后就不来了。

八月19号14:44的飞机，首都国际机场，直飞斯德哥尔摩。

我在瑞典的地址是:

Room No. 201 33 1 416

Kornstigen 15 C, S-784 52 BORLÄNGE

大家有空来玩哈:)

现在的手机号还在用

840116@gmail.com (Email, MSN)

衷心祝大家好！
--
God's In His Heaven,All Rights With The World !
http://840116.googlepages.com/

青春・散场

2007-06-10T08:21:00.001+02:00

有人说喝酒得看心情，

高兴了就是甜的，

伤心了就是苦的。

终于还是等到了这一天，

我们一起吃过散伙饭，

互相拥抱，

用眼泪告别。

说一声：珍重。

半夜酒醒，久不能寐。

努力的

回忆着你对我说的每一句话，

回忆我们在一起的每一件事，

我想把我们的点点滴滴都记下来，

我怕离开了就忘记了。

我想带大家去草原玩，

我想明年和大家一起做志愿者，

我想说我爱你。

可是，

我就要走了，

我不知道何日是归期，

我不知道我许下的诺言能否兑现，

我甚至都不知道将来我会在哪里

...

请原谅我。

再见了，

我最最亲爱的同学们，

后会有期。

--
God's In His Heaven,All Rights With The World !
http://840116.googlepages.com/