Correlation and Causation: A new look

Correlation does not imply causation. The logical fallacy that correlation implies causation will be starkly pointed to if you falsely make a cause-and-effect determination based on your observation of the world, your observation of data, or your study findings of a correlation.

Chris Anderson, editor-in-chief and famed writer of The Long Tail, has tackled the idea that the deluge of data makes the scientific method obsolete. That correlation is sufficient in the Petabyte Age as he calls it, and the practice of causation only leads us to realize that we understand even less about the world.

Jackson West rightly points out the dangers of throwing the scientific method under the bus for the sake of applied mathematics and relegating the creation of models of the world to high and mighty, “pansy” theoreticians. And he’s generally right to be critical of a loss of strict rationality and too much focus on faith.

Data can be manipulated to show what you want. Not always in an evil manner, but methodology and the very collection of data itself can affect the outcome. You can get what you ask for, or perhaps deserve as Mr. West concludes.

Take it further and a strict reliance on data leaves room for dangerous regimes. Tyranny of the majority, self-fulfilling prophecies: these are all dangers that are left on the table when strict rationality goes out the window.

Yet Anderson is certainly not suggesting this, only pointing out a real observation of the world in which we live. And yes applied mathematics can be profited from, theoretical mathematics not so much. Ian Ayres, a Yale Professor, is the author of Supercrunchers whose very title is number crunching is the way to be smart.

The examples are largely the same: information (Google), airline predictions (Farecast), legal discovery, predicting the outcome of an election. It’s easiest to see manipulation possible in the last example.

Oh well, so be it. I’ve written that blind faith is basically idiotic. A strict reliance on numbers may not mean you’re determining a causation, but perhaps correlation is enough. And for those who operate in a gray area where luck plays a part, where luck is better described as a set of circumstances beyond your control, Anderson’s piece could be comforting.

But more importantly, it outlines a real path to be smart and succeed. And so you might be wise to heed the words that correlation is enough, and that correlation supersedes causation in a world dictated by Petabytes of information ready to crunched.

Opportunities in the Food Chain of Google, Facebook

It’s a wild and crazy time, or even week for that matter, and I’m not just talking Halloween here. I’m talking about the showdown for the domination of the ultimate web platform.

Erick Schonfeld, new editor at TechCrunch, writes about Maka-Maka, a codename for adding a social layer across its entire suite of applications.

I think to anyone familiar with Google’s ambitions, this really shouldn’t be a shock. What’s crazy here is the effect that Facebook’s platform has had to really up the ante, accelerating plans across the board to capitalize on the creating of an ultimate social graph – or more importantly – of building the ultimate web platform. It’s rather remarkable that we now speak about Facebook in the same breath as Google ($212 billion market cap!).

As Schonfeld explains, it’s clear that Google has more data about your behavior than anyone. And some will be quick to point out the privacy, and big (and evil) brother concerns. Remember when there was initial resistance against AdWords advertising against Gmail data? Google’s applications (Gmail spam protection alone) are so well executed at times that they start to creep into your life. And I’m not necessarily upset about it.

At least they’re not in the middle of a scandal about nosy, meddlesome Facebook employees messing with people’s personal Facebook profile, as Valleywag reports. I’m not going to play conspiracy theorist, but I do know Facebook will be very wise to come down on this hard and fast, both with disciplinary and technology measures if they want to be the ultimate gathering place to live your online social life.

Stan Schroeder at Mashable writes about the possibility that Google could, just possibly, be vulnerable. No matter how much you might want to think that Google is vulnerable (especially EnjoyPerth.Net), it’s not very vulnerable in its central goal of organizing information.

So the question for startups is: what are the new opportunities for new startups? A Facebook or Google platform aren’t the only choices either: Salesforce.com is becoming a platform force with AppExchange, and you’ll be able to utilize not just Facebook, but MySpace, LinkedIn and other powerful APIs soon enough as well.

The question then becomes: can you aggregate a large enough audience to reach critical mass, or perhaps more importantly, to actually be relevant in the ultra-competitive startup world?

Want to start a niche social networking site? Most of the time, I would frown against it BUT a fun, useful app built on an existing infrastructure could be a more lightweight and more effective path.

Vertical search? Farecast is still my best example of doing this in an innovative, yet user-friendly manner. However, you still have to execute something that people want. People want to save on travel; they generally don’t need a special search engine to search blogs (Technorati). I’ve written about creating value mattering in the noisy world of startups, but it’s just as important that you actually solve a problem. Fix something that’s broken, plain and simple. If cocky companies like Facebook and Google can admit they can’t do everything, they may just have a point.

If this new platform hierarchy creates a food chain, then Google and Facebook (right now) are at the top of the food chain though: the pinnacle of where you want to be. Most everyone is probably going to have to rely on them somehow. By and large, that’s going to be OK, but we all need to realize that’s how the ecosystem is set up and prepare accordingly. If you want to think big, just don’t get stuck way, way at the bottom of the food chain.  This new game is NOT a zero sum game, and the smart ones who keep this ultimately in mind will find there’s plenty of opportunities to solve problems and make money.

Virgin Money Re-branded from CircleLending

Virgin Money was just launched today, re-branded from CircleLending, which Virgin Money purchased a majority stake in as the New York Times points out.

The article quotes Alenka Grealish that the segment is at a “smaller scale than Virgin usually enters into, but it’s an interesting one.” It’s also an opportunity that funded startups such as Prosper are attacking as well, clearly a sign that people see this as a huge growth opportunity.

Charismatic Virgin founder (and personal hero) Richard Branson will be giving away red dollar bills with him and his mother, a reminder of the help that his mother gave him by giving him exactly one of the loans that Virgin Money will be supporting.

What is slightly different is that Virgin Money (and CircleLending as its precedent) will focus on established relationships. Others like Prosper focus on peer-to-peer lending, something I find could prove more problematic but ambitious nonetheless. There is something to be said about the necessity of stability gained through transactions that involve established institutions, sometimes above the peer-to-peer ambitions. Virgin Money is going after banks to this end.

As an adjunct to talking about the future of web startups, it instantly hit me as the marriage of a great idea and one of my favorite brands.  If there’s a big market here, then Virgin Money is now bound to be a giant part of that.

TechCrunch also points out that today that Google leads social bookmarking above geek favorites such as del.icio.us and bloglines. Google, above all, is a testament to the magic that can occur when a truly useful product and golden brand are married.

Mint: when I first heard it instantly thought it was a great idea and got the hint that it was a great management team. Sure enough, it was the winner of TechCrunch40. Apply the same idea, and I think if Mint is able to create a great brand (a.k.a. Quicken), this could be a huge business.

Brought back to the future of web startups, it only strengthens the idea that small “idea” companies that execute well at early stages will marry with established brands. It’s only every once in a while that a Google pops up. Under every great brand from Google to Amazon to Virgin to Facebook and so on, many more ideas can see the light of day to a truly large viewing audience.