Why big data is a big deal at a big school

Cosmic background noise for placement only

The big data activities of three Harvard School of Public Health professors are discussed in the recent issue of Harvard Magazine (Mar.Apr 2014).'Why Big Data is a Big Deal' looks at their work, and it emerges that, basically, there is a pretty obvious connection with what is posited now as 'big data' and a couple of trends long enfolding.  Especially, computer analysis and data gathering in the social sciences - over many years – has grown – grown to the point that its tenets seem evident, natural, and broadly applicable beyond their initial use cases. The Harvard profs exemplify the emerging style. What is new? It is most evident in the case of Gary King, Weatherhead university professor and head of Harvard's Institute for Quantitative Social Science. He has used data with special imagination, yes. He has also found ways to use social media information and cell phone data as part of the analysis, even in places far afield. Like others he attaches a bit of mysticism – or 'capacity to drive good' - to the concept 'data' . Data as a lynchin-pin for a  movement is growing. And the Harvard crew is emblematic. 'Improved statistical & computational methods-not in growth of storage or computational capacity'  – Jack Vaughan

Data Gumbo for the Week of Noxious Fumes

Predicting legislation, or Follow the money http://blog.fiscalnote.com/2014/04/22/legislating-todays-science-fiction-tomorrow/

Don't touch that dial! Catch Jack Vaughan speaking with Nicole Laskowki about #GartnerBI on Talking Data podcast. bit.ly/1k4ef0y

A majority of financial enterprises (67%) present a "repeatable" level of #bigdata #analytics maturity. -per IDC http://www.idc.com/getdoc.jsp?containerId=prUS24808014

'Improved statistical & computational methods-not in growth of storage or computational capacity' http://harvardmagazine.com/2014/03/why-big-data-is-a-big-deal

Read Wayne E's big data/data warehouse clash http://bit.ly/Rwc5NY  Insightful: Reminds of other techno shifts w incumbent painted all bad!

Big data: big mistake? -  http://on.ft.com/P0PVBF  via @FT 'Big data” has arrived-big insights have not.'

Cringely: Big Data is the new Artificial Intelligence http://betane.ws/s0Am  via @BetaNews

Does #bigdata improve contextualization of science? http://philsci-archive.pitt.edu/9944/1/pietsch-bigdata_complexity.pdf

The Parable of the Google Flus: Traps in Big Data Analysis

Toward a Vision: Official Statistics and Big Data http://magazine.amstat.org/blog/2013/08/01/official-statistics/

What is going on with data management technology?

What is going on with data management technology?

One thing is pretty true: Information of all types is engulfing the corporation. There’s more…

• Web apps and distributed cloud computing have grown.
• And a slew of new technologies has arrived to help companies cope with the data influx and distributed data processing …
• … But sorting through those technologies is tough.
• And you can’t start fresh unless you are a startup.
• If you are established, you worry about startups taking your business …
• But the original ‘queriable’ relational database approach is still valid.
• Newbie software has to adapt too .. add old style capabilities, and vice versa.

Data privacy stories on SearchBusinessAnaltyics

Snowden by link up talks with Euros

My SearchBusinessAnalytics.com colleague Ed Burns has been at work on a fine series on emerging data privacy issues. In Data collection practices spark debate on big data ethics, privacy and Laws leave gray area between big data and privacy he paints a picture of the current landscape (and more is on the way).  One thing that emerges, and this is something that Ed speaks with me about on an upcoming Takling Data podcast: there is as sort of entropy going on here – it skews toward the status quo, which is, in broad brush, there really is no such thing as privacy and in too many cases your data is my cash cow. We've heard plenty of talk about the big data gold rush, and data as the new oil – what that translates into is some lip service to the notion of a data self. Burns and I are enthusiasts for the new possibilities of data but both of us I think suspect that data and analytics professionals have to be sure to treat what they do as a profession, and consider the ethics of data mining, as they would any other kind of mining. I think his series on privacy is a good step in laying the ground work for a discussion around this. Does the industry need another Snowden event to wake up to the need for ethical  standards? -Jack Vaughan

 Data collection practices spark debate on big data ethics,privacy - SearchBusinessAnaltyics.com

 Laws leave gray area between big data and privacy - SearchBusinessAnaltyics.com

The Unknown Known – On Rumsfeld's ridiculously sublime rumination

 


Nate Silver's well regarded The Signal and the Noise (2012) included a chapter in which the author intrepidly  gumshoes it to Donald Rumsfeld's office, mostly to discuss the former secretary of war's long-running enthusiasm for a little known 1962 book, Pearl Harbor: Warning and Decision by Roberta Wohlstetter.  Actually the greatest enthusiasm may be that which Rumsfeld held for the book's introduction, one penned by economist Thomas Schelling who wrote: "There is a tendency in our planning to confuse the unfamiliar with the improbable."

Before Pearl Harbor, the U.S. expected sabotage from Japan, but not a six-carrier air attack from the north. This formed what Silver might describe as a 'signal and noise' moment when a massive trove of information was not effectively sifted – and Pearl Harbor was not predicted.  There would seem to be a lesson in analytics there somewhere.

Rumsfeld somewhat famously circulated the book in Washington months before the Sept 11 2001 terrorist attacks, and he has a Xeroxed copy of the forward at hand when he meets Silver. After the fact, the Wohlstetter book's  theme seemed applicable, in Rumsfeld's – and, perhaps, Silver's - estimation, to Sept 11. And it may have formed a backdrop for Rumsfeld's ridiculously sublime rumination on known knowns, known unknowns and unknown unknowns, another  variation of which (the unknown known) form the title of Errol Morris' new film, which is what I came here to tell you about.

I call Rumsfeld's 'unknown unknown' wordsmithing ridiculously sublime because, upon viewing Morris' film, I conclude that Rumsfeld's 'understanding' of the Pearl Harbor lesson was more misunderstanding – was more a willful, spiteful and devilish confabulation of analytics. He took a bit of truth and with some technical exactitude mis-applied it to the case of Iraq and its purported troves of weapons of mass destruction, for his larger purpose (political bias) of, well, say, shaking up the Middle East. He took the idea that the Pearl Harbor debacle was caused by failure of imagination, and imagined a fabled debacle all his own. Prediction provides some very special care, evoking a rework of Bob Dylan line: "To live outside of time you must be honest."

"The Unknown Known" is not quite on par with Morris' portrait of Viet Nam era Defense Secretary Robert McNamara as a film and a story, the protagonist elicits less empathy in this viewer, but it is worthwhile in its probing pursuit of logical understanding – in its analysis. Also, like the earlier 'Fog of War', it has some nifty animation. - Jack Vaughan