The Second Machine Age: All roads lead to machine learning

Pew data points on NSA surveillance and public

Just caught up with a Pew Research Center/USA TODAY poll conducted in January that estimated overall approval of NSA surveillance had declined since last summer, when stories first broke based on Edward Snowden’s leaked information.... Democrats remain more supportive of the NSA surveillance program than Republicans, though support is down across party lines....While most of the public wants the government to pursue a criminal case against Snowden, young people offer the least support for his prosecution....

http://www.people-press.org/2014/01/20/obamas-nsa-speech-has-little-impact-on-skeptical-public/

Mystic crystal data pondered

Composer Philip Shepard sees links
between data and music.
He scored a film on chess
master-cum-crazy-man Bobbie Fisher.

The near mystic quality some people attribute to data should be some cause for concern. Something doesn’t usually go from familiar and forgettable to world-changing and magical overnight.  What portion of today's data paeans ("I love data") ("Data is the new punk") will flower dandelion-like then drift off on wind? It is hard to say.

But a feeling can hold that some of this is good, worthy. I've had a chance to see a few conference keynotes that dabbled more in the art – less in the science - of data. And some ring true. At the recent Enterprise Data World event, there was a session by a data strategist from Marvel comics that made the case for applying graphing database architecture to the 'need' to rationalize different incarnations of different super heroes as data elements.  Entertaining, yes -but not an effective poster for data as a new way of being. But another session, one led by composer Philip Sheppard, had more such merit. The Marvel guy seemed to admit that - by noting that he wouldn’t want to follow Sheppard's presentation.

AT EDW14, Philip Shepard discussed information as symphony. Sheppard does film scores - says #Music is #data. @PhilipSheppard #EDW14 He makes a case that data is poetic. It is true, as he point out, you can look a little and see lyrical whiffs in graphical renderings on data on public bicycle use, the slipstream of air pressure readouts for an F1 racer.

But really, the place where his insights into the special nature of data bear the most fruits is the place he is closest to – music. "What is music? He asks. It's loads of things. He answers. It's transformative. It is a form of solace. You can wallow in it and you don’t really know why. Memory is so connected to music. People learn whole caches of text, when there is music attached. Once it probably was the major way of encoding history. Much can be learnt from the way musicians cope with huge amounts of data under duress. The basic music message can change over time, depending, eg., on players' emphasis. Sheppard's words to the data folks assembled: "When you are dealing with things you have to look at them as fluid I think we are starting to look at #data that way."

He has a point. When I was at the symphony once it strikingly dawned on me that this was a message from a human in time. Ludwig. I wrote about it on my art blog (MoonTravellerHerald) under the persona of Shroud Jr.  
Was in the symphony one day – many rainy years ago -- and Beethoven’s message was just crystalline to me. Me, Shroud Jr. Like a telegraph message through the foam of time – Beethoven heard the birds, the guns, he was losing his hearing. He was writing it down. Sending it out. Shroud Jr. was pickin up on it.
Truth be told, I was taken away with Sheppard's music, and can't do his argument justice here! I don’t find much on the web of his that helps directly either.  But some links follow. Communication, music – an interesting path always. Now, revise as communication, music, data. – Jack Vaughan

Related
http://philipsheppard.bandcamp.com/album/bobby-fischer-against-the-world
http://philipsheppard.com/philip-sheppard-biography/
http://edw2014.dataversity.net/sessionPop.cfm?confid=79&proposalid=6305
http://philipsheppard.com/

Big White House Data Report 1

The White House has released a new report on big data and privacy. It has not yet released a report on the suspect and widely reported activities of intelligence agencies. In the NYT's estimation, the report does fine job of laying out some of the benefits and problems associated with extensive data collection and its use in business. Among the benefits is such data's value in medical research. But, the NYT in an editorial page article today asks, cant that same data also be used to discriminate in sales or services? The editorial (A Long Way to Privacy Safeguards) commends the report for its recommendation that law enformecemnt agencids seek court approval to access gidital content like email in the same way they do for physical letters – this as the Supreme Court is poised to consider warrantless searches of cell phones.  The story makes the point that consumers lose control over their information from the moment that it is collected, and the point of collection is the point of infection in data privacy.

Why big data is a big deal at a big school

Cosmic background noise for placement only

The big data activities of three Harvard School of Public Health professors are discussed in the recent issue of Harvard Magazine (Mar.Apr 2014).'Why Big Data is a Big Deal' looks at their work, and it emerges that, basically, there is a pretty obvious connection with what is posited now as 'big data' and a couple of trends long enfolding.  Especially, computer analysis and data gathering in the social sciences - over many years – has grown – grown to the point that its tenets seem evident, natural, and broadly applicable beyond their initial use cases. The Harvard profs exemplify the emerging style. What is new? It is most evident in the case of Gary King, Weatherhead university professor and head of Harvard's Institute for Quantitative Social Science. He has used data with special imagination, yes. He has also found ways to use social media information and cell phone data as part of the analysis, even in places far afield. Like others he attaches a bit of mysticism – or 'capacity to drive good' - to the concept 'data' . Data as a lynchin-pin for a  movement is growing. And the Harvard crew is emblematic. 'Improved statistical & computational methods-not in growth of storage or computational capacity'  – Jack Vaughan

Data Gumbo for the Week of Noxious Fumes

Predicting legislation, or Follow the money http://blog.fiscalnote.com/2014/04/22/legislating-todays-science-fiction-tomorrow/

Don't touch that dial! Catch Jack Vaughan speaking with Nicole Laskowki about #GartnerBI on Talking Data podcast. bit.ly/1k4ef0y

A majority of financial enterprises (67%) present a "repeatable" level of #bigdata #analytics maturity. -per IDC http://www.idc.com/getdoc.jsp?containerId=prUS24808014

'Improved statistical & computational methods-not in growth of storage or computational capacity' http://harvardmagazine.com/2014/03/why-big-data-is-a-big-deal

Read Wayne E's big data/data warehouse clash http://bit.ly/Rwc5NY  Insightful: Reminds of other techno shifts w incumbent painted all bad!

Big data: big mistake? -  http://on.ft.com/P0PVBF  via @FT 'Big data” has arrived-big insights have not.'

Cringely: Big Data is the new Artificial Intelligence http://betane.ws/s0Am  via @BetaNews

Does #bigdata improve contextualization of science? http://philsci-archive.pitt.edu/9944/1/pietsch-bigdata_complexity.pdf

The Parable of the Google Flus: Traps in Big Data Analysis

Toward a Vision: Official Statistics and Big Data http://magazine.amstat.org/blog/2013/08/01/official-statistics/

What is going on with data management technology?

What is going on with data management technology?

One thing is pretty true: Information of all types is engulfing the corporation. There’s more…

• Web apps and distributed cloud computing have grown.
• And a slew of new technologies has arrived to help companies cope with the data influx and distributed data processing …
• … But sorting through those technologies is tough.
• And you can’t start fresh unless you are a startup.
• If you are established, you worry about startups taking your business …
• But the original ‘queriable’ relational database approach is still valid.
• Newbie software has to adapt too .. add old style capabilities, and vice versa.

Data privacy stories on SearchBusinessAnaltyics

Snowden by link up talks with Euros

My SearchBusinessAnalytics.com colleague Ed Burns has been at work on a fine series on emerging data privacy issues. In Data collection practices spark debate on big data ethics, privacy and Laws leave gray area between big data and privacy he paints a picture of the current landscape (and more is on the way).  One thing that emerges, and this is something that Ed speaks with me about on an upcoming Takling Data podcast: there is as sort of entropy going on here – it skews toward the status quo, which is, in broad brush, there really is no such thing as privacy and in too many cases your data is my cash cow. We've heard plenty of talk about the big data gold rush, and data as the new oil – what that translates into is some lip service to the notion of a data self. Burns and I are enthusiasts for the new possibilities of data but both of us I think suspect that data and analytics professionals have to be sure to treat what they do as a profession, and consider the ethics of data mining, as they would any other kind of mining. I think his series on privacy is a good step in laying the ground work for a discussion around this. Does the industry need another Snowden event to wake up to the need for ethical  standards? -Jack Vaughan

 Data collection practices spark debate on big data ethics,privacy - SearchBusinessAnaltyics.com

 Laws leave gray area between big data and privacy - SearchBusinessAnaltyics.com

The Unknown Known – On Rumsfeld's ridiculously sublime rumination

 


Nate Silver's well regarded The Signal and the Noise (2012) included a chapter in which the author intrepidly  gumshoes it to Donald Rumsfeld's office, mostly to discuss the former secretary of war's long-running enthusiasm for a little known 1962 book, Pearl Harbor: Warning and Decision by Roberta Wohlstetter.  Actually the greatest enthusiasm may be that which Rumsfeld held for the book's introduction, one penned by economist Thomas Schelling who wrote: "There is a tendency in our planning to confuse the unfamiliar with the improbable."

Before Pearl Harbor, the U.S. expected sabotage from Japan, but not a six-carrier air attack from the north. This formed what Silver might describe as a 'signal and noise' moment when a massive trove of information was not effectively sifted – and Pearl Harbor was not predicted.  There would seem to be a lesson in analytics there somewhere.

Rumsfeld somewhat famously circulated the book in Washington months before the Sept 11 2001 terrorist attacks, and he has a Xeroxed copy of the forward at hand when he meets Silver. After the fact, the Wohlstetter book's  theme seemed applicable, in Rumsfeld's – and, perhaps, Silver's - estimation, to Sept 11. And it may have formed a backdrop for Rumsfeld's ridiculously sublime rumination on known knowns, known unknowns and unknown unknowns, another  variation of which (the unknown known) form the title of Errol Morris' new film, which is what I came here to tell you about.

I call Rumsfeld's 'unknown unknown' wordsmithing ridiculously sublime because, upon viewing Morris' film, I conclude that Rumsfeld's 'understanding' of the Pearl Harbor lesson was more misunderstanding – was more a willful, spiteful and devilish confabulation of analytics. He took a bit of truth and with some technical exactitude mis-applied it to the case of Iraq and its purported troves of weapons of mass destruction, for his larger purpose (political bias) of, well, say, shaking up the Middle East. He took the idea that the Pearl Harbor debacle was caused by failure of imagination, and imagined a fabled debacle all his own. Prediction provides some very special care, evoking a rework of Bob Dylan line: "To live outside of time you must be honest."

"The Unknown Known" is not quite on par with Morris' portrait of Viet Nam era Defense Secretary Robert McNamara as a film and a story, the protagonist elicits less empathy in this viewer, but it is worthwhile in its probing pursuit of logical understanding – in its analysis. Also, like the earlier 'Fog of War', it has some nifty animation. - Jack Vaughan