Molecular sugar simulations on Gene/Q

Researchers working with an IBM supercomputer have been able to model the structure and dynamics of cellulose at the molecular level. It is seen as a step toward better understanding of cellulose biosynthesis and how plant cell walls assemble and function. Cellulose represents one of the most abundant organic compounds on earth with an estimated 180 billion tonnes produced by plants each year, according to an IBM statement.

Using the IBM Blue Gene/Q supercomputer at VLSCI known as Avoca, scientists were able to perform the quadrillions of calculations required to model the motions of cellulose atoms.

The research shows that there are between 18 and 24 chains present within the cellulose structure of an elementary microfibril, much less than the 36 chains that had previously been assumed.

To download the research paper visit: http://www.plantphysiol.org/

To find out more about the Australian Research Council Centre of Excellence in Plant Cell Walls visit: http://www.plantcellwalls.org.au/


http://www-03.ibm.com/press/us/en/pressrelease/46965.wss

Data Journalism Hackaton

Took part in NE Sci Writers Assn Data Journalism Hackaton at MIT's Media Lab in April. iRobot Inc HQ! We tried to visualize a data story on California water crisis. 

Tool was iPython notebook. [I got 1+1= to work!] [How cool is this?!] Came up short but learned a lot about manipulating data along the way. My colleagues were par excellance. Greatest fun I know is to be part of a team that is firing on all cylinders. Gee it looked nice outside, tho. Playing hooky on part 2!   - Jack Vaughan





code-workshop.neswonline.com || CartoDB || geojson.org
ipython.org/notebook.html || more to come

5 minute history of the disintegration of application server

Cloud Computing - HorizonWatch 2015 Trend Report by @horizonwatching http://t.co/wjzbU2x1vD via @SlideShare

— Jack Vaughan at TT (@JackVaughanatTT) May 12, 2015

http://t.co/6yhhn9J1RT Fowler on friend look at Microservices

— Jack Vaughan at TT (@JackVaughanatTT) May 12, 2015

Dockercon State of the Art in Microservices by @adrianco #cloudcomputing #continuousdelivery http://t.co/Fi794TkzFY via @SlideShare

— Jack Vaughan at TT (@JackVaughanatTT) May 12, 2015

Introduction to Node.js by @vikasing #javascript #nodejs http://t.co/nAHVPHuFQo via @SlideShare

— Jack Vaughan at TT (@JackVaughanatTT) May 12, 2015

Telling winds from the cybernetic past

http://itsthedatatalking.blogspot.com/2015/01/the-best-laid-plans-of-mice-and-man.html http://mitpress.mit.edu/books/cybernetic-revolutionaries http://www.newyorker.com/magazine/2014/10/13/planning-machine https://www.jacobinmag.com/2015/04/allende-chile-beer-medina-cybersyn/

 Does a 1970s Utopian technology effort offer useful guides for those trying to assess the progress of new technology today? In one case, at least, yes. It is the story of Salvador Allende's attempt to build a working Socialist government in Chile with computer cybernetics.

The tale is told especially well, under the able hands of author and researcher Eden Medina. Medina rolls up the takeaways in a recent article in Jacobin magazine. It is a summary of some important lessons garnered during work on her 2013 book, The Cybernetic Revolutionaries.

You see, before CIA influencers sponsored Augusto Pinochet and company's junta, Allende's democratically government was trying to bring a new form of socialism that was data driven. In those days, what might pass for the big data enterprise today would be called cybernetics. This school of technology, founded by Norbert Wiener, studied feedback in systems, be they animal or machine. The automatic pilot was perhaps cybernetics crowning achievement. In the Chile case, technologist Stanford Beers was enlisted to bring the magic of realtime feedback to state planning. It was way ahead of its time, and burdened by lethal sniping.

 A chief lesson in all that conflag is that the state and its priorities shape how a technology is designed and used. In Allende's work to create a better state planning system based on the infant cybernetic architectures, Beers was given had a lot of rein to try and involve workers, ahead of engineers and government bureaucrats in the planning of production. Uber advocates might say that is going on with its upsurge today, though, we'd say, that is arguable.

"Computer innovation wasn’t born with Silicon Valley startups, and it can thrive by taking on design considerations that fall outside the scope of the market," writes Medina. Yet, the basic lesson is tremendously true: technologies get no more freedom to range than the political system gives them. That lesson may be taught at MIT, but it is largely buried in the footnotes or drowned out by the gush of venture capital, and its dreams.

Read more on this.

Spark stories

I feel as though I have never seen anything quite like Spark before.  It seems more than worthy of substantial media coverage, but it is also cause for pause.

I only came to the big data beat in 2013. So I didn’t go through all the run up of hype on Hadoop. I came in when it was in full swing – it seemed natural, and some enthusiasm was warranted. But, as Hadoop 2 was rolling out, and Spark was striding into view, I said, this town is not big enough for the two of them – that Hadoop had taken all the air out of the hyperbolic chamber. Is it or is it not just the new shiny thing in that room?

Now I wonder. Yes, the Hadoop people dutifully over time explained what was wrong with Hadoop (Mostly MapReduce). But as with technology marketing trends generally, it begged the answer. Now Spark seems like the answer.  Hadoop greased the skids for it. I guess one reason is that MapReduce was limited. But isn’t Spark limited too, if you look at it from many miles remove?

I boil down Spark's plusses to:

1-It includes more developers. Because it offers support for Python and Scala as well as Java. And runs on the Java Runtime.

2-It runs faster.

Now if you slowly walk away from that car you could say: its patrons face big obstacles in catching up with the Hadoop commercial train. And, it only has a few users at this point. We could go further and say "It is had more general utility than MapReduce, and seems more apt for analytics. Tony Baer has written that Spark use cases seem to be about complex analytics on varied data than about big data per se.

That is a delicate distinction, but probably worthwhile noting. So many technologies thrive on rend asunder based on a few delicate distinctions that tend not to be readily apparent.

I duly note your sense in a tweet that the brunt of use cases described at the sum. It think Spark and Hadoop borth are about developer centric solutions for cheap parallelism WITH focus on data processing.

There was a time when this all was called Data Processing, then there was Information Technology. Now, Data processing is back. - Jack Vaughan

Users view Databrick's Spark
It is in limited beta. But a lot of people have gotten their hands on it.

Hadoop and Spark are coming of age together
The Talking Data podcast features Hadoop and Spark, open source data technologies that gained attention at this year's Strata+Hadoop East event.

 Apache Spark : This year's MapReduce killer
Since the release of Apache Spark, big data vendors have touted it as faster, more flexible alternative to MapReduce.

Apache Spark meets the PDP-11
Apache Spark seems ready to upstage Hadoop. But it's best seen in the light of computing history, where it looks like yet another step on the long road of data.

Apache Spark goes 1.0, looks to improve on MapReduce ...
The Apache Software Foundation has released Version 1.0 of ApacheSpark, an open source data processing framework designed to outperform MapReduce ..

Spark framework gets big push from big data vendors
The Spark framework and processing engine is attracting the attention of vendors, who are touting it for use in iterative machine learning and other big data chores.