Entropy: 50 Shades of Smarty Pants

If a chatbot writes a complete essay from a short prompt, the entropy of the essay must be exactly that of the initial prompt, no matter the length of the final product.

— Nassim Nicholas Taleb (@nntaleb) September 8, 2023

NVIDIA Special Address at CES 2023

Digital Twin Spin

Another episode of I Read The News Today Oh Boy

InSAR Earth Eye - Some folks are thinking about twins in space A Research Profile looks at Application of satellite technology in infrastructure monitoring using Satellite Interferometric Synthetic Aperture Radar (InSAR) monitoring. The site is that of a 5 year UK program that dutifully shut down when it reached five years. Good sports, what? From <https://www.cdbb.cam.ac.uk/news>

ESG could be twinish. Since it monitors systems. In realtime maybe. And even as a RT market. ESG has its failings. Its honest critics who see Green washing is real. ESG also has its enemies – and these include Oil Loving Texans. A vision of the future is blurred when the Texas Public Policy Foundation, backed by oil and gas companies and Republican donors, enters the fray. From <https://www.nytimes.com/2022/12/04/climate/texas-public-policy-foundation-climate-change.html>

 

Supply chain twin view – From the MIT Digital Supply Chain Transformation lab. In Sloan Review it is written: “Digital twins observe their physical environment through a network of sensors that dynamically gather real-time data; they evolve by learning from this information and its contexts and by interacting with humans, devices, and other networked digital twins. Such a capability makes digital twins active and social tools, because they can continuously communicate and collaborate with their associated physical and digital objects and with humans. Digital twins support end-to-end visibility and traceability, enabling supply chain practitioners to spot patterns of highly complex and dynamic behavior.” From <https://sloanreview.mit.edu/article/unlocking-the-potential-of-digital-twins-in-supply-chains/>

 

Twins go to the city, or let’s get civil - At Columbia University, a three-year project — Hybrid Twins for Urban Transportation: From Intersections to Citywide Management — that began in 2021 is working to create a digital twin of key intersections and other locations in New York City. It is not alone. Other civil engineering projects involving digital twins include a computer model being created of the Houston water system and a sensor-based, real-time decision-support system used by the city of South Bend, Indiana, to better understand the hydraulic conditions in its sewer system. There’s more:  Civil engineers and others have also formed a new organization, the Coalition for Smarter Infrastructure Investments, to promote greater use of digital technology in infrastructure projects. From <https://www.asce.org/publications-and-news/civil-engineering-source/civil-engineering-magazine/article/2022/09/in-nyc-digital-twin-project-tackles-traffic> JV

 

RustNYC - Rewriting a high performance Vector Database in Rust with Jack...

The C word - and more

Song Han and Yoshua Bengio:

Y.B>: The C-word, consciousness, has been a bit of a taboo in many scientific communities. But in the last couple of decades, the neuroscientists, and cognitive scientists have made quite a bit of progress in starting to pin down what consciousness is about. And of course, there are different aspects to it. There are several interesting theories like the global workspace theory. And now I think we are at a stage where machine learning, especially deep learning, can start looking into neural net architectures and objective functions and frameworks that can achieve some of these functionalities. And what's most exciting for me is that these functionalities may provide evolutionary advantages to humans and thus if we understand those functionalities they would also be helpful for AI.

Related -
Full transcript
Global workspace theory

Name that tune, Now Playing!

A recent note on the Google AI blog discusses the company’s use of a deep neural network for music recognition on mobile devices. As it brings extreme-scale noodling (convolution) to bandwidth limited devices (smart phones) it could be a breakthrough on par with MPEG and JPEG, which dramatically transformed music distribution beginning in the 1990s. It’s known as Now Playing, and it can use a sequence of embeddings that run your music against its network and recognize the song, while conserving energy on the device. Each embedding has 96 to 128 dimensions. An embedding threshold is raised for obscure songs – which is the town where I live. I guess when you look at what Google has done with Search, it shouldn’t be that surprising – but the idea that so much of the work occurs on the Thing (device), is pretty astounding. I  asked it ‘what’s that song’ and it got it right. Slam dunk. “Ride Your Pony” by Lee Dorsey. Now, Shoot! Shoot! Shoot! Shoot!  Jack Vaughan

RELATED 

https://ai.googleblog.com/2018/09/googles-next-generation-music.html  - Google AI Blog, Sept 14, 2018

Speaking of Name That Tune – why not a little vignette from the time when Humans Walked the Earth?

How well can neurals generalize across hospitals?

Which features in any quantity influence a convolutional neural network’s (CNN’s) decision? To find the answer in radiology, work is needed, writes researcher John Zech on Medium. The matter gains increased importance as researchers look to ‘go big’ with their data, and to create models based on X-rays obtained from different hospitals.

Before tools are used to crunch big data for actual diagnosis "we must verify their ability to generalize across a variety of hospital systems" writes Zech.

Among findings:

that pneumonia screening CNNs trained with data from a single hospital system did generalize to other hospitals, though in 2 / 4 cases their performance was significantly worse than their performance on new data from the hospital where they were trained.

he goes further:

CNNs appear to exploit information beyond specific disease-related imaging findings on x-rays to calibrate their disease predictions. They look at parts of the image that shouldn’t matter (outside the heart for cardiomegaly, outside the lungs for pneumonia). Initial data exploration suggests they appear rely on these more for certain diagnoses (pneumonia) than others (cardiomegaly), likely because the disease-specific imaging findings are harder for them to identify.

These findings come against a backdrop: An early target for IBM’s Watson cognitive software has been radiology diagnostics. Recent reports question the efficacy thereof. Zech and collaborators’ work shows another wrinkle on the issue, and the complexity that may test estimates of early success for deep learning in this domain. - Vaughan

Related
https://arxiv.org/abs/1807.00431
https://medium.com/@jrzech/what-are-radiological-deep-learning-models-actually-learning-f97a546c5b98
https://en.wikipedia.org/wiki/Convolutional_neural_network
https://www.clinical-innovation.com/topics/artificial-intelligence/new-report-questions-watsons-cancer-treatment-recommendations

these deep neural nets just sort of keep getting deeper and bigger

Making intelligence intelligible with Dr. Rich Caruana https://t.co/s3jGUR4QRV

— Jack Vaughan at TT (@JackVaughanatTT) June 5, 2018

hard to open up those many layered neurals.

to wrap your head around a hundred million weights.

that's harder to udnerstand compared to linear regression.

these deep neural nets just sort of keep getting deeper and bigger.

cc: ummings

Cybernetic Sutra

I'd had an opportunity in college days to study comparative world press under professor Lawrence Martin Bittman, who introduced BU journalism students to the world of disinformation, a discipline he'd learned first hand in the 1960s, before his defection to the West, as a head of Czech Intelligence. We got a view into the information wars within the Cold War. This gave me a more nuanced view of the news than I might otherwise have known. Here I am going to make a jump. 

I'd begun a life-long dance with the news. 

I'd also begun a life-long study of cybernetics. 

And lately the two interests have begun oddly to blend. 

It was all on the back of Really Simple Syndication -RSS- and its ability to feed humongous quantities of online content in computer-ready form-It made me a publisher, as able as Gutenberg, and my brother a publisher, and my brother-in-law a publisher, and on ...

Cybernetics was a promising field of science that seemed ultimately to fizzle. After World War II, led by M.I.T.'s Norbert Wiener and others, cybernetics arose as, in Wiener's words, "the scientific study of control and communication in the animal and the machine."

It burst rather as a movement upon the mass consciousness at a time when fear of technology and the dehumanization of science were a growing concern. - As the shroud of war time secrecy dispersed, in 1948 penned Cybernetics, which was followed by a popularization.

Control, communication, feedback, regulation. It took its name for the Greek root cyber. Wiener - Brownian motion - artillery tables - development of the thermostat, autopilot, differential analyzer, radar, neural networks, back propagation.

Cybernetics flamed out in a few years, tho made an peculiar reentry in the era of the WWW. Flamed out but, somewhat oddly, continued as an operational style in the USSR for quite some time more. Control, communication, feedback, regulation played out there somewhat differently.

A proposal for a Soviet Institute of Cybernetics included "the subjects of logic, control, statistics, information theory, semiotics, machine translation, economics, game theory, biology, and computer programming."1 It came back to mate with cybernetics on the web in the combination of agitprop and social media, known as Russian meddling, that slightly tipped the scales, arguably, of American politics.

1 http://web.mit.edu/slava/homepage/reviews/review-control.pdf