Thursday, November 24, 2011

How to make sense of Big data

The advance of database technology and hardware enables us to built larger and larger data warehouses with massive amounts of data - big data. The increasing demands from financial transactions, network connections, and other areas makes this a requirement. Yet, trying to make sense of this data is like trying to take a sip of water from a fire-hose. The quantity of data is just too large to review using tables. Even data-mining requires hypothesis formation to write the query. The only way to access this data in a useful way is through visualization. Initially, this has been through graphs and diagrams. However, these are relatively clumsy and require reference back to the tables and charts. Our knowledge of human perception has advanced, along with algorithms and the capability of the Graphical Processing Unit to the point that Synerscope presents big data in a way which the human eye and brain finds natural. This means that humans can draw meaning from visualizations that hold millions of pieces of data.

Wednesday, November 23, 2011

The successful, incurious man or woman

There are many people who are successful without being curious, yet others are. You could not create iPods were you not curious, but you might be able to be a successful commodity trader. Curiosity is laudable, like virtue, in itself, not because of any material reward it brings. I do find it more interesting to be in the company of curious people, but those who are not may be just as decent, or perhaps more so. There is no correlation.

Monday, November 21, 2011

Magical thinking is not just for children any more.

So often we see intelligent adults assume that you can get the behavior and process you want by a simple change in one element. Whether this is as simple as an assumption that transferring policing or fighting wars to private enterprise will make it more effective and efficient, or that changing electoral systems will create greater democracy, it is usually too simplistic to be true. Both government and private enterprise have internal as well as external pressures. Many of these hurt efficiency as others help it. Voting systems can create greater democracy, but only if other conditions are present. To assume that making one simple macro change will automatically improve the way things are done is magical thinking of the sort that is normally found in 5 year old children.

Sunday, November 13, 2011

Is the BBC'S impact in the US growing by the personal presence of Katty Kay?

The BBC has long had a positive image in the US. Known for impartial news and high quality programs, whether entertainment or informative, it has nevertheless not been a major presence in the US. With the introduction of two cable channels, that has started to change. Yet, that is not itself enough. However, it seems that the main news anchor on BBC news America, Katty Kay, is everywhere these days. On Sunday morning talk shows, authoring a book, and speaking at conferences, she has made the BBC real to many Americans. If only there were more people like her showcasing the BBC, it would be more watched and listened to. Which does beg the question of why when one person does it so well, why are there not more people doing it at all?

Friday, November 04, 2011

Kachingle is a new model in monetizing the web

Kachingle,, is a new model that allows readers to reward content they like by making micro-payments. The reader sets a monthly budget, say $5 or much more. Then when they see a site they like, they click on the button to send a proportion of that to the site. It is more like NPR than the Wall Street Journal, as it relies on an honor system, but it does seem to work well.

Infrastructure is the reason why the US is crumbling

The recent snowstorm in the Northeast highlighted our crumbling infrastructure. 50% of homes in CT without power four days later. Apparently, CT is three times as forested now as it was in 1985. So, if you insist on hanging power lines from poles, the likelihood of them being brought down by falling trees has gone up several times. Add to that a 20% reduction in work crews and a reduction in regular tree trimming (now done only once every five years) and power outages are inevitable. Add to that, reductions in maintenance of roads, bridges, railroad tracks and rolling stock, and the US is the only developed or developing country where infrastructure is getting worse, not better.

The inevitable conclusion, if maintenance and repair continues to fall behind is continued erosion of infrastructure. Thinking ahead, as we used to in the 50s, 60s, and 70s, would prevent this from happening.