Can the web help fight corruption?

October 21st, 2007

Lessig turns his keen insight and trademark presentation style towards the analysis of corruption as he steps away from 10 years of focusing on intellectual property issues and their impacts on culture, creativity and innovation. I wish him luck as he will have the full force of the political and business establishments fighting him to their eventual unlamented demise.

For a quick take on the hour long lecture, skip to 8:23, a story about how the sugar institute managed to influence the food nutrition board (FDA) to set the recommended maximum intake of sugar to 25% of your daily calories, instead of the WHO’s advised maximum of 10%. Hang on through 10:30 where he talks about how lobbyists have skewed the global warming “debate.” Other issues he covers are the pharmaceutical industry targeting doctors with “bribes that are not considered bribes” and the influence of private interest funding on “scientific” research. His bottom line is that everywhere we look, money determines decisions more than ever, often trumping what is in the interests of the citizens.

It all seems depressing, but what gives Lessig and the rest of us hope for democracy is the potential of the internet, particularly the web, to render the forces behind policy decisions more transparent and available to a wider population of stakeholders, namely the citizens (I know this is a shocking concept!). Perhaps the availability of hitherto inaccessible information will inspire more of the population to become politically involved. I’m not yet holding my breath on this one, but let’s dare to dream.

Towards the end of the lecture, Lessig highlights MapLight which exists to expose the relationship between legislators voting patterns and their sources of funding in US congress. Also featured is the Sunlight Foundation:

The mission of the Sunlight Foundation is to use the transformative power of the Internet and new information technology to enable citizens to learn more about what Congress and their elected representatives are doing, and thus help reduce corruption, ensure greater transparence and accountability by government, and foster public trust in the vital institutions of democracy.

Aside: Canadians should check out the Data Libre project.

Corruption is a problem that is probably a human condition arising from social interaction and economics, unlikely to disappear, but perhaps like many of our weaknesses, it can be minimized. We are in the earliest days of the development of new tools that combine openly accessible data, visualization technology, and the reach of the web. It remains to be seen if they will help improve our democracy or become yet another system to be influenced and gamed by special interests.

The Web as operating system

October 19th, 2007

Alistair, who has been writing recently at GigaOM, puts together the most lucid synthesis I have seen on the topic of the “Web OS.”

Big Internet companies are making themselves the OS of the web 2.0 world. In addition to the fundamentals — operating a web application, storing data, handling logins — each company has a core expertise. In Google’s case, it’s page ranking and relevance; Facebook maps social relationships; Salesforce knows about customer relationships; and eBay has an auction and reputation engine.

Each of these web OS service platforms by their natures can in theory interoperate, and competitors can in theory be mixed and matched. Alistair points out that developers who use these sites as platforms still have tenuous relationships with the terms of use (such as the vibrant Facebook developer community). This will be pretty contentious, as many of the big web site “platforms” have incentives to build their own apps. But all software platform companies realize that developer communities are the key to success, and if Facebook doesn’t find the right balance of developer rights and incentives someone else will. In an ideal world we assume a free market where the user can basically choose their web OS options from a range of competitors. The question is how much competition can the market sustain (can there be multiple platforms for each service (who competes with eBay?) and can there be multiple winners within the ecosystems that run “atop” those platforms. I think the answer to the latter is yes, but my concern is with the former (competition for platforms) and how much collusion there will be between the established “components” to band together monopolistically (eBay and Facebook sitting in a tree), which has been a bit of an issue in the software world.

In a slight digression I leave you with the following totally awesome video on the nature of information and how the web is setting it free:

Can code be bad for the planet?

October 14th, 2007

Alistair has an interesting post on Earth2Tech, the thesis of which is that inefficient coding practices can lead to environmental harm.

I almost misled myself into thinking AC is blaming virtualization and SaaS/IaaS (infrastructure as a service) on creating such inefficiencies. Rather, he skipped past the obvious environmental benefits of server consolidation (improved resource utilization) and service centralization (via economies of scale) and instead builds on the consequences: code inefficiencies become more obvious when you’re no longer massively overprovisioning hardware. This is an opportunity as much as a challenge. Bad code did matter previously, but not to the extent that is forseen: we’re scaling web applications to a much larger degree than ever before. There are more users, and inefficiencies are multiplied.

So it is great news that virtualization and on demand infrastructure will allow us to focus more on code efficiency since as Alistair (incidentally a veteran in the monitoring of applications) points out, it exposes more granular economics of computing. These technologies are paving the way to greater infrastructure efficiencies and by forcing better utilization of hardware, putting more focus on the efficiency of the code that cohabits the infrastructure.

Increasing code efficiency has been generally unimportant except in edge cases. Stability and function has been more of a concern while Moore’s law and incomplete costing of infrastructure have been more than compensating for performance. Rapid application development platforms proliferate based upon the abilities of modern hardware to crunch “affordably” through the multiple layers of abstraction.

What me worry?

There’s definite potential for code to have an environmental impact. We have an existing ecological disaster on our hands with the castoff personal computing hardware of both enterprises and consumers. Almost all of that computing power was wasted idling, never used, just for the ability to load Microsoft Office applications quickly. How do we acheive more efficient code? As more people rely more on computing, the costing of which is becoming more accurate and granular, and as the barriers of entry for developers drop, we should witness an evolutionary process at work battling inefficiency assuming:

  • large population of users
  • competing applications
  • rapid generation spans with modification
  • market exerts selective pressure

While I think these evolutionary forces are already at work, the selective pressures have been weak, the environment has been overly abundant leading to a Cambrian explosion of inefficiencies that eventually will be represented in costs that the market will react to, assuming that the market has the freedom to do so. This is where intellectual property issues and the “one platform to rule them all” attitude may present a bit of a speedbump, but only that.

Alistair’s most important point is highlighting that proper costing of computing is essential: if we want to minimize environmental impact we need to measure the efficiency of work performed by applications and the true cost of the resources they consume. My conjecture is that an evolutionary process of anthropogenic artificial selection, automated or not, should optmize resource utilization. This rests upon the premise of a competitive market, which I believe we are just starting to see in the world of software.

For now I’m much more concerned with how poorly conceived code can compromise privacy, and restrictive code that restricts our freedom of communication and innovation. But those are stories for another post.

The world’s on fire

August 8th, 2007

The funds that Sarah McLachlan would have spent to produce a music video are donated for more practical uses, and the video is instead used to help us visualize just how much of a difference $150,000 can make to those less fortunate than us:

I’m a long time fan and proud she is a fellow Canadian.

Infreemation: just the facts

July 2nd, 2007

Apologies to RSS subscribers who will no doubt be affected by my blog consolidation. I’m pulling in all of the posts from www.infreemation.net and rebranding this blog Infreemation. Why the consolidation? Most of what I write in both blogs is about information: technology, policy, analysis, visualization. I haven’t been making too much time lately for my blogs, as I was making time for growing my businesses (more on this later), attending lots of interesting conferences (Mesh, Interop, Web2Expo) and most importantly…getting married and disappearing on a “lune de miel!” I’ll be blogging professionally soon which will move some of the more internet-geek type posts out of the personal blog and allow me to talk about my tangential, hobbyist interests here.

For the one or two of you who are still with me, let me reward you with one of my favorite exchanges caught on film as inspector Clouseau schools his sidesick in some of the finer points of deductive reasoning:

Clouseau: Facts, Hercule, facts, behind them lays the whole fabric of deductive truth. Now, Hercule let us examine these facts: 1 [holding up 1 finger]
Clouseau: she was found with the murder weapon in her hand, 2 [holding up 3 fingers]
Clouseau: the murder weapon was fresh with blood, 3 [holding up 4 fingers]
Clouseau: there were no fingerprints on the murder weapon other then hers and 4 [holding up all 5 fingers]
Clouseau: all the members of the Ballon household staff have perfect alibis. Now then, Hercule what do these facts add up to?
Hercule LaJoy: Maria Gambrelli killed Georges the gardener.
Clouseau: You are an idiot, only a fresh faced novice would come up with a conclusion like that.
Hercule LaJoy: But the facts.
Clouseau: Listen, who even killed Miguel, killed Georges the gardener and he did it to cover up the first crime. Now what he is trying to do is lay the blame at the foot of this, this poor servant girl.
Hercule LaJoy: Well who do you suspect?
Clouseau: I suspect everyone.
Hercule LaJoy: Well I suppose that is possible.
Clouseau: Possible? What do you mean possible? I deal in certainties.

I guess you kinda have to see it for yourself…If you haven’t already I recommend checking out “A Shot in the Dark”

Owning the rainbow: why spectrum can and should be freed

February 28th, 2007

Lessig’s presentation on the control of US airwaves is brilliantly clear, as usual:

An inconvenient graphic – visual analytics

February 12th, 2007

Some graphics from An Inconvenient Truth have been posted to Flickr. Copyright has most certainly been violated here but I suspect Gore would prefer the message spreads and perhaps this will inspire more folks to check out the movie.

An Inconvenient Graph

This graph showing a sharp rise in CO2 has generated a lot of controversy, especially because of the extrapolation of future CO2 levels. For an excellent analysis of the controversy and a discussion about how visualization approaches such as Gore’s slideshow can be used to turn complex data into information, look no further than David Womack’s excellent article: Seeing is believing: Information visualization and the debate over global warming.

Excepted from the above:

Information visualization is able to communicate the intricacies of global warming in a way no other discipline can. Its messages can be immediate and powerful, without sacrificing the level of detail necessary to represent the complex subject accurately. Not only is information visualization helping scientists and politicians communicate with the public, it is a primary tool for scientific study, and for the study of science itself. It is particularly telling that even the medium of film could not compete with the power of these visualizations. According to “An Inconvenient Truth” director Davis Guggenheim, “I thought, a film about a slide show? A filmed lecture? I don’t get it. And then I saw his slide show. The information in it is so powerful, and we all just felt like, what if we could give people a front-row seat to this.”

Wider adoption of visual analytics, while useful for scientific analysis and “misuseful” or easily manipulated for special interests, will at least stimulate wider debate and analysis by increasing the accessibility of data to society (see Gapminder). And this, I dare to believe, will lead to good things.

The internet is us

February 6th, 2007

Great video visualization of the evolution of the web:

via infosthetics.com as usual…

Taking ownership of information highways

February 4th, 2007

Bob Frankston continues his exploration of alternative information infrastructure ownership. He wonders, with the media (copper wires, fiber optics and radio waves) essentially commodity, what do we get from services providers that we cannot more efficiently provide for ourselves? With absurdly high communications bills of late, I can’t help but agree that there is a more cost effective model:

The idea is very simple. Our connectivity infrastructure is made up of Copper, Fiber and Radios and is a fixed asset that requires a small amount of maintenance. In the days of telegraphy and telephony it was deployed as a means of selling services.

Today the Internet and the networks in our homes make it clear that given the CFR we can do our own networking. Yet we fund the CFR by letting the incumbent service providers maintain their privileged ownership.

Funding the infrastructure by selling services makes no more sense that having men with pikes charging for use of their personal highways.

Issues like network neutrality are symptoms of far deeper problems with this model.

Continued here.

a-ma-zing

February 2nd, 2007

Very Minority Report (the movie that is). Apparently this amazing touch screen technology will soon be in mass production. I don’t know if that means it will really be affordable to most of us but I have no doubt it will be integrated into Smartboard-like devices and other presentation environments. Also potentially interesting for analysts where data visualization is an important tool to understand and explore relationships (see Intelligence 2.0).

Video is Jeff Han presenting at TED.

The web as a pidgin internet language

January 28th, 2007

or “How to Stop Worrying and Learn to Love the Internet”

A disproportionate amount of brilliant stuff was written by Douglas Adams, who in 1999 had this to say about the internet:

the biggest problem is that we are still the first generation of users, and for all that we may have invented the net, we still don’t really get it. In ‘The Language Instinct’, Stephen Pinker explains the generational difference between pidgin and creole languages. A pidgin language is what you get when you put together a bunch of people – typically slaves – who have already grown up with their own language but don’t know each others’. They manage to cobble together a rough and ready lingo made up of bits of each. It lets them get on with things, but has almost no grammatical structure at all.

However, the first generation of children born to the community takes these fractured lumps of language and transforms them into something new, with a rich and organic grammar and vocabulary, which is what we call a Creole. Grammar is just a natural function of children’s brains, and they apply it to whatever they find.

The entire article is a great read, along with most of what he wrote in his lifetime.

A dark matter: structure of the universe influenced by dark forces?

January 19th, 2007

The indispensable New Scientist reports that

the distribution of dark matter has been mapped in 3D for the first time, revealing how the mysterious substance has evolved over the lifetime of the universe. The results confirm that dark matter provided the scaffolding that allowed ordinary matter to clump together to form galaxies and clusters of galaxies.

They provide this interesting visualization: an evolution of the distribution of dark matter over time. It shows that the dark matter is, similar to other matter, getting “clumpier” over time:

3D visualization of dark matter

An impressive video showing a 3D visualization of the distribution of dark matter over time is available in hi-res on the European information page for the Hubble Telescope: here. You can view it quickly below.

New Scientist also reports:

The behaviour of the Bullet cluster – the poster-child for the existence of dark matter – is provoking some cosmologists to propose that there might be a fifth fundamental force.

This hypothesis is based on preliminary calculations that a cluster of normal matter is accelerated by the presence of dark matter 20 percent more than expected based on the normal force of gravity. Skeptics abound and Occam’s razor is likely to prevail.