The Books Ngram Viewer from Google Labs provides a fascinating insight into language usage in the past 200 years. An Ngram is a series of one or more items from a sequence, in this case a word or phrase from a published text. Google’s viewer plots the frequency of occurrence for Ngrams found in books published since 1800. It is possible to narrow the search to specific collections of books or corpus. Available corpora include American English, British English, English Fiction etc. Researchers at Harvard University’s Cultural Observatory have put together some tips for using this data and have invented a new word

Culturomics – The application of high-throughput data collection and analysis to the study of human culture.

A plot of major communication technologies from 1840 to 2000 illustrates the power, and pitfalls, of the viewer. There are problems with synonyms; over time television has become TV, and telephone has become phone. To get a clear view all synonyms need to be plotted. It would be nice if google offered the option to add the data for two or more Ngrams so their combined frequency could be plotted with the same line (they seem to have fixed this now). But, the real problem comes with homonyms, “cell” and “mobile” do not always mean phone. The multiple senses of the word cell, for example, make it impossible to extract the one sense that corresponds to cellular mobile phones. If plotted on the graph above the frequency of usage of the undifferentiated word “cell” swamps all the other words, I have omitted it from the plot because of this. Despite these problems the plot clearly shows the rise and fall of communication technologies in the past 150 years. Looking at this plot, and others I created with the Ngram viewer, it struck me that what is actually being measured is the social impact or visibility of each Ngram. This led to the idea that this might be a great tool to assess the veracity of Gartner’s Hype Cycles. Gartner position themselves as the world’s leading information technology research and advisory company, claiming to deliver the technology-related insight necessary to make the right decisions.

When new technologies make bold promises, how do you discern the hype from what’s commercially viable? And when will such claims pay off, if at all? Gartner Hype Cycles provide a graphic representation of the maturity and adoption of technologies and applications, and how they are potentially relevant to solving real business problems and exploiting new opportunities. Gartner Hype Cycle methodology gives you a view of how a technology or application will evolve over time, providing a sound source of insight to manage its deployment within the context of your specific business goals.

Gartner.com Research Methodologies, Hype Cycles

After some digging I found Understanding Gartner’s Hype Cycles, 2008 by Jackie Fenn for the outrageous sum of $495. Fortunately google found a free copy. The paper explains that the Hype Cycle is a combination of two curves, the first associated with early chatter about a technology, and the second associated with the actual adoption of the technology. The hype cycle is a measure of the general perception, or visibility, of a technology not its actual adoption. Gartner seem to be a little inconsistent about this, sometimes labeling the vertical axis “expectations” and sometimes “visibility”. The paper makes no mention of how the vertical axis of the two curves are normalized or why the first has a higher peak than the second. The paper has the following to say about how technologies are positioned on the curve.

Gartner analysts position technologies on the Hype Cycle based on a consensus assessment of hype and maturity. During the first part of the Hype Cycle, when there are many uncertainties regarding a technology, its position on the hype curve is guided more by its hype levels than its perceived maturity. At the later stages, as more information about maturity, performance and adoption become available, the hype plays a lesser role in determining the technology’s position on the Hype Cycle.

A technology may have radically different positions on different Hype Cycles. An enterprise’s ability to deploy a technology may exceed its ability to manage that technology well, resulting in different positions for the technology on, for example, the Hype Cycle for Application Development and the Hype Cycle for Enterprise Systems Management. In addition, different applications of a technology may lead to different positions — for example, speech recognition in the call center may be further evolved (on the Slope of Enlightenment) than speech recognition on the desktop (in the Trough of Disillusionment). Application considerations may lead to different positions of the same technology on different horizontal (for example, customer relationship management) or vertical (for example, government) Hype Cycles.

Below are two typical Hype Cycle diagrams labeled with the pompous names for the cycle phases. Trough of Disillusionment reminds me of The Pilgrims progress by Bunyan, sounding suspiciously like The Slough of Despond. Maybe Gartner, in their wisdom, think such pomposity builds credibility.

I find the Hype Cycle concept far too smug and subjective – ideal for MBA courses, but not much use in the real world. I can find no objective evidence in support of the claim that visibility or expectations for a technology are highest before it has reached more than 5 % adoption. I can find no evidence that successful technologies go through a visibility “trough” phase. Technologies appear to be positioned on the curve at the whim of Gartners “expert analysts”, and Gartner admit that different analysts will position the same technology in different places on different curves. Given all these admissions of subjectivity It amazes me that people are willing to pay for this nonsense.

A nice example of the failure of the Hype Cycle is the positioning of Microblogging in August 2010. With the benefit of hindsight it is possible to see this as completely inaccurate. Twitter was just starting to gain “visibility” in August 2010. Within 6 months Twitter had been a cited as a causal factor in overthrowing at least three governments and supporting massive turmoil to the middle east. hardly a technology in the slough of despond!

Relational Database Example

Relational database technology was first defined by Codd and Date in the late 1960’s and early 1970’s. It was commercialized in the late 1970’s by SDL which later became Oracle. The relational approach to data management became the industry standard and enabled Oracle to become one of the largest companies in Silicon Valley. As the graph below clearly shows; there was no “peak of inflated expectations” or “trough of disillusionment”. Relational databases took off and didn’t slow down for nearly 2 decades! I believe the flattening in the 1990s was probably due to their success. By the 1990s the qualifier “Relational” was no longer necessary – only non-relational databases needed a qualifier! Relational databases had become ubiquitous and banal, they were just databases.

Desktop Productivity Applications Example

Wherever there is desktop computing there are word processors, spreadsheets and email. Since the introduction of VisiCalc in 1975, through Lotus 1-2-3, and Excel, the spreadsheet has grown in popularity. Word processors like wordstar, wordperfect and Word have have become equally ubiquitous. Both these technologies emerged in the mid seventies but took off with the spread of the personal computer in the 1980s. Email has grown with the rise of the internet. None of curves for these technologies look like a hype cycle.

A Failed Technology Example

But what about failed tehcnologies? Technologies that were hyped but never made it. The technologies that Gartners analysts could save you from if only you’d listen to their sage advice. The Network Computer was a concept marketed by Oracle and Sun Microsystems in the mid 1990s. The concept defined a cheap diskless networked personal computer that downloaded its applications from the internet and stored it’s working files on internet servers. A sort of physical thin client. The interesting thing about network computers from my perspective is how long it took for the idea to die. Gartner suggest that visibility decays as fast as it grows but it clearly did not for the network computer.

I started investigating Gartner Hype Cycles with an open mind. But things started going downhill quickly. Gartner’s paper explaining the hype cycle concept cost $500 and is nothing more than flim-flam and arm-waving. Using the Google Books Ngram Viewer it is possible to plot quantified graphs of a paricular terms “visibility”. By carefully selecting terms the “visibility” of a technology over time can be plotted. These plots should look like Gartner Hype Cycles but they don’t. Gartner’s Hype cycles are a sham, the information technology equivalent of snake oil.

Evolutionary Psychology posits that many human behaviors are evolved adaptations. In his excellent books The Red Queen, and The Origins of Virtue, Matt Ridley explains the evolutionary origins of human sexuality, reciprocity, and collaboration. It is an easy mistake to assume that all common behaviors are adaptations of some kind, when in fact many are often merely side effects and confer no direct advantage. Despite this problem, the evidence that many behaviors are advantageous is compelling. With this in mind I am always on the look-out for evolutionary explanations of other behaviors, but remain wary that these may not imply adaptation.

My rating:4 of 5 stars. Pascal Boyer‘s book, Religion Explained. The Evolutionary origins of Religious Thought, starts with a fairly heavy description of how human minds use cognitive templates and define concepts, but soon starts to tackle the issue of religion itself. This book does not explain organized religion or any behavior that has arisen since the invention of writing and modern culture. Rather, the book covers deep behaviors that underpin religious thought; belief in spirits and the supernatural, witches and magic, and our fear of death and corpses. Some of the explanations are so simple and elegant that one is left thinking “why didn’t i think of that?”. For example. Our highly evolved, and measurable, ability to detect agency – to spot the signs of a predator stalking us – leads, necessarily, to many false positives. These false positives are obviously beneficial, since we avoid being eaten, but also lead to our belief in spirits and other supernatural agents. We know there was nothing real following us, but we are also convinced that there was something following us. Reflections in pools, and our own shadows, would have been mysterious to our ancestors and yet would have demanded an explanation. These may seem like trivial observations, yet, when combined, they start to re-enforce each other, and the evolutionary argument takes on some weight.

I think Professor Boyer could have strengthened his argument with a more concrete description of reciprocity. Boyer suggests that cooperation cannot be an evolutionary adaptation because it leaves the cooperator open to exploitation by freeloaders. Robert Axelrod‘s superb book The Evolution of Cooperation clearly proves that groups of collaborators can overcome, and force out, freeloaders. These groups are shown to be more successful when the have rules that encourage reciprocity, by both rewarding collaborators, and punishing defectors. These rules can be made more effective by the use of status, reputation and other labels that aid recognition. Such labels are often bestowed during rituals that are personally expensive and thereby prove group loyalty. Boyer touches on these behaviors but never really emphasizes that religious practices can provide the mechanism for strengthening group cooperation. Regardless of these problems Religion Explained is a fascinating book well worth reading.

Since reading Boyer’s book I have begun to wonder about other common human behaviors that may have some evolutionary origin. In particular I am curious about music and dance and the ritual role these play in all societies. As a subscriber to Scientific American I recently had the opportunity to pose a question to Jane Goodall and decided to ask her about this issue.

SA: John R. Harris asked via Facebook whether you have seen any behavior in primates that could shed light on the commonality of ritual music and dance in humans. Do you think these behaviors or some precursors were present in a common ancestor, or are they unique to the human lineage?

JG: Chimpanzees often perform amazing rhythmic displays, almost like dancing, when they come upon a waterfall way up in the mountains [in Gombe] that drops 80 feet onto a stony streambed and makes a roaring sound. The chimpanzees’ hair will stand on end and then they start this rhythmic swaying from side to side. It can last 20 minutes. Then sometimes at the end you will see them sitting and looking at the water, their eyes following it as it falls. If they could just talk with each other about the feelings that trigger these displays—which I believe must be something like wonder or awe—that could easily become a form of religion, the worship of the elements.

Scientific American Interview with Jane Goodall

I found the following video, with a commentary from Dr. Goodall on the Jane Goodall Institute website. The video is accompanied by a description by Bill Wallauer, a Gombe videographer, of the ritualistic behavior he has seen at Gombe.

Waterfall Displays from the Jane Goodall Institute on Vimeo.

This is tantalizing stuff but what I really want is a book entitled. The Evolutionary Origins of Music and Dance, that is written by someone as smart as Ridley or Boyer. I haven’t found it yet. There do seem to be a few people publishing papers on this area of research. Joseph Jordania seems to be fairly prolific and widely referenced, even though some of his ideas have collected some strong criticisms and his writing style seems a little colorful! He has suggested that humming is a contact-call that allows humans in a group to know they are among kin and therefore in no danger. This, he claims, explains why humming and soft musical sounds are relaxing for humans. At the first sign of danger however, the humming stops and members of the group become focused and prepared for action. He also suggests that choral polyphony is an adaptation to predation. Groups of people used loud rhythmic dissonant polyphony to scare potential predators and prepare themselves mentally for a fight.

…This was loud, responsorial singing of a large mixed group, rhythmically very precisely organized (most likely in a duple rhythm), accompanied by rhythmic movements, stomping and body percussions. The tempo rose during the singing/dancing, as well as the pitch, together with the general dynamics. Polyphony was based on ostinato, and possibly on parallelism, there was little or no text (mostly interjections), and the function of a bass was not yet separated…

…Some individual members of the group, as in every society, would be braver, and others could be more panic-stricken, particularly when facing a predator like a hungry lion. And here comes another function of the “lion dance”: relentless repetitive rhythm in a dramatic climax of standing your ground for your life against the lion must have had some kind of hypnotic effect on the whole group of stomping primates. This rhythm would unite everyone against the common mortal enemy, giving every member of the group the feel of communal safety…

…I suggest that the origins of the hypnotic drive of humans to follow the rhythmic pattern of the music with stomping, finger clicking, singing alone or just a rhythmic movement of any part of our body comes from these millions of years of relentless “rehearsals” of our ancestors under the threat of death…

Who Asked the First Question? The Origins of Human Choral Singing, Intelligence, Language and Speech by Joseph Jordania

The closest to this still practiced today may be the polyphonic singing of the Aka Pygmies of central africa which has been included in the UNESCO: Representative List of the Intangible Cultural Heritage of Humanity – 2008

All this is highly speculative, however, it starts to address fundamental questions about human nature. Religion, ritual, music, and dance, are human universals. It seems incredible to me that these are merely side effects, or cultural artifacts, that serve no evolutionary purpose. Time will tell if my hunch is correct, but the evidence that there is some evolutionary origin for these universal is starting to mount.

Zachary Sniderman on Mashable published an interesting article titled, Just How Open Is Your Internet?. I thought the featured map looked a little bit fishy – Mongolia has less internet censorship than Sweden? Really? What does that mean? It seemed to me that there might be something missing from the account. To his credit Mr Sniderman does note

…it raises some inherent problems with defining “censorship.” For example, screening out child pornography and illegal file sharing technically registers as “censorship” even though most people wouldn’t consider that a human rights offense.

Even accounting for his concerns it still looked a bit odd to me.

So I looked up the source of the data. The map is based on an analysis published by the Open Net Initiative and is based on data from a Reporters Sans Frontières (RSF) survey. I’ve been following RSF since 2003 when they got banned by the United Nations Commission on Human Rights for being righteously indignant and upsetting some of the hypocrites on the commission. I admire their work, which I think they pull off with admirable Gallic flair.

The raw data for the above map appears to be taken from the Press Freedom Index which RSF have been publishing yearly, since 2002 (2010 pdf version). The RSF methodology is published at the bottom of this page and seems sound to my untrained eye.

The RSF survey comprises 43 questions that seek to define the freedom of the press, or lack of it, for each surveyed country. Of these 43 questions just 4 deal with the internet.

Internet and New Media

1. Do the authorities control Internet service providers directly or indirectly? During this period, was there or were there (Yes/No):
2. Cases of access to websites being blocked by filtering mechanisms or being closed down by the authorities? Evaluate the level of this censorship on a scale of 0 (no censorship) to 5 (total censorship).
3. Cases of cyber-dissidents or bloggers being detained for more than a day? How many?
4. Cases of independent websites being the target of cyber-attack or counter-information campaigns?

I can’t find the ONI methodology on their website but the map appears to be based on the answers to question 41 from the RSF survey. Even if it is based on some weighted blend of the 4 RSF internet related questions it is pretty thin evidence – Especially when you consider what is left out. Below is a map, from Vis4.net showing internet usage as a percentage of total population. Most of the countries with “no censorship” on the ONI map have very low internet penetration. None of the countries in Africa, with “no censorship”, have an internet penetration higher than 10% . I think there is an obvious lesson here. It’s not worth censoring the message if no one is listening. Unless of course you are deeply paranoid.

Ultimately the RSF survey (see map below) seems to present a far more nuanced, informative, and meaningful account. Freedom of expression is about far more than internet “censorship”. It’s not just what you can read on the internet that is important, but what you can say in any public forum , and how many people you can say it to without fear of reprisal. Its about the many ways legitimate public discourse can be controlled or intimidated. Most of the countries that RSF classifies as having a “good” or “satisfactory” situation, with respect to freedom of the press, allow some censorship of the internet. As Mr Sniderman points out this is mostly blocking of child pornography and illegal file sharing, and most people would not consider this censorship.

I think the real concern is with countries whose populations have started to adopted the internet and have passed a kind of tipping point, where most people know someone who has access to the internet, my guess would be 15% to 30% penetration. Countries with this level of internet penetration, that are classified by RSF as having a “difficult” or “very serious” situation with respect to freedom of the press, seem ripe for strife. This volatile mixture of oppression and freedom exists today in; China, Columbia, Iran, Egypt, Mexico, Sudan, Burma and Vietnam, among others.

The Antikythera Mechanism was discovered on May 17th 1902, by archaeologist Valerios Stais when he was diving on the Antikythera wreck off Point Glyphadia on the Greek island of Antikythera. The wreck is believed to have sunk in the 1st century BCE and has yielded many spectacular artifacts. The most mysterious of these is the Antikythera mechanism, a solid lump of corroded bronze gears. It has taken over a century, the latest imaging technology, and decades of research from a few dedicated scholars of mechanical engineering to piece together what the mechanism did.

I put together this playlist of short youtube videos. Together they describe the latest advances in understanding the mechanism and how it worked. There are four videos that take about 20 minutes to watch. Just click on the video and all four will play.

It appears that the mechanism is at last beginning to yield it’s secrets. The most staggering being that there was nothing else remotely like it for over 1400 years! It’s like a steam engine being found in Roman Britian, or a flying machine in Baghdad before the Monguls arrived. While we are beginning to understand how the mechanism worked, there is still little information on how it was used or what it means in terms of ancient Greek technical capabilities. Nothing else like the mechanism has been found but, it cannot have been unique in the ancient world. There are several references to machines with similar capabilities in contemporaneous Greek documents. Furthermore, the capabilities of the mechanism and the techniques required to build it clearly required a well established tradition of mechanical device construction.

The recent discovery of the inscriptions on the mechanism’s dials has led to a new theory. The spelling of the month names suggest that it was constructed at Syracuse on Sicily between 150 and 100 BCE. This leads to the tantalizing possibility that it was designed, or even built, by Archimedes who died in 212 BCE when Syracuse was taken by Roman forces. Cicero (106-43 BCE) mentions that, after the capture of Syracuse, two mechanisms which showed the motion of the Sun, Moon and five planets were taken back to Rome.

Then Philus said: I am not about to bring you anything new, or anything which has been thought over or discovered by me myself. But I recollect that Caius Sulpicius Gallus, who was a man of profound learning, as you are aware, when this same thing was reported to have taken place in his time, while he was staying in the house of Marcus Marcellus, who had been his colleague in the consulship, asked to see a celestial globe which Marcellus’s grandfather had saved after the capture of Syracuse from that magnificent and opulent city, without bringing to his own home any other memorial out of so great a booty; which I had often heard mentioned on account of the great fame of Archimedes; but its appearance, however, did not seem to me particularly striking. For that other is more elegant in form, and more generally known, which was made by the same Archimedes, and deposited by the same Marcellus in the Temple of Virtue at Rome. But as soon as Gallus had begun to explain, in a most scientific manner, the principle of this machine, I felt that the Sicilian geometrician must have possessed a genius superior to anything we usually conceive to belong to our nature. For Gallus assured us that that other solid and compact globe was a very ancient invention, and that the first 371model had been originally made by Thales of Miletus. That afterward Eudoxus of Cnidus, a disciple of Plato, had traced on its surface the stars that appear in the sky, and that many years subsequently, borrowing from Eudoxus this beautiful design and representation, Aratus had illustrated it in his verses, not by any science of astronomy, but by the ornament of poetic description. He added that the figure of the globe, which displayed the motions of the sun and moon, and the five planets, or wandering stars, could not be represented by the primitive solid globe; and that in this the invention of Archimedes was admirable, because he had calculated how a single revolution should maintain unequal and diversified progressions in dissimilar motions. In fact, when Gallus moved this globe, we observed that the moon succeeded the sun by as many turns of the wheel in the machine as days in the heavens. From whence it resulted that the progress of the sun was marked as in the heavens, and that the moon touched the point where she is obscured by the earth’s shadow at the instant the sun appears opposite.

On The Common Wealth, Book I XVI. Cicero

But this only raises new questions. If the greeks could manufacture astronomical devices of such spectacular precision and complexity what else could they do? Unfortunately we are left with only a few references by ancient historians who probably barely understood and only partially believed what they were documenting. These references include; burning mirrors that could set ships ablaze, and “claws” that could sink ships, automatons, and even vending machines. Unfortunately these brief references are all we have until someone pulls up another machine from the depths of the Mediterranean. But now at least we know that each of these descriptions may in fact refer to a real technology.

My rating: 4 of 5 stars

I have read many books that discuss various aspects of evolution but never a complete overview of the subject. While I understand how evolution works, and have no doubt about its veracity, I was not aware of the overwhelming supporting evidence for the theory from multiple different scientific disciplines: Paleontology, Biogeography, Embryology, Genetics, Comparative Anatomy etc. Mr Coyne relishes his task, clearly presenting the evidence fact by fact. Within the first 100 pages he presents a broad, and consistent body of evidence in which he weaves together facts from multiple fields. It was hugely entertaining reading an academic, at the top of his game, build such an impressive case. The evidence is undeniable and overwhelming. Evolution is true!

The first section of the book presents the evidence for evolution, the middle section explains how evolution works and the last section explains how evolution has affected humans. The last chapter covers the current acceptance of the theory and hints that what the theory needs a modern Milton to explain it’s wonder. He finishes by pointing out we are the only species to have figured out how we came to be and that this is a wondrous thing.

This book is devastatingly convincing and very educational. I have read a lot of books on evolution but nothing this comprehensive. Jerry Coyne is a leading academic who set out to explain why evolution is undeniably true in layman’s terms. He has succeeded with a book that no thinking person should miss.