Archive

Archive for the ‘Online & Search Issues’ Category

Google versus Bing – a competitive intelligence case study

February 2, 2011 7 comments

Search experts regularly emphasise that to get the best search results it is important to use more than one search engine. The main reason for this is that each search engine uses a different relevancy ranking leading to different search results pages. Using Google will give a results page with the sites that Google thinks are the most relevant for the search query, while using Bing is supposed to give a results page where the top hits are based on a different relevancy ranking. This alternative may give better results for some searches and so a comprehensive search needs to use multiple search engines.

You may have noticed that I highlighted the word supposed when mentioning Bing. This is because it appears that Bing is cheating, and is using some of Google’s results in their search lists. Plagiarising Google’s results may be Bing’s way of saying that Google is better. However it leaves a bad taste as it means that one of the main reasons for using Microsoft’s search engine can be questioned, i.e. that the results are different and that all are generated independently, using different relevancy rankings.

Bing is Microsoft’s third attempt at a market-leading, Google bashing, search engine – replacing Live.com which in turn had replaced MSN Search. Bing has been successful and is truly a good alternative to Google. It is the default search engine on Facebook (i.e. when doing a search on Facebook, you get Bing results) and is also used to supply results to other search utilities – most notably Yahoo! From a marketing perspective, however, it appears that the adage “differentiate or die” hasn’t been fully understood by Bing. Companies that fail to fully differentiate their product offerings from competitors are likely to fail.

The story that Bing was copying Google’s results dates back to Summer 2010, when Google noticed an odd similarity to a highly specialist search on the two search engines. This, in itself wouldn’t be a problem. You’d expect similar results for very targeted search terms – the main difference will be the sort order. However in this case, the same top results were being generated when spelling mistakes were used as the search term. Google started to look more closely – and found that this wasn’t just a one-off. However to prove that Bing was stealing Google’s results needed more than just observation. To test the hypothesis, Google set up 100 dummy and nonsense queries that led to web-sites that had no relationship at all to the query. They then gave their testers laptops with a new Windows install – running Microsoft’s Internet Explorer 8 and with the Bing Toolbar installed. The install process included the “Suggested Sites” feature of Internet Explorer and the toolbar’s default options.

Within a few weeks, Bing started returning the fake results for the same Google searches. For example, a search for hiybbprqag gave the seating plan for a Los Angeles theatre, while delhipublicschool40 chdjob returned a Ohio Credit Union as the top result. This proved that the source for the results was not Bing’s own search algorithm but that the result had been taken from Google.

What was happening was that the searches and search results on Google were being passed back to Microsoft – via some feature of Internet Explorer 8, Windows or the Bing Toolbar.

As Google states in their Blog article on the discovery (which is illustrated with screenshots of the findings):

At Google we strongly believe in innovation and are proud of our search quality. We’ve invested thousands of person-years into developing our search algorithms because we want our users to get the right answer every time they search, and that’s not easy. We look forward to competing with genuinely new search algorithms out there—algorithms built on core innovation, and not on recycled search results from a competitor. So to all the users out there looking for the most authentic, relevant search results, we encourage you to come directly to Google. And to those who have asked what we want out of all this, the answer is simple: we’d like for this practice to stop.

Interestingly, Bing doesn’t even try to deny the claim – perhaps because they realise that they were caught red-handed. Instead they have tried to justify using the data on customer computers as a way of improving search experiences – even when the searching was being done via a competitor.  In fact, Harry Shum, a Bing VP, believes that this is actually good practice, stating in Bing’s response to a blog post by Danny Sullivan that exposed the practice:

“We have been very clear. We use the customer data to help improve the search experience…. We all learn from our collective customers, and we all should.”

It is well known that companies collect data on customer usage of their own web-sites – that is one purpose of cookies generated when visiting a site. It is less well known that some companies also collect data on what users do on other sites (which is why Yauba boasts about its privacy credentials). I’m sure that the majority of users of the Bing toolbar and other Internet Explorer and Windows features that seem to pass back data to Microsoft would be less happy if they knew how much data was collected and where from. Microsoft has been collecting such data for several years, but ethically the practice is highly questionable, even though Microsoft users may have originally agreed to the company collecting data to “help improve the online experience“.

What the story also shows is how much care and pride Google take in their results – and how they have an effective competitive intelligence (and counter-intelligence) programme, actively comparing their results with competitors. Microsoft even recognised this by falsely accusing Google of spying via their sting operation that exposed Microsoft’s practices – with Shum commenting (my italics):

What we saw in today’s story was a spy-novelesque stunt to generate extreme outliers in tail query ranking. It was a creative tactic by a competitor, and we’ll take it as a back-handed compliment. But it doesn’t accurately portray how we use opt-in customer data as one of many inputs to help improve our user experience.

To me, this sounds like sour-grapes. How can copying a competitor’s results improve the user experience? If it doesn’t accurately portray how customer data IS used, maybe now would be the time for Microsoft to reassure customers regarding their data privacy. And rather than view the comment that Google’s exposure of Bing’s practices was a back-handed compliment, I’d see it as slap in the face with the front of the hand. However what else could Microsoft & Bing say, other than Mea Culpa.

Update – Wednesday 2 February 2011:

The war of words between Google and Bing continues. Bing has now denied copying Google’s results, and moreover accused Google of click-fraud:

Google engaged in a “honeypot” attack to trick Bing. In simple terms, Google’s “experiment” was rigged to manipulate Bing search results through a type of attack also known as “click fraud.” That’s right, the same type of attack employed by spammers on the web to trick consumers and produce bogus search results.  What does all this cloak and dagger click fraud prove? Nothing anyone in the industry doesn’t already know. As we have said before and again in this post, we use click stream optionally provided by consumers in an anonymous fashion as one of 1,000 signals to try and determine whether a site might make sense to be in our index.

Bing seems to have ignored the fact that Google’s experiment resulted from their observation that certain genuine searches seemed to be copied by Bing – including misspellings, and also some mistakes in their algorithm that resulted in odd results. The accusation of click fraud is bizarre as the searches Google used to test for click fraud were completely artificial. There is no way that a normal searcher would have made such searches, and so the fact that the results bore no resemblance to the actual search terms is completely different to the spam practice where a dummy site appears for certain searches.

Bing can accuse Google of cloak and dagger behaviour. However sometimes, counter-intelligence requires such behaviour to catch miscreants red-handed. It’s a practice carried out by law enforcement globally where a crime is suspected but where there is insufficient evidence to catch the culprit. As an Internet example, one technique used to catch paedophiles is for a police officer to pretend to be a vulnerable child on an Internet chat-room. Is this fraud – when the paedophile subsequently arranges to meet up – and is caught? In some senses it is. However saying such practices are wrong gives carte-blanche to criminals to continue their illegal practices. Bing appears to be putting themselves in the same camp – by saying that using “honeypot” attacks is wrong.

They also have not recognised the points I’ve stressed about the ethical use of data. There is a big difference between using anonymous data tracking user  behaviour on your own search engine and tracking that of a competitor. Using your competitor’s data to improve your own product, when the intelligence was gained by technology that effectively hacks into usage made by your competitor’s customers is espionage. The company guilty of spying is Bing – not Google. Google just used competitive intelligence to identify the problem, and a creative approach to counter-intelligence to prove it.

The pursuit of justice and social media.

January 2, 2011 8 comments

You shall not pervert judgment; you shall not respect persons, nor take a bribe; for a bribe blinds the eyes of the wise, and perverts the words of the righteous. Justice, justice shall you pursue, that you may live, and inherit the land which the Lord your God gives you. (Deut. 16:19-20)

The world in 2011 is still split between the haves and the have-nots, the rich versus the less rich and the poor. Despite a global recession, many have profited – while millions look for work and struggle daily to survive. There has been reason for optimism in the last year – at the end of last year, Aung San Suu Kyi was released from unjust detention by the Burmese generals. However, this is the exception – and when it comes to justice it is difficult to be optimistic for many countries.

I think that it is worthwhile looking at a few news stories of the last month of 2010 and what they say about different views on justice, the rights of the individual, and also the potential impact of social media on calls for justice.

The first news story concerns the President of Iran’s bete-noire, Israel. Moshe Katsav was born in Iran, and moved to Israel in 1951, aged 5, as a refugee. He spent the next 4 years of his life, living in tents and a transit camp which eventually was built up to become the Israeli town of Kriyat Malakhi. At the age of 24, he was elected mayor of this town – the start of a life in the political limelight. He was elected to the Israeli Parliament (the Knesset) in 1977 and served as Minister of Housing & Construction; Labour & Welfare; Transportation; Tourism; and was Deputy Prime Minister between 1996-1999. In 2000 he stood for, and was elected President. In 2006 however, he was accused of sexual molestation and rape, and forced to resign in 2007. He was subsequently indicted and tried for rape. On 30 December 2010, Katsav was found guilty by a three judge panel and will shortly be sentenced, He can expect a mandatory jail term.

Although this is a highly unflattering story it is important as it shows how justice should work. It doesn’t matter how influential or senior somebody is, he or she should not be above the law. If they commit crimes then they should be tried and sentenced. The fact that a former President was accused, tried and found guilty shows that in Israel, nobody is above the law. Katsav is not alone – there are other public figures within Israel who have been or are being investigated for various crimes, and this is how it should be. As the Bible says “You shall not pervert judgement…” and have two levels of justice – one for those in positions of authority or with ability to pay, and one for everybody else.

In contrast, a recent news story from Bangkok shows how power and privilege can corrupt calls for justice as well as the potential influence of social media to ensure that justice does take place.

A few days before the Katsav judgement – 27 December 2010 – a road accident took place resulting in the deaths of 9 people (although the first news stories reported only 8). Initial media reports blamed a van driver for the deaths, but subsequently a different story emerged that was suppressed by Thai news outlets. This was rapidly circulated via a Facebook site calling for justice. Within 24 hours, the page had generated over 180,000 likes.  Currently over 270,000 people have said that they like the page, and there are numerous comments.

Driver on Blackberry after road accidentThe story that was suppressed, apparently backed up by CCTV and witness accounts, told of an impetuous 16-year old girl without a driving licence who got impatient with a slow moving van and tried to push it out of the way with her Honda Civic. The van crashed, resulting in the loss of life of a number of students at Thammasat University – one of the best in Thailand – plus an assistant to the dean at the university’s Faculty of Architecture and Planning, and researchers including a promising scientist from a very poor family who had won a national scholarship. The girl that caused the accident, in contrast, came from a well known family. Her father had been a general and her great great grandfather was King Rama V (1853 – 1910) – the king whose policies ensured that Thailand stayed independent (and not colonised like neighbouring countries) and who is viewed as having put the country on the road to modernization. Following the accident, the girl was photographed calmly using her Blackberry – apparently posting to a social networking site (although subsequently claiming to be calling her father).

Although some of the latest reports suggest that the girl will be prosecuted, the fact that she is described as a “minor” may give a get-out clause. (“ persons of that age were not entitled to a driver’s licence, nor could they be fully subject to criminal and civil liability for deaths and damage.“)

Although the comments from Thai Prime Minister, Abhisit Vejjajiva, that “nobody is above the law” suggest that Thailand, like Israel, will treat miscreants equally, that does not seem to be the belief of those who set up the Facebook page, especially taking account the initial reports blaming the van driver.

In 1991, Alvin Toffler published PowerShift (US). The book is a “study of power in the 1990s and beyond” and traced “the shifting global power structures and describes how the very definition of power has changed in modern times”. PowerShift was written before the Internet had become mainstream, and well before today’s social media tools. The book suggested that the balance of power was changing from the traditional sources to those who controlled information. Although such ideas have circulated for some years, social media – such as Facebook and Twitter – are allowing for injustices to be quickly publicised, and as such, it becomes easier to call for justice. They are an example of the democratisation of information and allow for genuine expressions of “people power”, the “power of the crowd” as well as the “power of the many over the few”. Such calls are challenges to the existing elites of the world – who are likely to do what they can to suppress them. One approach is that taken by China, who, as the year 2010 closed,  was reported to have  banned sites like Skype, Facebook and Twitter. Other ways are to attack challengers to the existing order and some rumours suggest that the Thai Facebook page supporters may even be punished.

Nevertheless, I believe that a genie has been let out of a bottle. Although most of the time, social media is used to communicate with friends and colleagues, it has a power of its own – to change the world. With over 500 million people connected to Facebook – around 10% of all people in the world – it becomes very difficult to suppress injustices and much easier to spread the concepts of freedom, justice and the truth – however much dictatorial and corrupt regimes may try and stop it. However with power comes responsibility. The responsibility is to ensure that what is spread is the truth. There is a real danger that such tools can also be used to spread false propaganda, lies and untruths – allowing for injustice to spread. There is the danger of mob-rule, where a suspect is condemned, without being given a chance to defend themselves – the 21st century equivalent of a lynch mob.

Social media can help ensure that privileged people don’t escape justice. In this, it will serve a positive purpose. It can also act to reinforce prejudice, irrational hatred and bigotry – as can be seen in groups that try to delegitimise and condemn Israel, despite ample evidence to the contrary, as in the example of Katsav’s trial.

You must not follow the crowd in doing wrong. When you are called to testify in a dispute, do not be swayed by the crowd to twist justice.(Exodus, 23:2)

Note: After I wrote this Blog post, I came across a link to a fascinating article by the Internet Guru, Clay Shirky, on the Political Power of Social Media – where he discusses issues relating to the power of social media to change governments, etc. He also considers the potential for change, and also the potential for achieving nothing positive. (Article is free but registration required. The article was summarised in the Economist – with comments. Evidently it was written prior to the Wikileaks affair – as some of the comments put the USA in the “control” corner rather than the “freedom” corner!)

Delicious humbug and monitoring News stories

December 20, 2010 Leave a comment

Effective competitive intelligence monitoring means keeping up with the news, and where news is likely to impact you, drawing up strategies to take into account changes.

The problem with instant news via twitter, blog posts and various other news feeds is that news updates sometimes happen too quickly, before the snow has even had a chance to settle. That’s fine – just so long as the source for the news is 100% reliable, and the news story itself is also totally accurate. (I’m using snow as a metaphor here – rather than the more normal dust – as outside there is around 15cm of the stuff with more promised during what looks like being the coldest winter in Europe for over 20 years).

Unfortunately, more often than not, one of these two aspects fails: the source may not be reliable, or the story may not be true, or may be only half-true. Typically however people pick up on the story and it spreads like wildfire (so not giving that snow a chance to settle before it gets melted all over the web).

An example of this has been taking place this last week – with numerous posts reporting the demise of the web-bookmarking service Delicious.

Delicious (originally located at http://del.icio.us) was founded in 2003 and acquired by Yahoo! in 2005. By 2008 (according to Wikipedia) it had over 5 million users and 180 million bookmarked URLs. This makes it an important source for web-searching as, unlike with a search engine such as Google or Bing, each URL will be human-validated and valued.

Apparently, during a strategy meeting held by Yahoo! looking at its products, Delicious was named as a “sunset” product.

Slide from Yahoo! strategy presentation - on plans for various products

An image of this slide was tweeted – and after Yahoo! failed to deny that Delicious was to be closed, posts quickly appeared denouncing the company for the decision. Nobody really cared that sites like AltaVista and AlltheWeb were going – as they were to all intents and purposes dead anyway. (Their search features have long been submerged into Yahoo!’s own – although I for one, still miss some of the advanced features these services offered. Alltheweb allowed searching of flash content, and AltaVista had a search option that nobody now offers: the ability to specify lower/upper case searches).

The problem is that many of the sources posting the story are normally extremely accurate and reliable so when they post something, it is reasonable to believe what they say. This then compounds the problem as the news then gets spread even further – and when the story is corrected, the news followers often fail to spot the corrections.

The example of Delicious is not isolated. There are many news stories that develop over time – and when making strategy decisions based on news it is important to take into account changes, but also not rush in, if a news item hasn’t been fully confirmed.

Ideally check the source – and if the source is a press item (or blog post) then look to see if there is a press release or where the original item came from, in case there is a bias, inaccuracy or mis-interpretation. Only when the news has been confirmed (or where there are no contra-indications) should strategy implementation take place (although of course, the planning stage should be considered immediately if the potential impact of the news is high).

In the case of Delicious their blog gives the real story. The slide leaked to twitter was correct – Delicious is viewed as a “Sunset” product. However that doesn’t mean it will be closed down – and Yahoo! states that they plan to sell the service rather than shut it down (although it is noticeable that they don’t promise to keep the service going if they fail to find a buyer).

There is, in fact, another lesson to be learned here, relating to company awareness on the impact of industry blogs and twitter. It is important to not only monitor what is said about your company, but also to anticipate what could be said.  In a world where governments can’t protect secrets being leaked via Wikileaks, it would be surprising if high-impact announcements from companies didn’t also leak out to industry watchers. Some companies constantly face leaks – Apple is notorious in this regard – and part of their strategies involve managing potential leaks before they do harm. In this Yahoo! failed. As a media company that depends on the web for its business this is a further example suggesting how Yahoo! seems to have lost its way. This is not negated by evidence such as the leaked slide mentioning Delicious, showing they are thinking about their future and product/service portfolio.

Wikileaks and Whistleblowing!

December 5, 2010 3 comments

As soon as Joseph saw his brothers, he recognized them, but he pretended to be a stranger….” (Genesis Ch. 42 v7).

There is a time for everything, and a season for every activity under the heavens….a time to be silent and a time to speak….” (Ecclesiastes Ch 4 v1, v7)

I wasn’t planning to write about the Wikileaks affair – as in essence, I agree with Wikileaks that excessive secrecy is wrong. At the same time, as the preacher (identified with King Solomon) in the book of Ecclesiastes says, there is a time to speak out, and a time to remain silent.

I believe that many of the items leaked deserved to be leaked. It is wrong to keep details of torture, rape, summary executions, and various other war-crimes secret, irrespective of whether such crimes were committed by the USA, the Iraqis or whoever. War-crimes should always be exposed, and prevented. If a government tries to keep such crimes hidden, then it is the duty of responsible people to expose them. Keeping such information secret just allows for a culture that views the enemy as non-human and dispensable – and ultimately, this makes all who allow this to happen complicit in the crime.

At the same time, there is good reason to keep diplomatic negotiations hidden, however duplicitous they may appear to be – so long as there is a procedure to ensure that such records are not kept secret permanently, but are released when they are no longer politically and diplomatically sensitive. Similarly, information that could put lives at risk – through the identification of people who oppose an oppressive government or who collaborate with others to end oppression – is totally wrong.

Essentially, a leak to prevent wrong-doing is, in my view, correct, whereas a leak for some warped belief that everything should always be out in the open and public is wrong.  Whistle-blowing to prevent corruption and criminal activity is right – whether it impacts commercial or government business. There is a time to speak out, and a time to keep silent, as in the biblical story of Joseph. Had Joseph identified himself when his brothers first visited him, he would have been unable to test their sincerity and repentance. It was important that he kept his status secret – in the same way that it is important that diplomatic cables shouldn’t be revealed as a general rule.

Sarah Palin & Mike Huckabee enter – stage right.

Sarah Palin

Sarah Palin

One reason I’d avoided discussion of Wikileaks was that Sarah Palin had just commented on the site. Having just written a blog post on her, I didn’t want to reprise some of my comments. Her view that Julian Assange, the founder of Wikileaks, should be hunted down “with the same urgency we pursue al-Qaeda and Taliban leaders” is yet another emotive, and ill-reasoned Palinesque comment showing a lack of understanding of what al-Qaeda stand for and what the Taliban represent. Assange may have caused damage to USA interests but in no way can he be seen as an overt enemy who would like to destroy everything that the USA stands for, and to impose a totalitarian belief system on the world.

However along comes Mike Huckabee who, apparently, does not wish to be outmanoeuvered by the outspoken Mrs Palin in his dreams of entering the White House. Huckabee has called for the execution of the person accused of leaking the material to Wikileaks. 

Bradley Manning

Bradley Manning

I agree that the man accused of leaking the material, Private Bradley Manning should be tried and if found guilty, punished (assuming that the trial is fair, which now with so much negative press is doubtful). I do not agree that he deserves the death penalty. He did not release the files to an enemy government and nor did he do it for cash rewards. According to reports he did it after seeing attempts to cover up possible war-crimes committed by the US – for instance an air-strike that killed a dozen people in Baghdad and where the air crew laughed at the dead, and another in Afghanistan, that killed dozens of children. This makes him a whistle-blower and not a spy or traitor, and as such, this needs to be taken into account in any penalty. An overly severe or unwarranted sentence will just serve to further deter whistle-blowing and allow corrupt officials, politicians or business managers to continue in their actions.

I also question whether it is Manning alone who should be blamed. The US government must share some blame in not protecting material they viewed as confidential. Apparently the material leaked was available to many thousands of people. Following the September 2001 terrorist outrages an attempt to stop the silo mentality that had prevented different bits of information being linked together, correctly allowed for improved sharing of intelligence. Evidently such sharing did not consider the security implications of making so much information available to so many – with minimal protection. If Manning had not leaked the material, I’m sure that somebody else with a moral conscience, seeing the Iraq video, would have.

Additionally, it was not Manning (if the leak came from him) who posted the material but Wikileaks. Wikileaks would like to be seen as a channel whereby whistle-blowers can alert the world of crimes (commercial or governmental) that are being kept hidden, and I believe there is a need for such a service. Had they fulfilled this role, they would have edited out any material that did not serve a public service in being released.

The free-rights-for-all crowd enter – stage left.

Facebook Group Logo for Boycotting Amazon over Wikileaks

Wikileaks, trying to remain online, used Amazon’s hosting service for the site. Of course, Amazon was then criticised for ostensibly supporting the service, and came under pressure to boot the service. Had they not done so, I’m sure that they would have faced a large and damaging right-wing campaign against them – especially as the peak holiday buying season approaches.  Their action however has, instead, led to a call for a boycott from those who believe in total freedom of speech regardless of the content, including a dedicated FaceBook fan page.

Had Amazon not hosted Wikileaks in the first place, neither side would have complained. Instead, Amazon has been criticised from both sides for doing what it felt was the right thing – both commercially and morally. They hosted the site – I’m sure because they believe in the moral principle of Freedom of Speech. It was not just a commercial decision – as I don’t believe that you will find any Nazi or Ku Klux Klan sites on Amazon servers. They host sites that they believe are not objectionable to their ethos. When, in the case of Wikileaks, this then threatened to be commercially damaging, they pulled the site – and get blasted by the “Freedom-of-Speech-at-all-costs” crowd who I’m sure would quickly campaign against the company if Amazon took this literally and started hosting racist, Nazi or kiddie-porn sites.

Julian Assange and Wikileaks

Wikileaks is not the first whistle-blowing web-site. My favourite – www.fuckedcompany.com – has unfortunately shut down, along with its sister site www.internalmemos.com. These two sites were important in warning investors of commercial shenanigans and companies that were having problems. Unfortunately I know of no other good sites offering such services. Wikileaks could, and should, have taken on this role. However with Assange as their editor-in-chief, they seem to be looking for headlines and controversy rather than fulfilling a role in preventing corruption and crimes being committed by both government and commerce.

Julian Assange

Julian Assange. (Is it just me who thinks that Assange has a strong resemblance to Lucius Malfoy in the Harry Potter films? He just needs to grow his hair a bit longer and they could be twins!)

Assange, according to Wikipedia, has led a peripatetic life. He claims to be constantly on the move – starting from his childhood, where his mother, in conflict with his father, hid Assange and his half-brother for five years. Obviously very bright, Assange became a leading computer hacker at the age of 16, and claims to have studied at university level, physics, mathematics, philosophy and neuroscience.

In 2006 he founded Wikileaks with an overtly political aim of encouraging leaks to change organisations that he felt were unjust or secretive:

…the more secretive or unjust an organisation is, the more leaks induce fear and paranoia in its leadership and planning coterie…

Prior to the current 2010 leaks, WikiLeaks has published material relating to extra-judicial killings in Kenya, information on toxic waste dumping off Africa, Church of Scientology manuals, a report on share price manipulation (that led to criminal charges and a jail sentence for the culprits) by the Icelandic Kaupthing Bank and many more reports and items.

What next?

The latest leaks have caused severe embarrassment for the USA and many of its allies. Worringly, the response by some of the opponents to Wikileaks show how freedom in the USA is at risk. Rather than accept that their security was lax and that the leaks show signs that illegal practices are being covered up, blame is being pinned on the message and the messengers (Manning and Assange). That is not to say that either are totally innocent. Manning, if he was responsible for leaking all the documents was naive to say the least. Assange strikes me as a petulant, spoilt and amoral man who loves the publicity he is getting, and doesn’t really care who gets hurt in the process.

Scene from the film "The Girl who Kicked the Hornet's Nest" - the third book of Stieg Larsson's Millennium Trilogy.

Meanwhile, I wonder whether the accusations of rape that have been made against Assange in Sweden are just an attempt by his enemies to put him behind bars. It would not be the first time that the Swedish authorities were accused of falsifying evidence to imprison an undesirable element linked to computer hacking, violence against women, espionage and the security services.  Stieg Larsson‘s Millenium Trilogy are works of fiction, detailing how corrupt elements within the Swedish secret services conspire to frame the heroine Lisbeth Salander, and keep her locked up, so as to save their own skins. Salander, like Assange, is a computer hacker who takes on and challenges authority. It would be ironic if the Swedish accusations against Assange also turned out to be false – and were an attempt by his enemies to put him away. However such things only happen in fiction…. don’t they?

Scene from the film "The Girl who Kicked the Hornet's Nest" - the third book of Stieg Larsson's Millennium Trilogy.

 

Getting found on the web

September 20, 2010 Leave a comment

I’d planned to write this post on business culture, working as part of a team and leadership. Meanwhile I’m still tinkering with WordPress – trying to get to know it better. There’s a couple of things I liked about Google’s Blogger tool that I’ve not yet managed to work out how to do on WordPress. Actually that’s not completely true. If you download WordPress and blog on your own server it’s fairly easy. However there are also things against doing that – for example some technical details, security & spam, etc. Conversely WordPress.com won’t let me download some of the plugins I wanted. Despite this, I’m pretty happy with WordPress as a blog platform.

While searching around, I came across a great presentation from Matt Cutts of Google, given at WordPress’s 2009 San Francisco conference.

Matt Cutts is well known as not only a Google expert (naturally) but also as an expert on search engine optimisation – in other words, how to get found on the web. There is so much in this one presentation that I think it should be compulsory viewing for everybody who writes for the web. Although I try and do most of what was said – there’s still more for me to do, and he had some great examples. The focus was on blogging using WordPress but in fact much of the content was much wider – with explanations on what search engines (and specifically Google) look for when indexing the web.

As not everybody will spare 45 minutes to watch the video, I’ll summarise some of the content – and the slides can be found at Matt’s web-site.

Matt starts by asking why write a blog in the first place, but quickly moves onto optimising sites for the web and how to increase your chances of being found. He gives a simple explanation for Google’s PageRank (named after Google founder, Larry Page, rather than that it measures the web page importance / popularity based on the number of links to the page).  Around half way through the presentation, he starts emphasising the most important thing about writing for the web (whether for a general site or for a blog). The writing has to be relevant and reputable. Good and interesting writing gets read. Boring, trite, repetitive writing doesn’t. In other words, if you don’t love what you are writing about, and don’t know or have anything to say, then don’t say anything. (For more on good writing, read the Write Way – my brother’s blog – covering how to produce technical documentation that’s understandable).

Then we get to the bits on SEO (Search Engine Optimisation – i.e. writing web-sites so that they can be found). When I take training courses on finding competitive intelligence on the web I always emphasise the need to understand how sites get to the top spots. If you understand this, then it becomes easier to think of ways of finding sites that aren’t found on the first page – and often these are the pages that hold the hidden gems that the competitor analyst has to find.

One key skill is to think of alternative terms. As a portable back-up device I tend to use a memory stick. However other terms for the same device are “flash drive“, “USB drive” and a few others. Searching for only one of these risks missing out sites not using that term but one of its synonyms. Cutts gives an example of searches for ipod car for connecting an ipod to a car’s radio / entertainment system. There is an alternative less costly technology called iTrip that also allows an iPod to be connected to the car radio. For every two searches using the term iPod Car, there was one that used the key word iTrip. This means that excluding the latter term from sites selling the former will result in them missing out on a third of the potential Internet traffic. From a competitive intelligence perspective, it would also mean missing out information on a competing technology. Just because it’s not exactly the same, using a different technological approach and costing less, doesn’t mean it’s not also a competitor – so searching for one and not the other would mean missing out on what customers are actually looking to purchase.

Other SEO techniques covered include web-page naming, establishing a reputation, monitoring visitors via analysis of log files / google analytics and how not to spam (and scam).

Lies, Damned Lies, Statistics & Facebook

June 10, 2010 Leave a comment

I’ve been impressed with the numbers of people using social networking sites – and the importance of social networking for marketing has become significant over the last few years.

Facebook claims 400 million users (i.e. nearly 6% of the global population that is approaching 7 billion people). I’ve always thought that this figure must include duplicate accounts – as I don’t believe that most people in China, India, Africa and many other areas of the world have Facebook accounts (or even computers – although the numbers are growing). The World Bank stated that there were just under 300m Internet users in China and 52m in India in 2008. (There’s a great graph of this at Google’s Public Data tool – that shows that in 2008 there were around 1.5bn web-users).

Even taking account the exponential growth – let’s assume that web users globally are now over 2 billion  people – Facebook’s figures imply that 1 in 5 users have a Facebook account.

I know of many people who don’t have an account and some who refuse to get one. In my age group (over 40), I’d guess that the majority don’t. So where this 400m figure came from and what it includes is a key question.

It now seems that Facebook has been boosting it’s membership figures. I just read this article from one of my favorite sites (www.pandia.com). Apparently Facebook has been telling advertisers that it has 1.6m users in Oslo. The trouble is that the greater Oslo metropolitan area only has 900,000 people. Facebook apparently counts members by IP address – and I guess that it is feasible that this could include users who access the site via Oslo based web-servers. However not if you consider the next statistic given. The Facebook advertiser tool says that there are 850,000 Facebook users between the ages of 20-29 in Norway – which is 235,000 more than the total numbers (613,000) in that age group.

This over-inflation isn’t just a Norwegian issue. According to CheckFacebook.com (a site that tracks data from the Facebook advertising tool giving Facebook membership numbers), almost 63% of online users in the UK now have a Facebook account. That’s 27m out of a total UK population of 62m. In some countries it’s even higher. Apparently all (100%) Nicaraguan, Qatari and Bangladeshi web users also have a Facebook account, as do 99% of Indonesians, 98% of Filipinos, 97% of Venezuelans, and 85% of Turks.

It’s possible that these statistics are true. However, if so, I’m sure that they also include occasional and infrequent users as well as dormant and duplicated accounts.

One of the most important types of competitive intelligence analysis is to not take everything at face value. When presented with figures, it’s important to sense check them – wherever possible by using other sources (e.g. official population statistics). Only then should such data be used in decision making. You should also ask whether there is an incentive to exaggerate or under-estimate statistics. If there is such an incentive, it is likely that this will be done, at least in the published data. Decisions made using such erroneous or manipulated figures will probably be poor decisions and fail to achieve the expected results. In the case of Facebook, the incentive in exaggerating membership figures is that they can then boost their attractiveness to advertisers, and consequently their advertising revenues.

RIP Kartoo

March 14, 2010 Leave a comment
When I conduct training sessions on how to search I always emphasise that it’s more important to know how to find information rather than to depend on a small selection of key web-sites.

Many searchers depend on their bookmark list but what happens when a key site disappears: if you don’t know how to search you are stuck.

Searching isn’t just going to google and typing your query in the search box. Expert searching demands that you consider where the information you are looking for is likely to be held, and in what format. It requires the searcher to understand the search tools they use – how they work and their strengths and weaknesses. Such skills are crucial when key sites disappear as happened in January with the small French meta-search engine, Kartoo.

Kartoo was innovative and presented results graphically. It enabled you to see links between terms and was brilliant for concept searching where you didn’t really know where to start. Unfortunately it’s now gone to cyber-heaven, or wherever dead web-sites disappear to. It will be missed – at least until something similar appears. Already Google’s wonderwheel (found from the “options” link just above the search results”) offers some of the functionality and graphic feel, and there are other sites that offer similar capabilities (e.g. Touchgraph). Kartoo however was special – it was simple, free and showed that Europeans can still come up with good search ideas.

Example of a Kartoo Search

Of course Kartoo isn’t the first innovative site to disappear. Over the years, many great search tools have gone. Greg Notess lists some in his SearchEngineShowdown blog – and an article in Online magazine. There are more. How many people remember IIBM’s Infomarket service – an early online news aggregator from 1995, or Transium.

In fact, it was learning that sites are mortal that led to my approach to searching: don’t depend on a limited selection of sites but rather know how to find sites and databases that lead you to the information wanted. That’s a key skill for all researchers and is as valid today in the Google generation as it was in the days before Google.

%d bloggers like this: