Who Is Tor Really For?

I have been trying to rationalise several apparently contradictory surveys of the "Dark Web" that have been published recently.  Some suggest the vast majority of Tor is being used for illicit purposes, others suggest a much smaller figure.  So what is the truth?

I wrote several weeks ago on research done by Kings College, London, which appeared to show that just over 50% of hidden services were being used for illicit purposes.  I wasn't surprised then to read a report from Intelliagg using the facilities from Darksum, which suggests that slightly less than 50% of such sites were involved in activities that would be considered illegal in the UK or US. Such a small variation in results could easily be a result of differences in definitions of illicit behaviour, those sites that were available during each survey, and so on.

The search engine used by Darksum does appear to be highly credible. I haven't had a chance to use it but it claims to use the same technology that DARPA used in their MEMEX programme, which was probably the best such search two years ago. I'm minded to take the results produced by this search as accurate.  However, there are a couple of caveats one needs to apply to the results, exactly as with the results from Kings College.

Tor Continues To Confound

Tor is, yet again, producing some data that seems to defy explanation.  Having talked a lot about how the number of unique .onion addresses has varied in recent weeks (and was apparently settling down) another metric has suddenly shown a dramatic change.  The amount of data being reported as using the hidden services has plummeted (and I use that word deliberately).

The immediate thought was that there had been another sudden drop in the number of unique .onion addresses and hence the "dark web" had contracted for some reason.  However, the data shows that the number of unique .onion sites remains stable as I was expecting when I wrote about the "new normal":

Who Is Spinning This Hidden Web?

For anyone who has been following the story of Tor hidden services, which began in February as we all noticed a sudden increase in the number of unique .onion addresses, it appears that it might be finding a new equilibrium.  Having said that, the last time I thought this it leapt up even higher than the original increase which caught our attention.

As of this morning the number of unique .onion addresses appears to be levelling out.  It looks considerably lower than the peak that was reached but it is still at or higher than the level which woke us up a few weeks ago.

Tor Suddenly Goes Into Reverse (Again)

The prevailing theory to account for the surge in Tor .onion addresses was that it was being driven by malware.  The most likely candidate was Locky.  I say "was" because, yet again, Tor has confounded us all by showing just as dramatic a fall in .onion addresses:

The initial surge that raised interest also had its own fall, followed by an even larger rise.  So, what happens from here is anyone's guess.  No one seems to know what is causing this - well someone does but they're not saying.

Whilst the surges do appear to have some correlation with Locky infections its not clear why the number of addresses would fall if it were Locky.  Is Locky somehow clearing up after itself - maybe just turning off the addresses once used?  Does the drop mean that people are paying up and the .onion address is no longer needed?  Lots of questions but few answers.

The Trouble With Anonymity

There is an old adage that on the Internet no one knows you're a dog. It accompanied a cartoon in The New Yorker in 1993, which eons in Internet evolutionary terms, but issues with identity remains as problematic today as ever.  This holds true for system to system communications as well as those from person to person.  One attack that has become well known in forging identities in peer to peer networks is the Sybil attack.

The essence of a Sybil attack is simple: you subvert the reputation of a system in a peer to peer network by setting up a large number of pseudonymous identities and thereby gain an undue influence allowing you to, for example, gather data that you would not otherwise be able to do.  The ease with which a Sybil attack can be mounted is largely a factor of how cheaply identities can be generated.

In the last 10 years much work has been done on how to defend against Sybil attacks in particular contexts, such as, social networks and peer to peer (P2P) networks.  All such defences basically rely on one approach: having a trusted agency certify identities.  Researchers showed as far back as 2002 that without a logically centralised authority Sybil attacks were always possible unless you make unrealistic assumptions about networked resources.

What Just Happened On The Tor Network?

A few days ago I was discussing how much of Tor's hidden services (known as .onion sites) was being sued for illegal purposes.  Then over the past 2-3 days something quite extraordinary happened.  The metrics reported by the Tor project themselves shows that the number of unique .onion sites has increased by well over 20,000:

Number of unique .Onion sites as recorded by the Tor project

There has never been an increase of that magnitude before.  What can account for it?  I can see a few possibilities:

1. Something was wrong with the way in which Tor was calculating the number of .onion sites, and there has been a correction to better reflect reality.  I think this unlikely as the technique (which I discussed previously) appears to be very sound.  I could understand such a correction in the early days when fewer nodes were reporting statistics but over the period that more and more nodes have become part of the reporting there has been no change like this.  With almost 50% of nodes now participating this would appear to be, prima facie, a real jump.
2. Somebody has set up a whole slew of new .onion sites.  For this to have happened in this volume there has been either a very large number of people collaborating on something or some form of automation.  But why?

How Much Of Tor Is Used For Illegal Purposes?

A paper just published by researchers at Kings College attempts to quantify how much of Tor is used for illegal purposes.  Or rather, what proportion of Tor's hidden services are used for illicit purposes.

The scans returned a total of 5,205 live websites within the hidden services network, out of which 2,723 were conducting one of the following activities:

Results from scans conducted by Kings College Reserachers

These results were reported with what was termed a "high degree of confidence".  Where sites were categorised as "None" it was because there was no content (hence counted neither as illegal or legal in nature), and "Unknown" means it was not possible to determine the nature of the content.