The Dark Web is More Like a Set of Dark Silos

The dark web is not nearly as social a place as the world wide web, researchers find.

Researchers recently conducted a comprehensive charting of the dark web, starting with the central hubs of “.Onion” domains, and created an algorithm to scan links from each site. They found 7,178 sites. 25,104 links connected all of these sites together.

Those sites which did not have links incoming could not be found. The researchers – Virgil Griffith, Yang Xu, Carlo Ratti – discovered that 87% of dark web sites do not link to other sites.To account for websites regularly appearing and disappearing, the authors only included sites that responded to their crawlers. “In our analysis, before pruning nonresponding domains, we found a graph of 13,117 nodes and 39,283 edges. After pruning, we have a graph of 7,178 nodes and 25,104 edges (55% and 64% respectively). In all results, we refer to this graph pruned of nonresponding domains as simply ‘the darkweb graph’”.

The researchers, through their research, determined the dark web is no web. But, rather, a set of  “dark silos,” according to the preliminary paper posted on ArXiv this week.

The researchers performed a second crawl to collect instances of dark web sites linking to the world wide web and compared the relative rates of outbound linking.

Due to the equal rates of outbound linking to the world wide web and the darkweb found, they concluded: “The low outbound linking is not due to the notorious impermanence of onion sites. If onionsites got drastically more stable, we would still see very low rates of linking. By elimination of the technological explanation, we suggest that people creating darkweb sites are, on average, simply less social than those creating sites on the www.” Therefore, the authors deduce that calling the dark web a “web” is misleading.

“It is more accurate to view it as a set of dark silos,” they post. “Unlike the www, the darkweb is a place of isolation.”

What are the implications?

“I personally find this rather strange, and interpret it as, socially speaking, people who create dark web sites are just less social beings,” says Virgil Griffith, as computer scientist at the Singapore-MIT Alliance for Research and Technology and the paper’s lead author. “Or at the very least, the dark web is barely used as a social mechanism—while the world wide web most definitely is.”

The researchers employed graph theory, a tool used to analyze social relationships on the world wide web. The researchers used these techniques in their investigating the of dark web, which they say is the first time such research avenues have been applied to the web. They focused in on “.onion” pseudo-top-level-domains

“The darkweb is infamously mysterious, and any insight into it for both harnessing it or informing social policy is welcome,” they write. The researchers contend that the world wide web and dark web are “immensely similar”.

They contend: “Therefore any differences between the structure of the darkweb versus the www likely indicate something about the societies inhabiting each.”

They admit that what they found on the darkweb turned out much different than similar studies looking into the clear web.

“The darkweb is indeed a very different graph than what in the world wide web—particularly, the glaring fact of so little linking to other websites,” the authors write. “The fundamental question is why there’s so little linking.”

For this, the authors offer two technological explanations. “In the darkweb, sites go up and go down all the time,” they suggested. “Why bother linking if there’s little chance that the destination will still exist?”

They also offer a social explanation. “As-is, people who create sites on the darkweb are cut from a different social cloth than those who create sites on the www.”

The Tor Project keeps basic statistics, saying there exist 60,000 distinct, current .onion addresses. In their analysis the authors found just  7, 178 active .onion domains.

“We attribute this high-discrepancy to various messaging services— particularly TorChat, Tor Messenger, and Ricochet,” write the authors. “In all of these services, each user is identified by a unique .onion domain.

Many people claim it is tough to know how much content is on the deep web. One estimate posits there is 500 times the content.

Featured image from Shutterstock.