The team said it was “surprised” by the scale of the monitoring. Copyright holders could use the data to crack down on illegal downloads.
The three-year research was carried out by a team of computer scientists who developed software that acted like a BitTorrent file-sharing client and logged all the connections made to it. BitTorrent is a method of obtaining files by downloading from many users at the same time.
The logs revealed that monitoring did not distinguish between hardcore illegal downloaders and those new to it.
“You don’t have to be a mass downloader. Someone who downloads a single movie will be logged as well,” said Dr Tom Chothia, who led the research.
“If the content was in the top 100 it was monitored within hours,” he said. “Someone will notice and it will be recorded.” Less popular content was also monitored although less frequently, the study indicated.
The research identified about 10 different monitoring firms logging content. Of these, a handful were identifiable as copyright-enforcement organisations, security firms and even other research labs.
But about six of the biggest-scale monitors were harder to identify, as the companies behind them used third-party hosting firms to run the searches for them.
Why such firms wanted the massive amounts of data was unclear, said Dr Chothia. “Many firms are simply sitting on the data. Such monitoring is easy to do and the data is out there so they think they may as well collect it as it may be valuable in future,” he said.
Some firms alleged to be carrying out mass-scale monitoring have been accused of selling the data to copyright holders for marketing purposes.
“The data shows what content is popular and where,” said Dr Chothia. The study also revealed that so-called blocklists, used by some illegal file-sharers to prevent monitors from connecting to their computers, might not be much use.
“Many of the monitors we found weren’t on the blocklists so these measures to bypass the monitors aren’t really working,” said Dr Chothia.