Attributor is working hard to give our users the ability to monetize their content and benefit from revenue sharing. We work with the publishers and sites that are reusing their content, helping them claim their content and recover a portion of the revenue made from advertisements on these sites. Think of it as a monetized hyper syndication – syndicators are able to use the content, and as a result, the more the content is copied, the more money publishers make.

In the past few months we have established highly strategic relationships with ad networks, in turn allowing us to roll this service out to the general public.

Stay tuned – within a month or two you can expect Attributor to unveil something very, very interesting. In the meantime if you have any questions, hints, tips or ideas, please email me at dejan(at)attributor(dot)com.

Today, Dejan Diklic, chief crawling architect at Attributor, contributes his second post about online gaming piracy. Dejan manages the text, image and video crawling team and is responsible for the company’s crawling architecture, algorithms and operations.

Most recently, Dejan began researching online gaming piracy, and shares new insight again today.

In a recent post, we talked a bit about online game piracy and our first set of numbers as they relate to Wii games. We searched for 15 games and found each game on the web and roughly 16 different pages that provided us with links to each game hosted on multiple different file hosting sites (sometimes also called cyberlockers).

In this post, we’ll examine a rather interesting popularity question: “Who is the most pirated one of all?” To do so, we used the list of top 10 games from gamespot.com for each of the 3 top consoles (Wii, Xbox, PS3). We also used the top 10 games for the PC market. We then crawled the web to find links to these games on the top 25 file hosting sites. As a disclaimer, a problem with this method arises from the fact that not all games exist on all four platforms and one of the platforms might have more popular games than the others. Since we are looking at the top 10 most popular games for each platform, this problem is largely marginalized, though not entirely eliminated.

As in the previous study, we dumped the pages from the production system for each platform and then verified them. The results were really interesting:

As far as the consoles go, Wii games resulted in most pages with links to games. We found 220 pages with links to Wii games. Xbox and PS3 have much lower rates of piracy as we found only 59 and 22 pages respectively. The PC games were the absolute winner in terms of piracy. We found 50% more pages for the top 10 PC games than for Wii. The most popular copied game out there is “Battlefield: Bad Company 2” for PCs with 82 pages linking to ISO files. It is followed immediately by “Sonic & SEGA All-Stars Racing” for Wii with 79 pages linking to ISOs.

My hunch is that these numbers can be explained through ease of piracy for a given console. Piracy for PC and Wii games require pretty much just copying the CDs. PS3 and Xbox require a bit more effort in hardware.

In conducting this research, I came across an interesting post on Shack News about the most pirated Japanese DS and PSP games. The claim is that the top two games on Nintendo DS and PSP were downloaded 2 million and 5 million times, respectively. While we didn’t look at piracy for those two consoles, the numbers of downloads are staggering. With average game price of around $30 the potential loss to developers is huge.

To top these numbers, CESA came out with a report saying total piracy loss from 2004 to 2009 was around $41B. While this report needs to be taken with a grain of salt, the numbers are again huge.

In the next post, we will take a peek at game copying on P2P networks and attempt to quantify the value there as well.

Today’s post comes from Dejan Diklic, chief crawling architect at Attributor. Dejan manages the text, image and video crawling team and is responsible for the company’s crawling architecture, algorithms and operations. He plays an integral role in the strategy, development and implementation of Attributor’s research, which has brought international attention to the depth of content piracy across the Internet.

Most recently, Dejan began researching online gaming piracy, and shares his insight for the first time today.

As Chief Crawler here at Attributor, I spend a lot of time seeing what’s on the Web. Recently, computer game software piracy caught my attention and I thought I’d take a first look at what’s happening in this industry. When I started the research, I expected the game piracy to be a well-established “trade” concentrated on just a few sites. But as it turns out, online games are being pirated even more then expected and they are available everywhere.

Since Wii is my personal favorite, I chose this as the segment of the industry to examine. For this blog post, I looked at a total of 15 games on 20 of the largest hosting sites (most of which we identified in previous research for books and magazines). The system crunched through our 40-billion-page index and found several hundreds of thousands of pages that might be relevant for these games. A page is considered relevant if it is classified as being “game related,” meaning it has a lot of keywords that are normally associated with games, has links to known hosting sites and talks about the specific game we searched for.

After the system processed all the pages, the algorithm ranked several thousand pages as almost 100% likely to contain the games in our search. During a one-day period in June 2010, Attributor professional services reviewed 300 pages from that list and found 250 pages linking to desired content, which resulted in roughly 3,200 links leading to copies of the games. We found a copy of every game we looked for. As usual, most infringements were found on rapidshare. The top 5 domains — are rapidshare, hotfile, megaupload, x7 and mediafire — were responsible for 85% of the infringements.

If we take a rather unscientific average of these numbers, we get more than 200 links found for each game and 16 pages referring to valid links for each game (keep in mind these are numbers for just one day of processing). This simply means that anybody looking for any particular Wii game can find it on 16 different sites, on average. These results show that the current state of piracy control on hosting sites is non-existent since anybody can go to a search engine and find any pirated game they want.

The distribution of hosting sites is also rather interesting, as now there is a new domain in the top 5 that we haven’t seen as prominently before, x7.to.

The next step in my project will be to look at game piracy across different platforms and see who is the most pirated out there. Stay tuned!

Our research looks at a segment of the magazine industry, 133 English language magazine titles, and the infringement that occurs on just 20 of the more than 2,000 domains that illegally host full-issue downloads of these magazines.

Because this report focuses on downloads, and the complete download numbers for these magazine issues are not available, we’re unable to provide the depth of analysis that we wanted. We did not want to make estimations or projections on download numbers, potential views or potential lost revenue without more accurate figures to base it on. We believe there’s more research to be done on magazine piracy and plan to dig deeper into the article-based content reuse in the coming months for a more complete picture of the problem.

That said, it’s important to note that we have not attempted to estimate what it means for the larger industry, but we do believe this research shows that 1) magazine piracy occurs commonly on the Web and 2) it is causing publishers to lose sales.

The research has two parts; first, we look at the entire library of 133 magazine titles and instances of issues available for download, and second, we look at cases of infringement for just the most recent issue, March 2010.

The complete report is below and can be downloaded here.

• As part of its ongoing crawling operations, Attributor estimates that there are more than 2,000 domains (cyberlockers and torrents) that host infringing copies of magazines. An infringement is defined as a full-issue copy of a magazine that is available for download.

• During a 1-week period in March 2010, Attributor followed the complete library (all issues) of 133 English language magazines on 20 of these infringing sites. (*The sites were selected based on high frequency of infringement found through Attributor’s book service. A complete list of these 20 sites is available upon request.)

1. Attributor’s technology verified 3,996 instances of downloadable, full issues of these 133 magazines on these 20 sites.
2. 84 of the 133 (63%) magazines had infringements.
3. These 84 magazines averaged 48 infringements each.
4. The rate of infringement varied by category.
5. Top 5 sites contribute 85% of all infringements. Attributor’s research for publications in other languages indicates that while infringements of publications in a particular language tend to be concentrated on a small number of sites, the particular sites vary greatly by language.

• Attributor also looked at infringements that occur soon after an issue is published as that presumably has a direct impact on newsstand sales. To shed light on that, Attributor performed a secondary count of only the March 2010 editions of the top 30 infringed magazines in the data reported above

1. An average of 33 infringements was found for these issues.
2. The details are as follows:

Attributor is not attempting to make any estimate about the total number of illegal downloads that is occurring for these magazines or about the consequent loss in revenue to these magazine publishers with this brief study. Nevertheless, it is clear that magazine piracy occurs commonly on the Internet.

Book publishers frequently ask us how much online piracy is impacting their revenue.  Today, with the release of the first study to quantify book piracy in the U.S., we’re pleased to announce new capabilities to help book publishers answer this critical question.

Key findings of the research are listed below, and the full study can be downloaded here (.pdf).

We plan to update and add more depth to these findings regularly as we expand our anti-piracy services.  If you publish books or journals and wish to see how FairShare Guardian can protect your titles, please contact us.

Key Findings

• Significant amounts of pirated book downloads are taking place online, representing potential losses of $2.75-3 billion, representing roughly 10% of the total United States book sales
• Across the 913 books included in the study, nearly 10,000 pirated copies of every title in the study was available for free download.
• The  Business and Investing, Professional & Technical and Science genres have the largest potential lost sales per title.

Source: Attributor January, 2010

Source: Attributor January, 2010

Examples

Listed below are examples of pirated titles and a link to just one of the 25 sites included in the study (Copies downloaded figures are as of 01/14/10).

• Arts and Photography
  • Architect’s Drawings by Kendra Schank Smith, 10,010 copies downloaded

• Biographies & Memoirs
  • Dreams from My Father by Barack Obama, 2,850 copies downloaded

• Business & Investing
  • Freakonomics by Steven Levitt, 1,132 copies downloaded

• Culinary and Hospitality
  • Mastering the Art of French Cooking by Julia Child, 659 copies downloaded

• Fiction
  • The Girl with the Dragon Tattoo by Stieg Larsson, 1,732 copies downloaded
  • Bed of Roses by Nora Roberts, 1,156 copies downloaded
  • Angels and Demons by Dan Brown, 8,177 copies downloaded

• Science
  • Advanced Calculus by Wilfred Kaplan, 3,526 copies downloaded
  • Molecular Biology of the Cell by Bruce Alberts, 3,584 copies downloaded

Methodology

• FairShare Guardian™ service monitored piracy for 913 popular books in categories representative of the industry across the the top 25 one-click hosting sites starting in October 2009 for a period of 90 days.  Download .pdf copy to view list of sites.

• FairShare Guardian captured the number of successful downloads completed for each of the 913 titles as reported on four file hosting sites that make the download data available (4shared.com, scribd.com, wattpad.com and docstoc.com).  Across these four sites, a total of 3.2 million downloads occurred.

• Across the top 25 one-click hosting sites, a total download figure of over 9 million copies was projected using the 36.4% share of one-click hosting sites that the four above-mentioned sites represent. Download .pdf copy  to view share figures.

• The retail value of these 9 million copies was calculated to reach $380 million. Each book’s retail price and category/genre information was collected from Amazon.com.

• The 913 titles in this study represent works from publishers totaling 13.5% of the U.S. book publishing market. Projecting this $380 million value to the entire industry results in total potential piracy figure of $2.8 billion.

• This study does not to answer the question, “How many of these pirated books would have been purchased legally if piracy was not an option?” Previous piracy studies assume a one-to-one substitution, meaning all pirated material would have been purchased and thus the market value of pirated books is equal to the actual loss, though Attributor feels this is an overly optimistic assumption. This issue will be addressed in a future research phase.

While we welcome more discussion among these players about the contours of consumers’ right and copyright law, it’s ironic that the same companies alleging exaggerated copyright notices are profiting from duplicate content.

“Big Content” and “Big Technology” are clearly trying to spin the issue. To clarify, let me breakdown the four factors of Fair Use and show where Attributor can provide objective metrics to guide Fair Use determination . . . without boring you to death.

Factor 1: The purpose and character of the use, including whether such use is of a commercial nature is for nonprofit educational purposes.

Detectable. While Attributor won’t identify if the usage is transformative, we automatically detect if the page on which reuse occurs has advertising present. As evidenced by recent moves by the New York Times, advertising is clearly driving the online content economy making commercial use an increasingly important factor.

Also, you can learn a lot about the purpose and character of a use by whether or not attribution is provided, which in the online world, amounts to links from the copy to the original – we report back on attribution for every match we find.

Factor 2: The nature of the copyrighted work

Not Detectable. Sorry, we can’t determine whether your content is fiction or non-fiction, but we’ll add it as a feature request!

Factor 3: The amount and substantiality of the portion used in relation to the copyrighted work as a whole.

Detectable. This is a fancy way of saying that the less of your content that is taken, the more likely it qualifies as Fair Use. For each match, we report back on the percentage of the original content that has been reused.

Factor 4: The effect of the use upon the potential market for or value of the copyrighted work.

Detectable. Not only will we indicate if ads are present on the reusing site, but we will also provide the amount of monthly traffic for the site. We’re also adding functionality that will help you understand the impact of content reuse on your ranking in search engines. As noted in our Harry Potter research, much of content reuse is occurring on sites that appear higher in search engine rankings than the original content owner. This can have a major impact on the relative market value of the original work.

Attributor won’t remove all the emotion from the room in copyright discussions, but it will provide an objective means to evaluate Fair Use disputes and (hopefully) result in less litigation and less posturing between “Big Content” and “Big Technology”.

The chief villains are the shifty sites reusing the lyrics, but the article goes a step further by pointing out the lucrative role of search engines:

“Google is selling advertising on all the big copyright-infringing lyric Web sites,” Rousmaniere said. “It may seem like small potatoes, but lyrics are a huge search term on the Internet–these sites (and Google) are probably pulling in hundreds of thousands of dollars monthly, all on the back of copyrighted material.”

Google takes a lot of copyright “heat”, but it is not the only search engine to profit from unauthorized use of copyrighted material — and consider the size of the “dirty” money when you include the ad networks who serve display ads on these pages and the sites hosting the content in the first place.

It is an intricate plot that needs to be sorted out before publishers and content creators like Rousmaniere lose incentive to put their original content online.

Here are our findings:

• 2,806 sites lifted the book content
• Duplication by type of site breaks down as follows
• 54% Forums/Blogs (other than Harry Potter fan sites)
• 27% Splogs or other commercial sites
• 19% Harry Potter fan sites
• Across all sites, the percentage of full chapter text copied is ~71%
• Over 80% of the sites duplicating the content have ads on their pages
• Sites duplicating the book are based in 43 different countries.

By all accounts sales of the book are phenomenal, and judging by an informal Attributor office poll, the impact on the first weekend’s sales appears to be zilch.

That said significant portions of the book continue to pop up all over the web making the downstream impact of the duplication unknown.

One thing for certain– the hysteria over the book’s release has filled many sploggers pockets. We just hope they repaid Scholastic by buying a few copies of the book!

August 1st Update

After reading about the Spanish spoiler’s release via TechCrunch, we loaded the 1st 10 Spanish chapters of the book into Attributor.

Here are our findings:

• 440 sites lifted the book content in Spanish
• Duplication by type of site breaks down as follows
• 48% Splogs or other commercial sites
• 40% Forums/Blogs (other than Harry Potter fan sites)
• 12% Harry Potter fan sites
• Across all sites, the percentage of full chapter Sapnish text copied is 60%
• Over 85% of the sites duplicating the content have ads on their pages
• Sites duplicating the book are based in 11 different countries.

The increase in splog site duplication is further proof of how easy it is to monetize popular search terms using Adsense or Yahoo Search Marketing text links.

This is the first of a series of analyses we’ll be sharing in the coming months. We hope to provide insights on how the content economy works and how it could be better managed with web-wide visibility and accountability.

• We found 574 unique pages duplicating the spoiler content. New sites are coming in at a rate of ~20 per hour.
• The duplication is spread over 27 countries including the United States, Russia, the Netherlands, UK, China, Germany, Italy, Germany, Poland, British Virgin Islands, Argentina, Hungary, Brazil, Croatia, Samoa, Spain, Columbia, Singapore, Malaysia, Philippines, Peru, Mexico, Australia, Vietnam, Canada, Czech Republic, Indonesia and the Ukraine.
• Harry Potter fan sites represent ~10% of the duplication, indicating that many fans don’t want to know the ending before they buy the book; instead matches are primarily “splogger” sites–these are sites that place ads around the lifted content and game the search engines to appear high in search rankings. This enables them to profit from the increased Harry Potter search activity.

5PM Update

• There are now 708 pages duplicating the Harry Potter and the Deathly Hallows spoiler content. Many sites that appeared on our list in the first 12 hours have taken down the spoiler content; however, new sites are popping up at a faster rate. Attributor keeps a cached copy of all matches, and we will tally the gross match number in a future post.
• 9% of the sites duplicating the spoiler content are Harry Potter fan sites.
• 559 (79%) of the sites have ads on their pages.
• Most of the duplication is verbatim. The percentage copied across all domains is >80%
• Over the last 24 hours, duplication on Chinese sites has grown the fastest.

More info to come, including a thorough “post-game” analysis of what Attributor found.

Yesterday we added the web page that includes excerpts from the unreleased novel Harry Potter and the Deathly Hallows. Attributor’s monitoring platform immediately found 312 separate reuses of the spoiler page across the Web.

The top 3 sites hosting the re-used content are

1. livejournal.com
2. groups.google.com
3. twoj.net

With Attributor, instead of having to manually search and sort through a haystack of tertiary matches, we provide specific citations where your content are being re-used and enables you to act appropriately — by issuing a licensing request, or sending automatic DMCA takedown notices. We also enable more innovative approaches such as allowing a teaser portion of your content to be posted freely as long as the reuser links back to your site or includes one of your widgets to sell the full version.

We will update the blog over the next few days with the latest numbers and analysis of the sites that are re-using the content. Watch this space and let us know what you want to find out.