Referrer Spam can be a very tricky problem to deal with and with so many conflicting ideas on how best to fix the issue available we thought we would try to help you find a simple solution.
We will look at the 3 main spam referrers and give you an idea of how to deal with them, these are;
- Ghost Referrals that never actually visit your site but use software to make it look like they have and Google records the visit in your reports e.g. darodar.com / buy-cheap-online.info/ pornhub-forum.ga
- Creepy Crawlers that neither identify themselves nor follow rules. It is recommended that you do not follow these links as it could lead to your getting a virus e.g. maridan.com.ua and blog.ranksonic.com.
- Well Behaved Bots & Spiders might appear with no malicious intentions.
We will also be bringing you some information at the end about a brand new, free referrer spam removal tool that is quick and easy and can help you clear your analytics.
But why would people go to the bother of doing this? It could be that they are looking for information or site vulnerabilities, another reason could be to target website owners and try and get them to their own site to sell a service or a product or finally it could be used as a way to get traffic. Unscrupulous SEO Services could claim to be boosting your traffic but there will be no real value to it as it is just people trying to find out who you are with no interest in your service, brand or product.
So how do we eliminate this traffic that is causing chaos to so many reports? Employing tactics against all 3 spam referrer types is recommended to give more wide scale protection against the problem with some techniques being more effective in certain fields than others.
Before we start there is also some good advice from LunaMetrics that should be observed to ensure you don’t butcher your Google Analytics settings.
1. Make sure you always have an unfiltered view in your property that has zero filters.
2. Don’t implement it immediately in your main view. Create a new test view that mirrors your main one in every other respect, and then add the filter(s).
3. If you’re happy with the new filter based on this test, then go ahead and implement it in the main view.”
It is also recommended by some sites to use the referral exclusion list feature but it can sometimes change a session to a direct visit and will therefore remain in your reports. More often than not this doesn’t work so we would just stay clear of it.
So, here we go. First to get rid of Ghost Referrals you can set tailored filters to stop the bad traffic coming in but this can be time consuming and it is sometimes a better idea to create a filter that lets the good traffic IN.
If you do choose to go down this route ensure you include all the valid hostnames or you could exclude valid traffic. If you start with the Multi Year Report showing just the hostnames, you will find this at Audience>Technology>Network>Hostname. You can then identify the valid ones from there.
It is also worth noting here that if you see and conversions from not set hostnames you might want to investigate why. They might not be associated with page view and could be event based call logging. You might have to adjust tracking code snippets and filters.
You can then create a filter including an expression capturing all valid domains. It is also worth testing before committing. You can try the Advanced Segment to get an idea of results without losing any data.
To create your filter expression identifies your hostname from the Google Analytics report. Enter your hostname, if you have multiple hostnames separate them by a vertical bar (|) you may need to enter any third party payment services like Paypal. If you do use the vertical bar do not use it at the end of the expression and do not leave any spaces.
Spammers can use recognisable sites so don’t add them to your list unless they are your pages on your sites.
It is extremely important to maintain this filter each time you put your tracking ID into a new web service and it is worth confirming you don’t exclude valid traffic if you use an unfiltered view on a monthly basis.
To eliminate ghost referrals from your historical reports, use a segment.
Some people insist changing .htaccess files work because the comments say to wait 2-3 days for it to take effect, and the spam traffic changes in that time, so it appears the blocking worked.
To exclude Creepy Crawlers from your site you can create a filter in Google Analytics by identifying their unique signature and basing the filter on that. It is also possible to filter on Campaign Source with a matching domain, this will usually work. Filtering just on Referral will not work as it must match the Full Referrer not only the domain. For more information you can read Google’s instructions on making filters.
It is possible to modify the htaccess file if you use an Apache web server. This would be advisable because the visits will still continue despite your filters, this modification will block the visits from your server.
It has been thought that the match pattern should be “semalt.com” as it’s a regular expression (the dot should be “escaped” with a backslash to stop it being interpreted as meaning ‘any character’), however Google doesn’t push it in their documentation for single domain matches.
Because there is a limit on the amount of filters you can use and a 255 character limit in the filter expression it is worth checking back every 2-3 months. If a bot is not getting any success it can drop off allowing you to free up a filter.
Spam referrals adapt, because of this you will also have to adapt and regularly updating your filters should stop any problems.
Well Behaved Bots and Spiders should be easier to deal with because there is no secret motive behind their appearance. Google Analytics has a checkbox that you can use however you need to enable it for every View you use.
If you go to the Admin section of your Google Analytics and navigate to each view you use then select View Settings you can then click the box that will exclude hits from known bots and spiders.
Employing all these tactics should see you on your way to getting reliable information in your Google Analytics reports but there are a few other handy tools on the market that can be a great help.
The free, Boyd Digital Referrer Spam Removal tool does all the hard work for you when you enter your Google Analytics information.
You simply enter your information and the tool scans your Google Analytics account before giving you the opportunity to select the flagged spam referrers as well as any non-spam referrers. The tool then generates either .htaccess or web.config rules which block the referrers. The tool does not modify your Google Analytics account and access is only granted to generate the spam referrers list.
For more information or to try the free Spam Referrer Tool you can visit http://refererspamtool.boyddigital.co.uk.