[VIDEO] Track the Initial Traffic Source of the Visitor with Google Tag Manager

This week, the format of the content is a bit different. I did the 2nd guest video on Julian’s MeasureSchool channel (I’m pretty sure that no introduction is needed here and every one of you has watched at least several of his videos). This time, I’ve explained a technique that will be useful if you want to know not only the last traffic source/medium that drove the conversion but also the very first one that introduced your brand to that visitor.

If you are in a hurry, go ahead and watch the video. If you have more time, feel free to read the blog post as well.

 

Why do we need this solution?

Out of the box, Google Analytics offers Acquisition reports and one of the most popular features there are Source/Medium reports. They enable you to see which traffic sources (e.g. Google, Facebook, Email campaigns) are driving the most visitors and if you have configured goals/ecommerce tracking, you can also see which of those sources are actually converting the best.

Google Analytics uses last non-direct click attribution, therefore, the conversion will be attributed to the most recent non-direct traffic source. But what if you wanted to see what was the initial traffic source that initially drove the visitor to your website in the first place? Even though it’s possible to compare attribution models in GA, you’d like to have more data outside the Model Comparison Tool.

In this guide, I’ll show you how to track the initial traffic source that drove the visitor to your website, thus you will be able to use this data and compare your dimensions and metrics not only under the last non-direct click attribution but also as the first-click attribution.

Although the solution is far from being perfect and may be inaccurate in some cases, it may still give you some good insights after you gather more data for a longer period of time.

Interested? Let’s start!

The workflow

This time, I will be using a modified script that is based on the UTMZ Cookie Replicator (created by the team of Lunametrics). This replicator is an awesome solution that imitates the cookie used by the Classic Google Analytics. That cookie contained the traffic acquisition data of the current session, therefore, you could use its value as a tag-firing condition or pass that information to other tools.

However, this utmz cookie went missing after the Universal Analytics was introduced. Luckily, good people at Lunametrics shared their own solution that imitates the cookie and lets us use it in GTM.

The solution (explained in this blog post) is heavily based on the JavaScript that is used in the UTMZ replicator, however, there are some modifications made, so please use the script that I’ll share with you below.

So what’s the workflow?

1. A visitor lands on a page
2. A Custom HTML tag (with the modified Lunametrics’ JavaScript) fires and sets the initialTrafficSource cookie. That cookie contains the traffic acquisition data. If such cookie already existed in a browser, the script will not overwrite and leave as it is.
3. Then, with the help of 1st-party cookie variable, GTM will read the cookie’s value and will pass the data over to Google Analytics as a user-scoped custom dimension.
4. Test it. Done!

 

#1. Custom HTML tag

Let’s start with a tag. Here is the JavaScript that you need to fire on all pages (at least for now):

<script>
/**
* Original script is created by Lunametrics 
* https://www.lunametrics.com/labs/recipes/utmz-cookie-replicator-for-gtm/
* Modified by Analytics Mania https://www.analyticsmania.com/
*
* Data is stored in the __initialTrafficSource cookie in the following format; brackets
* indicate optional data and are aexcluded from the stored string:
*
* utmcsr=SOURCE|utmcmd=MEDIUM[|utmccn=CAMPAIGN][|utmcct=CONTENT]
* [|utmctr=TERM/KEYWORD]
*
* e.g.:
* utmcsr=example.com|utmcmd=affl-link|utmccn=foo|utmcct=bar|utmctr=biz
*/
(function(document) {

  var referrer = document.referrer;
  var gaReferral = {
    'utmcsr': '(direct)',
    'utmcmd': '(none)',
    'utmccn': '(not set)'
  };
  var thisHostname = document.location.hostname;
  var thisDomain = getDomain_(thisHostname);
  var referringDomain = getDomain_(document.referrer);
  var sessionCookie = getCookie_('__utmzzses');
  var cookieExpiration = new Date(+new Date() + 1000 * 60 * 60 * 24 * 30 * 24);
  var qs = document.location.search.replace('?', '');
  var hash = document.location.hash.replace('#', '');
  var gaParams = parseGoogleParams(qs + '#' + hash);
  var referringInfo = parseGaReferrer(referrer);
  var storedVals = getCookie_('__utmz') || getCookie_('__utmzz');
  var newCookieVals = [];
  var keyMap = {
    'utm_source': 'utmcsr',
    'utm_medium': 'utmcmd',
    'utm_campaign': 'utmccn',
    'utm_content': 'utmcct',
    'utm_term': 'utmctr',
    'gclid': 'utmgclid',
    'dclid': 'utmdclid'
  };

  var keyFilter = ['utmcsr', 'utmcmd', 'utmccn', 'utmcct', 'utmctr'];
  var keyName,
     values,
    _val,
    _key,
    raw,
    key,
    len,
    i;

  if (sessionCookie && referringDomain === thisDomain) {

    gaParams = null;
    referringInfo = null;

  }

  if (gaParams && (gaParams.utm_source || gaParams.gclid || gaParams.dclid)) {

    for (key in gaParams) {

      if (typeof gaParams[key] !== 'undefined') {

        keyName = keyMap[key];
        gaReferral[keyName] = gaParams[key];

      }

    }

   if (gaParams.gclid || gaParams.dclid) {

    gaReferral.utmcsr = 'google';
    gaReferral.utmcmd = gaReferral.utmgclid ? 'cpc' : 'cpm';

   }

  } else if (referringInfo) {

    gaReferral.utmcsr = referringInfo.source;
    gaReferral.utmcmd = referringInfo.medium;
    if (referringInfo.term) gaReferral.utmctr = referringInfo.term;

  } else if (storedVals) {

    values = {};
    raw = storedVals.split('|');
    len = raw.length;

    for (i = 0; i < len; i++) {

      _val = raw[i].split('=');
      _key = _val[0].split('.').pop();
      values[_key] = _val[1];

    }

    gaReferral = values;

  }

  for (key in gaReferral) {

    if (typeof gaReferral[key] !== 'undefined' && keyFilter.indexOf(key) >-1) {

      newCookieVals.push(key + '=' + gaReferral[key]);

    }

  }

  if (!getCookie_('initialTrafficSource')) {
    writeCookie_('initialTrafficSource', newCookieVals.join('|'), cookieExpiration, '/', thisDomain);
  }

  writeCookie_('__utmzzses', 1, null, '/', thisDomain);

  function parseGoogleParams(str) {

    var campaignParams = ['source', 'medium', 'campaign', 'term', 'content'];
    var regex = new RegExp('(utm_(' + campaignParams.join('|') + ')|(d|g)clid)=.*?([^&#]*|$)', 'gi');
    var gaParams = str.match(regex);
    var paramsObj,
      vals,
      len,
      i;

    if (gaParams) {

      paramsObj = {};
      len = gaParams.length;

      for (i = 0; i < len; i++) {

        vals = gaParams[i].split('=');

        if (vals) {

          paramsObj[vals[0]] = vals[1];

        }

       }

     }

     return paramsObj;

  }

  function parseGaReferrer(referrer) {

    if (!referrer) return;

    var searchEngines = {
      'daum.net': {
        'p': 'q',
        'n': 'daum'
      },
      'eniro.se': {
        'p': 'search_word',
        'n': 'eniro '
       },
      'naver.com': {
        'p': 'query',
        'n': 'naver '
      },
      'yahoo.com': {
        'p': 'p',
        'n': 'yahoo'
      },
      'msn.com': {
        'p': 'q',
        'n': 'msn'
      },
      'bing.com': {
        'p': 'q',
        'n': 'live'
      },
      'aol.com': {
        'p': 'q',
        'n': 'aol'
      },
      'lycos.com': {
        'p': 'q',
        'n': 'lycos'
      },
      'ask.com': {
        'p': 'q',
        'n': 'ask'
      },
      'altavista.com': {
        'p': 'q',
        'n': 'altavista'
      },
      'search.netscape.com': {
        'p': 'query',
        'n': 'netscape'
      },
      'cnn.com': {
        'p': 'query',
        'n': 'cnn'
      },
      'about.com': {
        'p': 'terms',
        'n': 'about'
      },
      'mamma.com': {
        'p': 'query',
        'n': 'mama'
      },
      'alltheweb.com': {
        'p': 'q',
        'n': 'alltheweb'
      },
      'voila.fr': {
        'p': 'rdata',
        'n': 'voila'
      },
      'search.virgilio.it': {
        'p': 'qs',
        'n': 'virgilio'
      },
      'baidu.com': {
        'p': 'wd',
        'n': 'baidu'
      },
      'alice.com': {
        'p': 'qs',
        'n': 'alice'
      },
      'yandex.com': {
        'p': 'text',
        'n': 'yandex'
      },
      'najdi.org.mk': {
        'p': 'q',
        'n': 'najdi'
      },
      'seznam.cz': {
        'p': 'q',
        'n': 'seznam'
      },
      'search.com': {
        'p': 'q',
        'n': 'search'
      },
      'wp.pl': {
        'p': 'szukaj ',
        'n': 'wirtulana polska'
      },
      'online.onetcenter.org': {
        'p': 'qt',
        'n': 'o*net'
      },
      'szukacz.pl': {
        'p': 'q',
        'n': 'szukacz'
      },
      'yam.com': {
        'p': 'k',
        'n': 'yam'
      },
      'pchome.com': {
        'p': 'q',
        'n': 'pchome'
      },
      'kvasir.no': {
        'p': 'q',
        'n': 'kvasir'
      },
      'sesam.no': {
        'p': 'q',
        'n': 'sesam'
      },
      'ozu.es': {
        'p': 'q',
        'n': 'ozu '
      },
      'terra.com': {
        'p': 'query',
        'n': 'terra'
      },
      'mynet.com': {
        'p': 'q',
        'n': 'mynet'
      },
     'ekolay.net': {
        'p': 'q',
        'n': 'ekolay'
     },
     'rambler.ru': {
       'p': 'words',
       'n': 'rambler'
     },
     'google': {
       'p': 'q',
       'n': 'google'
     }
   };
   var a = document.createElement('a');
   var values = {};
   var searchEngine,
     termRegex,
     term;

   a.href = referrer;

   // Shim for the billion google search engines
   if (a.hostname.indexOf('google') > -1) {

    referringDomain = 'google';

   }

  if (searchEngines[referringDomain]) {

    searchEngine = searchEngines[referringDomain];
    termRegex = new RegExp(searchEngine.p + '=.*?([^&#]*|$)', 'gi');
    term = a.search.match(termRegex);

    values.source = searchEngine.n;
    values.medium = 'organic';

    values.term = (term ? term[0].split('=')[1] : '') || '(not provided)';

  } else if (referringDomain !== thisDomain) {

    values.source = a.hostname;
    values.medium = 'referral';

  }

   return values;

  }

function writeCookie_(name, value, expiration, path, domain) {

    var str = name + '=' + value + ';';
    if (expiration) str += 'Expires=' + expiration.toGMTString() + ';';
    if (path) str += 'Path=' + path + ';';
    if (domain) str += 'Domain=' + domain + ';';

    document.cookie = str;

}

      function getCookie_(name) {

        var cookies = '; ' + document.cookie
        var cvals = cookies.split('; ' + name + '=');

        if (cvals.length > 1) return cvals.pop().split(';')[0];

      }

function getDomain_(url) {

  if (!url) return;

  var a = document.createElement('a');
  a.href = url;

  try {

    return a.hostname.match(/[^.]*\.[^.]{2,3}(?:\.[^.]{2,3})?$/)[0];

  } catch(squelch) {}

 }

})(document);
</script>

 

 

This JavaScript will read the data from:

• document.referrer global JavaScript variable (read referrer)
• UTM parameters
• gclid and dclid query parameters
• will try to match the referrer data against the know search engines

And if none of the above data exists, the traffic source will be set as (direct).

The format of the initialTrafficSource cookie formatted as follows: utmcsr=example.com|utmcmd=affl-link|utmccn=foo|utmcct=bar|utmctr=biz

• utmcsr is campaign source
• utmcmd is campaign medium
• utmccn is campaign name
• utmcct is campaign content
• utmctr is campaign term

So after you have created the Custom HTML tag and set it to fire on all pages, save it.

Refresh the GTM Preview and Debug mode and head over to the website you’re currently working on. Refresh the page and the GTM debugging console will appear. Check whether your Custom HTML cookie-setting tag has fired. If yes, you need to check whether the cookie was properly created.

#1.1. Checking the cookie

There are several options how you can do that. One is to use the browser’s built-in developer tools. The other one is to use browser extensions, like EditThisChookie.

This time, I’ll use the default browser’s functionality. If you are on Chrome, click Three Dots icon in the top right corner, then choose More Tools > Developer Tools.

Go to Applications tab and under the Storage section choose Cookies. Click the domain that you’re currently on. This will open a list of all the 1st-party cookies that are stored in your browser by this domain. Start looking for the initialTrafficSource cookie. Next to its name, you’ll see the value.

Its value may be different from yours because it depends on how you landed on that page in the first place. But if its format looks similar to the screenshot above, then you’re on the right track.

By default, Google Tag Manager does not recognize cookies and does not turn them into variables, thus you won’t see that cookie in your Preview and Debug (P&D) mode’s Variables tab… unless you create a variable by yourself.

In GTM, go to Variables > New and create a 1st-party cookie variable with the following settings:

After you’re done, save the variable, refresh the P&D mode and then refresh the page that you’re currently working on. In P&D console, choose any event (in the left side) and then go to the Variables tab. Scroll down and keep looking for that 1st-party cookie variable. Here it is:

If the value of your cookie variable is undefined, please check the name of the cookie that you have entered in that variable. It is case-sensitive. It must be precisely initialTrafficSource, NOT initialtrafficSource, InitialTrafficSource, or anything else.

 

#2. Create a Custom Dimension

The next step is to pass this custom data over to Google Analytics. We’ll achieve this thanks to a feature called Custom Dimensions.

First, go to your Google Analytics account, choose the property that you wish to send the data to and click the gear icon in the bottom left corner. Then in the Property section go to Custom Definitions > Custom Dimensions.

Create a new Custom Dimension. You can name it whatever you want but I prefer Initial Traffic Source. Choose the User as a scope.

Click Save and take a closer look at the number that you’ll see in the little code snippets. In my case, it’s 1 but in your property the number may (and probably will) be totally different. Memorize this number.

Now, go back to your Google Tag Manager account. Let’s set that Custom Dimension. There are two options how you can do that and it depends on how you implemented other GA tag configurations. If you use Google Analytics Settings Variable, set the Custom Dimension there.

If you set the GA settings directly in every GA tag, you’ll need to set the following settings in the Pageview tag. The screenshot below will be based on the GA Settings Variable.

Open the variable and click More Settings > Custom Dimensions and enter the following:

• In the Index field, enter the number of the custom dimension that you have just created.
• In the Dimension Value field, insert the 1st-party cookie variable that you have created in the previous chapter. You can do that by clicking the button with a Lego brick and choosing the variable from the list.

Save the variable.

 

#3. Fire the Custom HTML tag before the Pageview tag

The next step is very important. You need to make sure that the Custom HTML tag (that sets the initialTrafficSource cookie) fires before the GA pageview tag. Otherwise, some of your visitors (that just land and then bounce) won’t get the Custom Dimension sent to Google Analytics (because of the cookie not being available in the moment of the Pageview event).

So the next GTM feature that we’ll make use of is Tag Sequencing. It allows you to fire a tag before another tag fires.

In your GTM account, go to Tags and open the GA Pageview tag. Click Advanced Settings > Tag Sequencing. Click the checkbox Fire a Tag before the GA Pageview fires and choose that Custom HTML tag that you created in step #1. Click Save.

Also, open the Custom HTML tag and remove the All Pages trigger leaving only the Tag Sequencing as the only way that fires that tag.

 

#4. Let’s test

Let’s do a quick recap of what we just did:

1. We created a Custom HTML tag that creates an initialTrafficSource cookie. If such cookie already exists, the script will do nothing.
   1. We created a 1st-party cookie variable that reads the initialTrafficSource cookie.
2. Created a Custom Dimension called Initial Traffic Source and set its scope to User. This means that all hits of the current and future sessions of that user will be set with the value of our initialTrafficSource cookie. If you’re new to scopes, read this guide on Google’s knowledge base. Also, we updated the Google Analytics Settings variable. From now on, it will be pushing the Initial Traffic Source as a custom dimension to Google Analytics.
3. We set the Custom HTML tag to fire before the GA Pageview Tag does (thanks to the feature called Tag Sequencing).

Let’s test. For a proper testing you’ll need to do the following:

• Delete the initialTrafficSource cookie from the browser (after you open browser’s developer tools, go to Application > Storage > Cookies and delete the initialTrafficSource cookie). You can delete it by hitting the DELETE key in your keyboard. If you’re using some other browser (not Chrome), you’ll need to find out how to do this on your own.
• Use the Google Tag Assistant Chrome Extension. If you’re new to it, read this guide.

Before you refresh the page, click the Tag Assistant icon and enable diagnostics:

Now, refresh the page. The Custom HTML tag must fire before the GA Pageview tag. If it did, check whether the cookie was properly stored:

1. First check in the Developer Tools of your browser
2. Then check whether your 1st-party cookie variable properly reads its value (you can see that in the GTM Preview and Debug console)
3. Then click the Tag Assistant extension’s icon once again and click Google Analytics
   
4. Then click Pageview.
   
   Go to Custom Metrics and see if the value of your cookie is displayed there. It won’t look as pretty as in the GTM’s Preview and Debug console. That’s because the Tag Assistant displays the value with the “equals” sign encoded:
   
   But that’s totally fine. In GA reports this custom dimension will be displayed more properly.
5. Wait for several hours for the data to appear in your GA reports. For you, it might take even longer. Don’t panic for at least the first 24 hours.

 

Where can I see this data?

That’s up to you and your imagination. Here’s an example. Go to Acquisition reports > All Traffic > Source/Medium and add the Initial Traffic Source as the secondary dimension. If you have set goals or ecommerce tracking in GA, you can see in a single table what was the first traffic source of that user and what was the last traffic source of that conversion.

In the screenshot below I don’t have the goals or ecommerce set up (because that is a dummy account) but you get the idea.

 

Caveats

Even though this solution looks awesome, there are some serious drawbacks that you need to keep in mind.

• This solution is based on cookies. If a visitor clears the browser or lands on a page with a different device, he/she will get a new initialTrafficSource cookie that will not represent the accurate situation. However, that’s the fundamental problem with many web analytics tools (GA included). So look at the data with a grain of salt.
• This script cannot access historical data of the user. So if you implement it now, your already acquired users/visitors will probably get incorrect values. So you need to give it some time and collect more data. As time goes by, the number of actual new users will increase, thus the impact of your current visitors/users will be mitigated (to an extent).

 

Track the initial traffic source of a visitor: final words

I have to admit, trying this tracking technique, recording a video and writing a blog post was really fun. Although it required more time that it usually does for me to create content, that’s totally worth it. Huge thanks to Julian from MeasureSchool for inviting to his channel once again. If you haven’t yet, go definitely check his Youtube channel. That’s one of the places of the interwebz where I learned GTM.

If you just skimmed the blog post and scrolled down straight to the end of this article, here’s a quick recap of what I’ve done:

• Implemented the Custom HTML tag that reads the referral data, UTMs, gclid and dclid parameters in the link, matches the referral data to known search engines and then creates a cookie with that data. That cookie is set only once (unless it gets deleted at some point).
• Then we instructed GTM to read that cookie and send its value as a custom dimension to Google Analytics.
• In order to be sure that the cookie is created before the GA Pageview tag fires, we used GTM’s Tag Sequencing.

This solution is not bulletproof, therefore, please have in mind its caveats as well (read the chapter right before the conclusion of this blog post).