GTM Monitoring using Google Cloud Pixel Tracking

Google Tag Manager Monitoring - BigQuery Featured Image

A while ago, Simo Ahava and Mark Edmondson described How To Build A Google Tag Manager Monitor. They discussed several solutions, but their choice was Cloud Functions, since it is easy to get the solution quickly up and running.

They did initially consider Pixel Tracking via a Load Balancer, but they found this method less accessible. But as they wrote:

If you do have a custom domain name at hand, the load balancer is probably the most robust solution to utilize, since it doesn’t require the overhead of running a programmed application.

So, this blog post is just more or less a “note to self” about how to do Google Tag Manager Monitoring using Google Cloud Serverless Pixel Tracking via a Load Balancer. Follow the instructions from Simo and Mark up to the headline 3.1.3. Technology of choice: Cloud Functions. Then head back to this blog post, and follow the instructions here. I just stand on the shoulders of the awesome work from Simo and Mark.

Just so I have mentioned it, this setup will have a small cost.

Table of Contents

1. Google Cloud setup – Part 1

I link to all the Google Cloud products used in this blog post. But, if you find it difficult to navigate around in Google Cloud, the perhaps easiest way is to use the Search Field (A), and search for the product.

1.1. Create a Google Cloud Storage Bucket

  1. Make sure you have selected the correct project. My project is called “Savio – GTM Monitoring“.
  2. Give your Bucket a meaningful name.GTM Monitoring - Name your Google Cloud Storage Bucket
  3. Choose a Default Storage Class. Since you have chosen to use Serverless Pixel Tracking because of performance, Multi-Regional is your choice. Choose a Location that suit you best. Since I live in Europe, EU is my preferred location.
    Choose a Default Storage Class
  4. Choose how to Control Access to Objects and Set object-level and bucket-level permissions. Skip Advanced settings, and click Create.Choose how to Control Access to Objects
  5. After your bucket have been created, Activate Cloud Shell. You do that from the icon on the right hand side of (5) in the top of your browser, or use the shortcut g then s. We are using Cloud Shell to create our pixel (although you can upload a pixel as well).
    Activate Cloud Shell
  6. Copy this code, and paste it into Cloud Shell:

    Replace [YOUR_BUCKET_NAME] with the name of the bucket you just created, and hit Enter. If pixel.png was created successfully, close Cloud Shell.
    Cloud Shell - Create Pixel
  7. You are now going to Edit Permissions for your pixel.
    Cloud Bucket - Edit Pixel Permissions
  8. Add User Permissions to the pixel. In the Add users field, enter allUsers. In the Select a role list, select Reader.
    Cloud Bucket - Add Pixel Permissions

The Google Cloud Storage Bucket setup is now completed.

1.2. Create a Google Cloud Load Balancer

You now have a Cloud Storage Bucket with a single, invisible, publicly accessible pixel. Next, you set up a way to log all the requests made to the bucket by creating an HTTP Load Balancer in front of that bucket.

You are also going to set up a Custom Domain (ex. with your DNS provider.

  1. Start Configuring your Load Balancer. Select HTTP(S) Load Balancing.
    Cloud Load Balancer - Start Configuration.
  2. Give your Load Balancer a Name.
    Cloud Load Balancer - Name your Load Balancer
  3. In the Backend Configuration, choose Create a Backend Bucket.
    Cloud Load Balancer - Create Backend Bucket
  4. In the Backend Bucket settings, give it a short and relevant name, then browse for and select the bucket you created earlier. Also make sure to select the Enable Cloud CDN option before you click the “Create” button.
    Cloud Load Balancer - Backend Bucket Settings
  5. Skip the Host and path rules section as the defaults are fine, and move on to the Frontend configuration section. Here you need to ensure the “HTTPS” protocol is selected, and from the IP address dropdown select the “Create IP address” option. You can also choose IP version.
    Cloud Load Balancer - Frontend settings- Create IP Address
  6. In the Reserve a new static IP address setting, give your IP address a relevant Name, and click Reserve.
    Cloud Load Balancer - Frontend settings- Reserve IP-address
  7. Our Load Balancer frontend is going to handle HTTPS traffic, so it needs a SSL Certificate. Click Create a new certificate.
    Cloud Load Balancer - Frontend settings- Create a new SSL Certificate
  8. In the Create a new certificate settings, you can either upload your own certificate, or use a Google Managed Certificate. I’m using a Google Managed Certificate.  Enter a relevant Name for your certificate, and select Create Google-managed certificate. In the Domains setting, add the custom domain you want to use for the CDN that fronts your tracking pixel. Click the Create button at the bottom of the page, to save the new certificate.
    Cloud Load Balancer - Frontend settings- SSL Certificate settings
  9. Click Done button to finalize the HTTPS frontend settings.
    Cloud Load Balancer - Frontend settings- Click Done button
  10. Review the settings. If everything looks good, click Create. The load balancer and certificate will then start provisioning.
    Cloud Load Balancer - Review Setting - Click Create button

The Load Balancer setup is now completed.

It can take a some time for the certificate in particular to provision, so while you wait, head over to your web host, and set up your custom domain using the IP address that you reserved in the Load Balancer setup. The interface for creating a CNAME record varies from one DNS provider to another, but here is mine.

DNS settings - A record

2. Google Tag Manager setup

While you wait for your DNS to propagate, head over to Google Tag Manager, and edit the Google Tag Manager Monitor Template and Tag.

2.1. Google Tag Manager Monitoring Tag Template

Edit the Allow URL Match Patterns so it reflects your Custom Domain. It should be in this format:

Google Tag Manager Monitor Tag Template

2.2. Google Tag Manager Monitoring Tag

In your GTM Monitoring Tag, edit the Get request endpoint.

Google Tag Manager Monitor Tag

Now run GTM in Preview mode, and check the Network tab in the Browser Console.

Network Tab in Browser Console

When status 200 is returned, your DNS is updated, and you can publish your changes in GTM.

3. Google Cloud setup – Part 2

You GTM Monitoring data have now started to flow into Google Cloud. The next step is to get the data into BigQuery.

3.1. Stackdriver Logging Export

Head over to Stackdriver Logging, and you should see your GTM Monitoring data. Click Create Export.

Stackdriver Logging - Logs Viewer

Give your Stackdriver Logging Export a useful Sink name (1), select BigQuery as Sink service (2), and choose Sink destination (3). Either create a new BigQuery dataset, or choose an existing dataset (like the one you have already created for your GTM Monitor project).

Stackdriver Logging - Create BigQuery Export

3.2. BigQuery

The BigQuery Table Name will be in the format requests_YYYY-MM-DD, and is predefined. Table Schema is  also predefined, and contains more information than you need for this project (since it contains the complete log from Stackdriver).

GTM Monitoring - BigQuery Table from Stackdriver Logging

3.2.1. Make a BigQuery View or Data Studio Custom Query

The next step is to make this data available in a useful format. You can either parse the data into a BigQuery View, or if Data Studio is your next step, you you can run the SQL code in a Data Studio Custom Query Connector.

Remember to replace your-project.your_dataset in the FROM part of the query with your project and dataset name.

Do also notice that this query is using a Table Wildcard Function, which means that you will make a query against all the data in the table if you run this query. Since BigQuery is a paid tool, you  may want to limit the query, or use other cost savings methods.

Although you can run the SQL Query above directly in Data Studio using a Custom Query, it’s recommended to modify the query a little bit. BigQuery parameters can optimize query cost and dashboard performance. Thanks to my colleague Simen André Hansen for the tip. Below is a query like this illustrated. Only the last part of the query is shown.

4. Some Final Words

This Google Cloud setup is inspired by the Google Cloud tutorial How to do serverless pixel tracking.

Although this isn’t described in this blog post, I’ve made some adjustment to the original GTM Monitor Template. I only send data if the GTM Event contains a Tag. I do also track some more information, as shown in my Tag Monitoring settings below:

GTM Monitoring Tag: More Choices settings

  • Track GTM Container ID: Sometimes it can be useful to track the Container ID. Ex. if you are using Google Tag Manager (360) Zones.
  • Track Page URL: To identify if Tags are triggered on the correct pages, Page Path can be of value. If the GTM Container is handling several domains, the Hostname can be useful to track.

You find the Adjusted GTM Monitoring Template below.

For other tracking and analysis of your GTM Container, take a look at the blog post GTM Monitor 2.0 by Doug Hall.

If you think anything could be improved, please leave a comment.

4.1. Adjusted GTM Monitoring Template

4.1.1. Adjusted BigQuery SQL

If you use the template above, you may want to adjust your BigQuery SQL so it includes the extra data added.

Happy analysing your GTM Container Tags using Google Cloud Serverless Pixel Tracking.

Leave a comment

Your email address will not be published.


This site uses Akismet to reduce spam. Learn how your comment data is processed.