Q. Since the BPA Interactive audit is dependent on the placement of JavaScript tags, why not allow a site to use any analytics tool that deploys a tag rather than only the BPA-issued one?
A. The BPA Interactive audit process uses JavaScript tags to establish site traffic levels upon which we base our audit. Key traffic metrics include page impressions, user sessions (visits), and unique browsers among others.
In developing our process we opted to implement a single-tag technology for one important reason: standardization.
Critical to the audit process is that metrics, requirements, procedures and reported results are held to an equitable standard. This ensures a level playing field for all audited properties and comparable data sets for those using audit results.
The decision to use a single-tag technology was influenced by several factors including the results of a white paper issued by Stone Temple Consulting entitled “Web Analytics Shootout” (http://www.stonetemple.com/articles/analytics-report-august-2007.shtml). The paper focuses on the many reasons why different (tag) analytic tools produce different traffic counts and led us to conclude that a single tag, consistently applied, was the preferred audit approach.
An excerpt from the white paper:
Web analytics packages, installed on the same web site, configured the same way, produce different numbers. Why?
- By far the biggest source of error in analytics is implementation error. There are dozens (possibly more) implementation decisions made in putting together an analytics package that affect the method of counting used by each package.
- Placement of JavaScript on the site.
- Differences in the definition of what each package is counting. The way that analytics packages count visitors and unique visitors is based on the concept of sessions. There are many design decisions made within an analytics package that will cause it to count sessions differently, and this has a profound impact on the reported numbers.
Points 1 and 2 are detectable and correctable through an audit. Point 1 is a bit more difficult to address because implementation has typically taken place prior to audit and errors, while correctable, can impose upon available IT resources.
Point 3, differences in definitions of metrics, is a major issue which totally defeats standardization and makes comparison of results one of apples to oranges.
Another key aspect of the BPA Interactive audit process is the filtering of mechanical (spider/robot) and internal traffic from the results. Using a single tag and analysis platform allows BPA to control the filters and ensure they are consistently applied across all audited sites. For a given site, we can set the filters once and periodically adjust as necessary.
To allow non-BPA tags to be used would require direct access to a site’s operating system and nearly constant monitoring of filters. Such an approach is not practical both in terms of the auditor time involved and site security concerns related to outside party access.
In summary, BPA chose not to be agnostic when it comes to tags since:
1. Tags from different analytic platforms define and count metrics differently meaning audit results are not comparable.
2. Non-BPA tags would require extensive monitoring that involves significant auditing manpower resources and the willingness of sites to allow outside system access for checking and testing purposes.
Comments