Forensic Tools for In-Depth Performance Investigations

October 15th, 2015

Another talk Philip Tellis and I gave at Velocity New York 2015 was about the forensic tools we use for investigating performance issues.  Check it out on Slideshare:

forensic-tools-for-in-depth-performance-investigations

In this talk, we cover a variety of tools such as WebPagetest, tcpdump, Wireshark, Cloudshark, browser developer tools, Chrome tracing, netlog, Fiddler, RUM, TamperMonkey, NodeJS, virtualization, Event Tracing for Windows (ETW), xperf and more while diving into real issues we’ve had to investigate in the past.

The talk is also available on YouTube.

Measuring the Performance of Single Page Applications

October 15th, 2015

Philip Tellis and I recently gave this talk at Velocity New York 2015.  Check out the slides on Slideshare:

measuring-the-performance-of-single-page-applications

In the talk, we discuss the three main challenges of measuring the performance of SPAs, and how we’ve been able to build SPA performance monitoring into Boomerang.

The talk is also available on YouTube.

UserTiming in Practice

May 29th, 2015

UserTiming is a specification developed by the W3C Web Performance working group, with the goal of giving the developer a standardized interface to log timestamps (“marks”) and durations (“measures”).

UserTiming utilizes the PerformanceTimeline that we saw in ResourceTiming, but all of the UserTiming events are put there by the you the developer (or the third-party scripts you’ve included in the page).

UserTiming is currently a Recommendation, which means that browser vendors are encouraged to implement it. As of early 2015, 59% of the world-wide browser market-share support NavigationTiming.

How was it done before?

Prior to UserTiming, developers have been keeping track of performance metrics, such as logging timestamps and event durations by using simple JavaScript:

var start = new Date().getTime();
// do stuff
var now = new Date().getTime();
var duration = now - start;

What’s wrong with this?

Well, nothing really, but… we can do better.

First, as discussed previously, Date().getTime() is not reliable.

Second, by logging your performance metrics into the standard interface of UserTiming, browser developer tools and third-party analytics services will be able to read and understand your performance metrics.

Marks and Measures

Developers generally use two core ideas to profile their code. First, they keep track of timestamps for when events happen. They may log these timestamps (e.g. via Date().getTime()) into JavaScript variables to be used later.

Second, developers often keep track of durations of events. This is often done by taking the difference of two timestamps.

Timestamps and durations correspond to “marks” and “measures” in UserTiming terms. A mark is a timestamp, in DOMHighResTimeStamp format. A measure is a duration, the difference between two marks, also measured in milliseconds.

How to use

Creating a mark or measure is done via the window.performance interface:

partial interface Performance {
    void mark(DOMString markName);

    void clearMarks(optional  DOMString markName);

    void measure(DOMString measureName, optional DOMString startMark,
        optional DOMString endMark);

    void clearMeasures(optional DOMString measureName);
};

interface PerformanceEntry {
  readonly attribute DOMString name;
  readonly attribute DOMString entryType;
  readonly attribute DOMHighResTimeStamp startTime;
  readonly attribute DOMHighResTimeStamp duration;
};

interface PerformanceMark : PerformanceEntry { };

interface PerformanceMeasure : PerformanceEntry { };

A mark (PerformanceMark) is an example of a PerformanceEntry, with no additional attributes:

  • name is the mark’s name
  • entryType is "mark"
  • startTime is the time the mark was created
  • duration is always 0

A measure (PerformanceMeasure) is also an example of a PerformanceEntry, with no additional attributes:

  • name is the measure’s name
  • entryType is "measure"
  • startTime is the startTime of the start mark
  • duration is the difference between the startTime of the start and end mark

Example Usage

Let’s start by logging a couple marks (timestamps):

// mark
performance.mark("start"); 
// -> {"name": "start", "entryType": "mark", "startTime": 1, "duration": 0}

performance.mark("end"); 
// -> {"name": "end", "entryType": "mark", "startTime": 2, "duration": 0}

performance.mark("another"); 
// -> {"name": "another", "entryType": "mark", "startTime": 3, "duration": 0}
performance.mark("another"); 
// -> {"name": "another", "entryType": "mark", "startTime": 4, "duration": 0}
performance.mark("another"); 
// -> {"name": "another", "entryType": "mark", "startTime": 5, "duration": 0}

Later, you may want to compare two marks (start vs. end) to create a measure (called diff), such as:

performance.measure("diff", "start", "end");
// -> {"name": "diff", "entryType": "measure", "startTime": 1, "duration": 1}

Note that measure() always calculates the difference by taking the latest timestamp that was seen for a mark. So if you did a measure against the another marks in the example above, it will take the timestamp of the third call to mark("another"):

performance.measure("diffOfAnother", "start", "another");
// -> {"name": "diffOfAnother", "entryType": "measure", "startTime": 1, "duration": 4}

There are many ways to create a measure:

  • If you call measure(name), the startTime is assumed to be window.performance.timing.navigationStart and the endTime is assumed to be now.
  • If you call measure(name, startMarkName), the startTime is assumed to be startTime of the given mark’s name and the endTime is assumed to be now.
  • If you call measure(name, startMarkName, endMarkName), the startTime is assumed to be startTime of the given start mark’s name and the endTime is assumed to be the startTime of the given end mark’s name.

Some examples of using measure():

// log the beginning of our task (assuming now is '1')
performance.mark("start");
// -> {"name": "start", "entryType": "mark", "startTime": 1, "duration": 0}

// do work (assuming now is '2')
performance.mark("start2");
// -> {"name": "start2", "entryType": "mark", "startTime": 2, "duration": 0}

// measure from navigationStart to now (assuming now is '3')
performance.measure("time to get to this point");
// -> {"name": "time to get to this point", "entryType": "measure", "startTime": 0, "duration": 3}

// measure from "now" to the "start" mark (assuming now is '4')
performance.measure("time to do stuff", "start");
// -> {"name": "time to do stuff", "entryType": "measure", "startTime": 1, "duration": 3}

// measure from "start2" to the "start" mark
performance.measure("time from start to start2", "start", "start2");
// -> {"name": "time from start to start2", "entryType": "measure", "startTime": 1, "duration": 1}

Once a mark or measure has been created, you can query for all marks, all measures, or specific marks/measures via the PerformanceTimeline. Here’s a review of the PerformanceTimeline methods:

window.performance.getEntries();
window.performance.getEntriesByType(type);
window.performance.getEntriesByName(name, type);
  • getEntries(): Gets all entries in the timeline
  • getEntriesByType(type): Gets all entries of the specified type (eg resource, mark, measure)
  • getEntriesByName(name, type): Gets all entries with the specified name (eg URL or mark name). type is optional, and will filter the list to that type.

Here’s an example of using the PerformanceTimeline to fetch a mark:

// performance.getEntriesByType("mark");
[
    {
        "duration":0,
        "startTime":1
        "entryType":"mark",
        "name":"start"
    },
    {
        "duration":0,
        "startTime":2,
        "entryType":"mark",
        "name":"start2"
    },
    ...
]

// performance.getEntriesByName("time from start to start2", "measure");
[
    {
        "duration":1,
        "startTime":1,
        "entryType":"measure",
        "name":"time from start to start2"
    }
]

Standard Mark Names

There are a couple of mark names that are suggested by the W3C specification to have special meanings:

  • mark_fully_loaded: The time when the page is considered fully loaded as marked by the developer in their application
  • mark_fully_visible: The time when the page is considered completely visible to an end-user as marked by the developer in their application
  • mark_above_the_fold: The time when all of the content in the visible viewport has been presented to the end-user as marked by the developer in their application
  • mark_time_to_user_action: The time of the first user interaction with the page during or after a navigation, such as scroll or click, as marked by the developer in their application

By using these standardized names, other third-party tools can pick up on your meanings and treat them specially (for example, by overlaying them on your waterfall).

Obviously, you can use these mark names for anything you want, and don’t have to stick by the recommended meanings.

Benefits

So why would you use UserTiming over just Date().getTime()?

First, it uses the PerformanceTimeline, so marks and measures are in the PerformanceTimeline along with other events

Second, it uses DOMHighResTimestamp instead of Date so the timestamps have sub-millisecond resolution, and are monotonically non-decreasing (so aren’t affected by the client’s clock).

UserTiming is very efficient, as the native browser runtime should be able to do math quicker and store things more performantly than your JavaScript runtime can.

Developer Tools

UserTiming marks and measures are currently available in the Internet Explorer F12 Developer Tools. They are called “User marks” and are shown as upside-down red triangles below:

UserTiming in IE F12 Dev Tools

Chrome and Firefox do not yet show marks or measures in their Developer Tools.

If you call console.timeStamp("foo"), a marker will show up in Chrome and Firefox timelines. However, if you do both console.timeStamp() and performance.mark(), you will get two markers in Internet Explorer.

So for now, if you want a cross-browser compatible way of showing a single marker in dev tools, you would have to sniff the UA (which is generally not recommended). There is an open bug on Chromium to get UserTiming marks to show in the dev tools.

Use Cases

How could you use UserTiming? Here are some ideas:

  • Any place that you’re already logging timestamps or calculating durations could be switched to UserTiming
  • Easy way to add profiling events to your application
  • Note important scenario durations in your Performance Timeline
  • Measure important durations for analytics

Compressing

If you’re adding UserTiming instrumentation to your page, you probably also want to consume it. One way is to grab everything, package it up, and send it back to your own server for analysis.

In my UserTiming Compression article, I go over a couple ways of how to do this. Versus just sending the UserTiming JSON, usertiming-compression.js can reduce the byte size down to just 10-15% of the original.

Availability

UserTiming is available in most modern browsers. According to caniuse.com 59% of world-wide browser market share supports ResourceTiming, as of May 2015. This includes Internet Explorer 10+, Firefox 38+, Chrome 25+, Opera 15+ and Android Browser 4.4+.

CanIUse - UserTiming

As usual, Safari (iOS and Mac) doesn’t yet include support for UserTiming.

However, if you want to use UserTiming for everything, there are polyfills available that work 100% reliably in all browsers.

I have one such polyfill, UserTiming.js, available on Github.

DIY / Open Source / Commercial

If you want to use UserTiming, you could easily compress and beacon the data to your back-end for processing.

WebPageTest sends UserTiming to Google Analytics, Boomerang and SOASTA mPulse:

WebPageTest UserTiming

SOASTA mPulse collects UserTiming information for any Custom Timers you specify (I work at SOASTA, on mPulse and Boomerang):

SOASTA mPulse

Conclusion

UserTiming is a great interface to log your performance metrics into a standardized interface. As more services and browsers support UserTiming, you will be able to see your data in more and more places.

That wraps up our talk about how to monitor and measure the performance of your web apps. Hope you enjoyed it.

Other articles in this series:

More resources:

Updates

  • 2015-12-01: Added Compressing section

ResourceTiming in Practice

May 28th, 2015

ResourceTiming is a specification developed by the W3C Web Performance working group, with the goal of exposing accurate performance metrics about all of the resources downloaded during the page load experience, such as images, CSS and JavaScript.

ResourceTiming builds on top of the concepts of NavigationTiming and provides many of the same measurements, such as the timings of each resource’s DNS, TCP, request and response phases, along with the final “loaded” timestamp.

ResourceTiming takes its inspiration from resource Waterfalls. If you’ve ever looked at the Networking tab in Internet Explorer, Chrome or Firefox developer tools, you’ve seen a Waterfall before. A Waterfall shows all of the resources fetched from the network in a timeline, so you can quickly visualize any issues. Here’s an example from the Chrome Developer Tools:

ResourceTiming inspiration

ResourceTiming is currently a Working Draft, which means it is still a work-in-progress, but many browsers already support it. As of early 2015, 67% of the world-wide browser market-share supports ResourceTiming.

How was it done before?

Prior to ResourceTiming, you could measure the time it took to download resources on your page by hooking into the associated element’s onload event, such as for Images.

Take this example code:

var start = new Date().getTime();
var image1 = new Image();
var loadCallback = function() {
    var now = new Date().getTime();
    var latency = now - start;
    alert("End to end resource fetch: " + latency);
};

image1.onload = loadCallback;
image1.src = 'http://foo.com/image.png';

With the code above, the image is inserted into the DOM when the script runs, at which point it sets the start variable to the current time. The image’s onload attribute is set to loadCallback, which calculates how long it took for the resource to be fetched.

While this is one method for measuring the download time of an image, it’s not very practical.

First of all, it only measures the end-to-end download time, plus any overhead required for the browser to fire the onload callback. For images, this could also include the time it takes to parse and render the image. You cannot get a breakdown of DNS, TCP, SSL, request or response times with this method.

Another issue is with the use of Date.getTime(), which has some major drawbacks. See our discussion on DOMHighResTimeStamp in the NavigationTiming discussion for more details.

Most importantly, to use this method you have to construct your entire web app dynamically, at runtime. Dynamically adding all of the elements that would trigger resource fetches in <script> tags is not practical, nor performant. You would have to insert all <img>, <link rel="stylesheet">, and <script> tags to instrument everything. Doing this via JavaScript is not performant, and the browser cannot pre-fetch resources that would have otherwise been in the HTML.

Finally, it’s impossible to measure all resources that are fetched by the browser using this method. For example, it’s not possible to hook into stylesheets or fonts defined via @import or @font-face statements.

ResourceTiming addresses all of these problems.

How to use

ResourceTiming data is available via several methods on the window.performance interface:

window.performance.getEntries();
window.performance.getEntriesByType(type);
window.performance.getEntriesByName(name, type);

Each of these functions returns a list of PerformanceEntrys. getEntries() will return a list of all entries in the PerformanceTimeline (see below), while if you use getEntriesByType("resource") or getEntriesByName("foo", "resource"), you can limit your query to just entries of the type PerformanceResourceTiming, which inherits from PerformanceEntry.

That may sound confusing, but when you look at the array of ResourceTiming objects, they’ll simply have a combination of the attributes below. Here’s the WebIDL (definition) of a PerformanceEntry:

interface PerformanceEntry {
    readonly attribute DOMString name;
    readonly attribute DOMString entryType;

    readonly attribute DOMHighResTimeStamp startTime;
    readonly attribute DOMHighResTimeStamp duration;
};

Each PerformanceResourceTiming is a PerformanceEntry, so has the above attributes, as well as the attributes below:

interface PerformanceResourceTiming : PerformanceEntry {
    readonly attribute DOMString initiatorType;

    readonly attribute DOMHighResTimeStamp redirectStart;
    readonly attribute DOMHighResTimeStamp redirectEnd;
    readonly attribute DOMHighResTimeStamp fetchStart;
    readonly attribute DOMHighResTimeStamp domainLookupStart;
    readonly attribute DOMHighResTimeStamp domainLookupEnd;
    readonly attribute DOMHighResTimeStamp connectStart;
    readonly attribute DOMHighResTimeStamp connectEnd;
    readonly attribute DOMHighResTimeStamp secureConnectionStart;
    readonly attribute DOMHighResTimeStamp requestStart;
    readonly attribute DOMHighResTimeStamp responseStart;
    readonly attribute DOMHighResTimeStamp responseEnd;
};

Interlude: PerformanceTimeline

The PerformanceTimeline is a critical part of ResourceTiming, and is the interface that you use to fetch ResourceTiming data, as well as other performance information, such as UserTiming data.

The methods getEntries(), getEntriesByType() and getEntriesByName() that you saw above are the primary interfaces of the PerformanceTimeline. The idea is to expose all browser performance information via a standard interface.

All browsers that support ResourceTiming (or UserTiming) will also support the PerformanceTimeline.

Here are the primary methods:

  • getEntries(): Gets all entries in the timeline
  • getEntriesByType(type): Gets all entries of the specified type (eg resource, mark, measure)
  • getEntriesByName(name, type): Gets all entries with the specified name (eg URL or mark name). type is optional, and will filter the list to that type.

We’ll use the PerformanceTimeline to fetch ResourceTiming data (and UserTiming data in the next article).

Back to ResourceTiming

ResourceTiming takes its inspiration from the NavigationTiming timeline.

Here are the phases a single resource would go through during the fetch process:

ResourceTiming timeline

To fetch all of the resources on a page, you simply call one of the PerformanceTimeline methods:

var resources = window.performance.getEntriesByType("resource");

/* eg:
[
    {
        name: "https://www.foo.com/foo.png",
        entryType: "resource",
        startTime: 566.357000003336,
        duration: 4.275999992387369,
        initiatorType: "img",
        redirectEnd: 0,
        redirectStart: 0,
        fetchStart: 566.357000003336,
        domainLookupStart: 566.357000003336,
        domainLookupEnd: 566.357000003336,
        connectStart: 566.357000003336,
        secureConnectionStart: 0,
        connectEnd: 566.357000003336,
        requestStart: 568.4959999925923,
        responseStart: 569.4220000004862,
        responseEnd: 570.6329999957234
    }, ...
]
*/

Looking at these attributes, you can see they are a combination of the base PerformanceEntry attributes (i.e. name, entryType, startTime and duration) with additional PerformanceResourceTiming attributes that are specific to ResourceTiming.

Please note that all of the timestamps are DOMHighResTimeStamps, so they are relative to window.performance.timing.navigationStart. Thus a value of 500 means 500 milliseconds after the page load started.

Here is a description of all of the ResourceTiming attributes:

  • name is the fully-resolved URL of the attribute (relative URLs in your HTML will be expanded to include the full protocol, domain name and path)
  • entryType will always be "resource" for ResourceTiming entries
  • startTime is the time a resource started being fetched
  • initiatorType is the localName of the element that initiated the fetch of the resource (see below for details)
  • redirectStart and redirectEnd encompass the time it took to fetch any previous resources that redirected to the final one listed. If either timestamp is 0, there were no redirects, or one of the redirects wasn’t from the same origin as this resource.
  • fetchStart is the time this specific resource started being fetched, not including redirects
  • domainLookupStart and domainLookupEnd are the timestamps for DNS lookups
  • connectStart and connectEnd are timestamps for the TCP connection
  • secureConnectionStart is the start timestamp of the SSL handshake, if any. If the connection was over HTTP, or if the browser doesn’t support this timestamp (eg. Internet Explorer), it will be 0.
  • requestStart is the timestamp that the browser started to request the resource from the remote server
  • responseStart and responseEnd are the timestamps for the start of the response and when it finished downloading
  • duration is the overall time required to fetch the resource

duration will include time to fetch all redirected resources (if any) as well as the final resource. To track the overall time it took to fetch just the final resource, you may want to use (responseEndfetchStart).

initiatorType is the localName of the element that fetched the resource — in other words, the name of the associated HTML element. The most common values seen for this attribute are:

  • img
  • link
  • script
  • css: url(), @import
  • xmlhttprequest
  • iframe

Note that CSS may have an initiatorType of "link" or "css" because it can either be triggered via a <link> tag or via an @import in a CSS file.

The iframe initiator type is for <IFRAME>s on the page, and the duration will be how long it took to load that frame (e.g. how long it took for the frame’s onload event to fire). This means that the duration will include any statically-included sub-resources within that frame. This is good news, as it can give you insight into cross-origin <IFRAME>s (such as advertisements) that you normally can’t access from ResourceTiming.

ResourceTiming does not (yet) include attributes that expose the HTTP status code of the resource, the number of transferred bytes, or the final object size.

What Resources are included

All of the resources that your browser fetches to construct the page should be listed in the ResourceTiming data. This includes, but is not limited to images, scripts, css, fonts, videos, IFRAMEs and XHRs.

Some browsers (eg. Internet Explorer) may include other non-fetched resources, such as about:blank and javascript: URLs in the ResourceTiming data. This is likely a bug and may be fixed in upcoming versions, but you may want to filter out non-http: and https: protocols.

Additionally, some browser extensions may trigger downloads and thus you may see some of those downloads in your ResourceTiming data as well.

Not all resources will be fetched successfully. There might have been a networking error, due to a DNS, TCP or SSL/TLS negotiation failure. Or, the server might return a 4xx or 5xx response. How this information is surfaced in ResourceTiming depends on the browser:

  • DNS failure
    • Chrome: No ResourceTiming event
    • Internet Explorer: domainLookupStart through responseStart are 0. responseEnd and duration are non-zero.
    • Firefox: domainLookupStart through responseStart are 0. duration is 0 in some cases. responseEnd is non-zero.
  • TCP failure
    • Chrome: No ResourceTiming event
    • Internet Explorer: domainLookupStart through responseStart are 0. responseEnd and duration are non-zero.
    • Firefox: domainLookupStart through responseStart and duration are 0. responseEnd is non-zero.
  • 4xx/5xx response
    • Chrome: No ResourceTiming event
    • Internet Explorer: All timestamps are non-zero.
    • Firefox: All timestamps are non-zero (though duration is 0 in FF < 41)

The working group is attempting to get these behaviors more consistent across browsers. You can read this post for further details.

Note that the root page (your HTML) is not included in ResourceTiming. You can get all of that data from NavigationTiming.

Cached Resources

Cached resources will show up in ResourceTiming right along side resources that were fetched from the network.

There’s no direct indication on the cached resource that it was served from the cache. In practice resources with a very short duration (say under 10 milliseconds) are likely to have been served from the browser’s cache. They might take a few milliseconds due to disk latencies.

There’s no definitive indication that a resource was served from cache due to privacy concerns (have-you-been-to-X-before attack).

The ResourceTiming Buffer

There is a ResourceTiming buffer (per document / IFRAME) that stops filling after its limit is reached. By default, all modern browsers currently set this limit to 150 entries.

The reasoning behind limiting the number of entries is to ensure that, for the vast majority of websites that are not consuming ResourceTiming entries, the browser’s memory isn’t consumed indefinitely holding on to a lot of this information. In addition, for sites that periodically fetch new resources (such as XHR polling), we would’t want the ResourceTiming buffer to grow unbound.

Thus, if you will be consuming ResourceTiming data, you need to have awareness of the buffer. If your site only downloads a handful of resources for each page load (< 100), and does nothing afterwards, you probably won’t hit the limit.

However, if your site downloads over a hundred resources, or you want to be able to monitor for resources fetched on an ongoing basis, you can do one of two things.

First, you can listen for the onresourcetimingbufferfull event which gets fired on the document when the buffer is full. You can then use setResourceTimingBufferSize(n) or clearResourceTimings() to resize or clear the buffer.

As an example, to keep the buffer size at 150 yet continue tracking resources after the first 150 resources were added, you could do something like this;

if ("performance" in window) {
  function onBufferFull() {
    var latestEntries = performance.getEntriesByType("resource");
    performance.clearResourceTimings();

    // analyze or beacon latestEntries, etc
  }

  performance.onresourcetimingbufferfull = performance.onwebkitresourcetimingbufferfull = onBufferFull;
}

Note onresourcetimingbufferfull is not currently supported in Internet Explorer (10, 11 or Edge).

If your site is on the verge of 150 resources, and you don’t want to manage the buffer, you could also just safely increase the buffer size to something reasonable in your HTML header:

<html><head>
<script>
if ("performance" in window 
    && window.performance 
    && window.performance.setResourceTimingBufferSize) {
    performance.setResourceTimingBufferSize(300);
}
</script>
...
</head>...

Note: As of Chrome 42, these are still prefixed as webkitSetResourceTimingBufferSize and webkitClearResourceTimings.

Don’t just setResourceTimingBufferSize(99999999) as this could grow your visitors’s browser’s memory unnecessarily.

Compressing ResourceTiming Data

The HTTP Archive tells us there are about 100 HTTP resources on average, per page, with an average URL length of 85 bytes.

On average, each resource is ~ 500 bytes when JSON.stringify()‘d.

That means you could expect around 45 KB of ResourceTiming data per page load on the “average” site.

If you’re considering beaconing ResourceTiming data back to your own servers for analysis, you may want to consider compressing it first.

There’s a couple things you can do to compress the data, and I’ve written about these methods already. I’ve shared an open-source script that can compress ResourceTiming data that looks like this:

{
    "responseEnd":323.1100000002698,
    "responseStart":300.5000000000000,
    "requestStart":252.68599999981234,
    "secureConnectionStart":0,
    "connectEnd":0,
    "connectStart":0,
    "domainLookupEnd":0,
    "domainLookupStart":0,
    "fetchStart":252.68599999981234,
    "redirectEnd":0,
    "redirectStart":0,
    "duration":71.42400000045745,
    "startTime":252.68599999981234,
    "entryType":"resource",
    "initiatorType":"script",
    "name":"http://foo.com/js/foo.js"
}

To something much smaller, like this (which contains 3 resources):

{
    "http://": {
        "foo.com/": {
            "js/foo.js": "370,1z,1c",
            "css/foo.css": "48c,5k,14"
        },
        "moo.com/moo.gif": "312,34,56"
    }
}

Overall, we can compresses ResourceTiming data down to about 15% of its original size.

Example code to do this compression is available on github.

Timing-Allow-Origin

A cross-origin resource is any resource that doesn’t originate from the same domain as the page. For example, if your visitor is on http://foo.com/ and you’ve fetched resources from http://cdn.foo.com or http://mycdn.com, those resources will both be considered cross-origin.

By default, cross-origin resources expose timestamps for only the fetchStart and responseEnd attributes. This is to protect your privacy (so an attacker can’t load random URLs to see where you’ve been). This means that all of the following attributes will be 0 for cross-origin resources:

  • domainLookupStart
  • domainLookupEnd
  • connectStart
  • connectEnd
  • secureConnectionStart
  • requestStart
  • responseStart

Luckily, if you control the domains you’re fetching other resources from, you can overwrite this default precaution by sending down the Timing-Allow-Origin HTTP response header:

Timing-Allow-Origin = "Timing-Allow-Origin" ":" origin-list-or-null | "*"

In practice, most people that send the Timing-Allow-Origin HTTP header just send a wildcard origin:

Timing-Allow-Origin: *

So if you’re serving any of your content from another domain name, i.e. from a CDN, it is strongly recommended that you set the Timing-Allow-Origin header for those responses.

Thankfully, third-party libraries for widgets, ads, analytics, etc are starting to set the header on their content. Only about 5% currently do, but this is growing (according to the HTTP Archive). Notably, Google, Facebook, Disqus, and mPulse send this header for their scripts.

Blocking Time

Browsers will only open a limited number of connections to each unique origin (protocol/server name/port) when downloading resources.

If there are more resources than the # of connections, the later resources will be “blocking”, waiting for their turn to download.

Blocking time is generally seen as “missing periods” (non-zero durations) that occur between connectEnd and requestStart (when waiting on a Keep-Alive TCP connection to reuse), or between fetchStart and domainLookupStart (when waiting on things like the browser’s cache).

The duration attribute includes Blocking time. So in general, you may not want to use duration if you’re only interested in actual network timings. Unfortunately, duration, startTime and responseEnd are the only attributes you get with cross-origin resources, so you can’t easily subtract out Blocking time from cross-origin resources.

To calculate Blocking time, you would do something like this:

var blockingTime = 0;
if (res.connectEnd && res.connectEnd === res.fetchStart) {
    blockingTime = res.requestStart - res.connectEnd;
} else if (res.domainLookupStart) {
    blockingTime = res.domainLookupStart - res.fetchStart;
}

ServiceWorkers

If you are using ServiceWorkers in your app, there is an ongoing discussion to update ResourceTiming so you can get information about how long it took for the ServiceWorker to process each resource.

There is a new workerStart attribute that will be exposed. The difference between workerStart and fetchStart is the processing time of the ServiceWorker:

var workerProcessingTime = 0;
if (res.workerStart && res.fetchStart) {
    workerProcessingTime = res.fetchStart - res.workerStart;
}

This isn’t available in any browsers yet, but there are open bugs to add it.

Use Cases

Now that resource timing data is available in the browser in an accurate and reliable manner, there are a lot of things you can do with the information. Here are some ideas:

  • Send all resource timings to your backend analytics
  • Raise an analytics event if any resource takes over X seconds to download (and trend this data)
  • Watch specific resources (eg third-party ads or analytics) and complain if they are slow
  • Monitor the overall health of your DNS infrastructure by beaconing DNS resolve time per-domain
  • Look for production resource errors (eg 4xx/5xx) in browsers that add errors to the buffer it (IE/Firefox)
  • Use ResourceTiming to determine your site’s “visual complete” metric by looking at timings of all above-the-fold images

The possibilities are nearly endless. Please leave a comment with how you’re using ResourceTiming data.

DIY and Open-Source

Here are several interesting DIY / open-source solutions that utilize ResourceTiming data:

Andy Davies’ Waterfall.js shows a waterfall of any page’s resources via a bookmarklet: github.com/andydavies/waterfall

Andy Davies' Waterfall.js

Mark Zeman’s Heatmap bookmarklet / Chrome extension gives a heatmap of when images loaded on your page: github.com/zeman/perfmap

Mark Zeman's Heatmap bookmarklet and extension

Nurun’s Performance Bookmarklet breaks down your resources and creates a waterfall and some interesting charts: github.com/nurun/performance-bookmarklet

Nurun's Performance Bookmarklet

Boomerang also captures ResourceTiming data and beacons it back to your backend analytics server: github.com/lognormal/boomerang

Commercial Solutions

If you don’t want to build or manage a DIY / Open-Source solution to gather ResourceTiming data, there are many great commercial services available.

Disclaimer: I work at SOASTA, on mPulse and Boomerang

SOASTA mPulse captures 100% of your site’s traffic and gives you Waterfalls for each visit:

SOASTA mPulse Resource Timing

New Relic Browser:

New Relic Browser

App Dynamics Web EUEM:

App Dynamics Web EUEM

Availability

ResourceTiming is available in most modern browsers. According to caniuse.com, 67% of world-wide browser market share supports ResourceTiming (as of May 2015). This includes Internet Explorer 10+, Firefox 36+, Chrome 25+, Opera 15+, and Android Browser 4.4+.

CanIUse - ResourceTiming - May 2015

Again, notably absent, is Safari (both iOS and Mac). This means a significant share of the mobile browser market does not have performance timing data available, which is a shame. Let’s hope that changes soon.

There are no polyfills available for ResourceTiming, as the data is just simply not available if the browser doesn’t expose it.

Tips

Here are some additional (and re-iterated) tips for using ResourceTiming data:

  • For many sites, most of your content will not be same-origin, so ensure all of your CDNs and third-party libraries send the Timing-Allow-Origin HTTP response header.
  • ResourceTiming data does not include some aspects of the resource, such as it’s transfer size, content size or HTTP code for privacy concerns.
  • If you’re going to be managing the ResourceTiming buffer, make sure no other scripts are managing it as well (eg third-party analytics scripts). Otherwise, you may have two listeners for onresourcetimingbufferfull stomping on each other.
  • The duration attribute includes Blocking time (when a resource is blocked behind other resources on the same socket).
  • Each IFRAME will have its own ResourceTiming data, and those resources won’t be included in the parent FRAME/document. You’ll need to traverse the document frames to get all resources. See github.com/nicjansma/resourcetiming-compression.js.
  • about:blank and javascript: URLs may be in the ResourceTiming data for some browsers, and you may want to filter them out.
  • Browser extensions may show up in ResourceTiming data, if they initiate downloads. We’ve seen Skype and other extensions show up.

ResourceTiming – Coming Soon

All of the notes above document what current browsers support as of May 2015. There is ongoing work on the specification to add some additional attributes. Some of the new attributes may include:

  • nextHopProtocol: ALPN Protocol ID
  • transferSize: Bytes transferred for HTTP header and response
  • decodedBodySize: Size of the body after removing any applied content-codings
  • encodedBodySize: Size of the body after prior to removing any applied content-codings

In addition, there is an ongoing discussion for a more performant way to observe new ResourceTiming entries instead of polling getEntriesByType().

Conclusion

ResourceTiming exposes accurate performance metrics for all of the resources fetched on your page. You can use this data for a variety of scenarios, from investigating the performance of your third-party libraries to taking specific actions when resources aren’t performing according to your performance goals.

Next up: Using UserTiming data to expose custom metrics for your JavaScript apps in a standardized way.

Other articles in this series:

More resources:

Updates

  • 2016-01-03: Updated Firefox’s 404 and DNS behavior via Aaron Peters

NavigationTiming in Practice

May 27th, 2015

NavigationTiming is a specification developed by the W3C Web Performance working group, with the goal of exposing accurate performance metrics that describe your visitor’s page load experience (via JavaScript).

NavigationTiming was the first specification developed by the working group, and exposes valuable performance metrics to JavaScript that were impossible to accurately measure before.

NavigationTiming is currently a Recommendation, which means that browser vendors are encouraged to implement it. As of May 2015, 81% of the world-wide browser market-share supports NavigationTiming (and more on that later).

There’s a follow-up spec, NavigationTiming2 that is currently being developed and improves on the original spec.

Let’s take a deep-dive into NavigationTiming.

How it was done before?

NavigationTiming exposes performance metrics to JavaScript that were never before available, such as your root HTML’s network timings. Prior to NavigationTiming, you could not measure your page’s DNS, TCP, request or response times because all of that occurred before your application (JavaScript) started up, and the browser did not expose them.

Before NavigationTiming was available, you could still estimate some performance metrics, such as how long it took for your page’s sub-resources to download. To do this, you can hook into the browser’s onload event, which is fired once all of the sub-resources on your page (such as JavaScript, CSS, IMGs and IFRAMES) have been downloaded.

Here’s sample code:

<html><head><script>
var start = new Date().getTime();

function onLoad {
  var pageLoadTime = (new Date().getTime()) - start;
}

body.addEventListener(“load”, onLoad, false);
</script></head></html>

What’s wrong with this?

First, it only measures the time from when the JavaScript runs to when the last sub-resource is downloaded.

If that’s all you’re interested in measuring, that’s fine, but there’s a large part of the user’s experience that you’ll be blind to.

Let’s review the main phases that the browser goes through when fetching your HTML:

  1. DNS resolve: Look up the domain name to find what IP address to connect to
  2. TCP connect: Connect to your server on port 80 (HTTP) or 443 (HTTPS) via TCP
  3. Request: Send a HTTP request, with headers and cookies
  4. Response: Wait for the server to start sending the content (back-end time)

It’s only after Phase 4 (Response) is complete that your HTML is parsed and your scripts would start running.

Phase 1-4 timings will vary depending on the network. One visitor might fetch your content in 100 ms while it might take another user, on a slower connection, 5,000 ms before they see your content. That delay translates into a much worse user-experience.

Thus if you’re only monitoring your application from JavaScript in the <HEAD> to to onload (in the snippet above), you are blind to a large part of the overall experience.

So the primitive approach above has several downsides:

  • It only measures the time from when the JavaScript runs to when the last sub-resource is downloaded
  • It misses the initial DNS lookup, TCP connection and HTTP request phases
  • Date().getTime() is not reliable

Interlude – DOMHighResTimeStamp

What about #3? Why is Date.getTime() (or Date.now() or +(new Date)) not reliable?

Let’s talk about another modern browser feature, DOMHighResTimeStamp.

DOMHighResTimeStamp is a new data type for performance interfaces. In JavaScript, it’s typed as a regular number primitive, but anything that exposes a DOMHighResTimeStamp is following several conventions.

Notably, DOMHighResTimeStamp is a monotonically non-decreasing timestamp with an epoch of navigationStart and sub-millisecond resolution. It is used by several W3C webperf performance specs, and can always be queried via window.performance.now();

Why not just use the Date object?

DOMHighResTimeStamp helps solve three shortcomings of Date. Let’s break its definition down:

  • monotonically non-decreasing means that every time you fetch a DOMHighResTimeStamp, its’ value will always be at least the same as when you accessed it last. It will never decrease.

  • timestamp with an epoch of navigationStart means it’s value is a timestamp, whose basis (start) is window.performance.timing.navigationStart. Thus a DOMHighResTimeStamp of 10 means it’s 10 milliseconds after time time given by navigationStart

  • sub-millisecond resolution means the value has the resolution of at least a millisecond. In practice, DOMHighResTimeStamps will be a number with the milliseconds as whole-numbers and fractions of a millisecond represented after the decimal. For example, 1.5 means 1500 microseconds, while 100.123 means 100 milliseconds and 123 microseconds.

Each of these points addresses a shortcoming of the Date object. First and foremost, monotonically non-decreasing fixes a subtle issue with the Date object that you may not know exists. The problem is that Date simple exposes the value of your end-user’s clock, according to the operating system. While the majority of the time this is OK, the system clock can be influenced by outside events, even in the middle of when your app is running.

For example, when the user changes their clock, or an atomic clock service adjusts it, or daylight-savings kicks in, the system clock may jump forward, or even go backwards!

So imagine you’re performance-profiling your application by keeping track of a start and end timestamp of some event via the Date object. You track the start time… and then your end-users atomic clock kicks in and adjusts the time forward an hour… and now, from JavaScript Date‘s point of view, it seems like your application just took an hour to do a simple task.

This can even lead to problems when doing statistical analysis of your performance data. Imagine if your monitoring tool is taking the mean value of operational times and one of your users’ clocks jumped forward 10 years. That outlier, while “true” from the point of view of Date, will skew the rest of your data significantly.

DOMHighResTimeStamp addresses this issue by guaranteeing it is monotonically non-decreasing. Every time you access performance.now(), you are guaranteed it will be at least equal to, if not greater than, the last time you accessed it.

You should’t mix Date timestamps (which are Unix epoch based, so you get sample times like 1430700428519) with DOMHighResTimeStamps. If the user’s clock changes, and you mix both Date and DOMHighResTimeStamps, the former could be wildly different from the later.

To help enforce this, DOMHighResTimeStamp is not Unix epoch based. Instead, its epoch is window.performance.timing.navigationStart (more details of which are below). Since it has sub-millisecond resolution, this means that the values that you get from it are the number of milliseconds since the page load started. As a benefit, this makes them easier to read than Date timestamps, since they’re relatively small and you don’t need to do (now - startTime) math to know when something started running.

DOMHighResTimeStamp is available in most modern browsers, including Internet Explorer 10+, Firefox 15+, Chrome 20+, Safari 8+ and Android 4.4+. If you want to be able to always get timestamps via window.performance.now(), you can use a polyfill. Note these polyfills will be millisecond-resolution timestamps with a epoch of “something” in unsupported browsers, since monotonically non-decreasing can’t be guaranteed and sub-millisecond isn’t available unless the browser supports it.

As a summary:

DateDOMHighResTimeStamp
Accessed viaDate().getTime()performance.now()
Resolutionmillisecondsub-millisecond
StartUnix epochperformance.timing.navigationStart
Monotonically Non-decreasingNoYes
Affected by user’s clockYesNo
Example14201475246063392.275999998674

Back to NavigationTiming

So, how do you access NavigationTiming data?

All of the performance metrics NavigationTiming exposes are available underneath the new window.performance DOM object that most of the W3C webperf specs utilize.

NavigationTiming’s metrics are primarily available underneath window.performance.navigation and window.performance.timing. The former provides performance characteristics (such as the type of navigation, or the number of redirects taken to get to the current page) while the latter exposes performance metrics (timestamps).

Here’s the WebIDL (definition) of the interfaces:

window.performance.navigation:

interface PerformanceNavigation {
  const unsigned short TYPE_NAVIGATE = 0;
  const unsigned short TYPE_RELOAD = 1;
  const unsigned short TYPE_BACK_FORWARD = 2;
  const unsigned short TYPE_RESERVED = 255;
  readonly attribute unsigned short type;
  readonly attribute unsigned short redirectCount;
};

window.performance.timing:

interface PerformanceTiming {
    readonly attribute unsigned long long navigationStart;
    readonly attribute unsigned long long unloadEventStart;
    readonly attribute unsigned long long unloadEventEnd;
    readonly attribute unsigned long long redirectStart;
    readonly attribute unsigned long long redirectEnd;
    readonly attribute unsigned long long fetchStart;
    readonly attribute unsigned long long domainLookupStart;
    readonly attribute unsigned long long domainLookupEnd;
    readonly attribute unsigned long long connectStart;
    readonly attribute unsigned long long connectEnd;
    readonly attribute unsigned long long secureConnectionStart;
    readonly attribute unsigned long long requestStart;
    readonly attribute unsigned long long responseStart;
    readonly attribute unsigned long long responseEnd;
    readonly attribute unsigned long long domLoading;
    readonly attribute unsigned long long domInteractive;
    readonly attribute unsigned long long domContentLoadedEventStart;
    readonly attribute unsigned long long domContentLoadedEventEnd;
    readonly attribute unsigned long long domComplete;
    readonly attribute unsigned long long loadEventStart;
    readonly attribute unsigned long long loadEventEnd;
};

The NavigationTiming Timeline

Each of the timestamps above corresponds with events in the timeline below:

NavigationTiming timeline

Note that each of the timestamps are Unix epoch-based, instead of being navigationStart-based like DOMHighResTimeStamps. This has been addressed in NavigationTiming2.

The entire process starts at timing.navigationStart. This is when your end-user started the navigation. They might have clicked on a link, or hit reload in your browser. The navigation.type property tells you what type of page-load it was: a regular navigation (link- or bookmark- click) (TYPE_NAVIGATE = 0), a reload (TYPE_RELOAD = 1), or a back-forward navigation (TYPE_BACK_FORWARD = 2). Each of these types of navigations will have different performance characteristics.

Around this time, the browser will also start to unload the previous page. If the previous page is the same origin (domain) as the current page, the timestamps of that document’s onunload event (start and end) will be filled in as timing.unloadEventStart and timing.unloadEventEnd. If the previous page was on another origin (or there was no previous page), these timestamps will be 0.

Next, in some cases, your site may go through one or more HTTP redirects before it reaches the final destination. navigation.redirectCount gives you an important insight into how many hops it took for your visitor to reach your page. 301 and 302 redirects each take time, so for performance reasons you should reduce the number of redirects to reach your content to 0 or 1. Unfortunately, due to security concerns, you do not have access to the actual URLs that redirected to this page, and it is entirely possibly that a third-party site (not under your control) initiated the redirect. The difference between timing.redirectStart and timing.redirectEnd encompasses all of the redirects. If these values are 0, it means that either there were no redirects, or at least one of the redirects was from a different origin.

fetchStart is the next timestamp, and indicates the timestamp for the start of the fetch of the current page. If there were no redirects when loading the current page, this value should equal navigationStart. Otherwise, it should equal redirectEnd.

Next, the browser goes through the networking phases required to fetch HTML over HTTP. First the domain is resolved (domainLookupStart and domainLookupEnd), then a TCP connection is initiated (connectStart and connectEnd). Once connected, a HTTP request (with headers and cookies) is sent (requestStart). Once data starts coming back from the server, responseStart is filled, and is ended when the last byte from the server is read at responseEnd.

Note that the only phase without an end timestamp is requestEnd, as the browser does not have insight into when the server received the response.

Any of the above phases (DNS, TCP, request or response) might not take any time, such as when DNS was already resolved, a TCP connection is re-used or when content is served from disk. In this case, the timestamps should not be 0, but should reflect the timestamp that the phase started and ended, even if the duration is 0. For example, if fetchStart is at 1000 and a TCP connection is reused, domainLookupStart, domainLookupEnd, connectStart and connectEnd should all be 1000 as well.

secureConnectionStart is an optional timestamp that is only supported in some browsers (notably, it is missing in Internet Explorer). If not 0, it represents the time that the SSL/TLS handshake started.

After responseStart, there are several timestamps that represent phases of the DOM’s lifecycle. These are domLoading, domInteractive, domContentLoadedEventStart, domContentLoadedEventEnd and domComplete.

domLoading, domInteractive and domComplete correspond to when the Document’s readyState are set to the corresponding loading, interactive and complete states.

domContentLoadedEventStart and domContentLoadedEventEnd correspond to when the DOMContentLoaded event fires on the document and when it has completed running.

Finally, once the body’s onload event fires, loadEventStart is filled in. Once all of the onload handlers are complete, loadEventEnd is filled in. Note this means if you’re querying window.performance.timing from within the onload event, loadEventEnd will be 0. You could work around this by querying the timestamps from a setTimeout(..., 10) fired from within the onload event, as in the code example below.

Note: There is a bug in some browsers where they are reporting 0 for some timestamps. This is a bug, as all same-origin timestamps should be filled in, but if you’re consuming this data, you may have to adjust for this.

Browser vendors are also free to ad their own additional timestamps to window.performance.timing. Here is the only currently known vendor-prefixed timestamp available:

  • msFirstPaint – Internet Explorer 9+ only, this event corresponds to when the first paint occurred within the document. It makes no guarantee about what content was painted — in fact, the paint could be just the “white out” prior to other content being displayed. Do not rely on this event to determine when the user started seeing actual content.

Example data

Here’s sample data from a page load:

// window.performance.navigation
redirectCount: 0
type: 0

// window.performance.timing
navigationStart: 1432762408327,
unloadEventEnd: 0,
unloadEventStart: 0,
redirectStart: 0,
redirectEnd: 0,
fetchStart: 1432762408648,
connectEnd: 1432762408886,
secureConnectionStart: 1432762408777,
connectStart: 1432762408688,
domainLookupStart: 1432762408660,
domainLookupEnd: 1432762408688,
requestStart: 1432762408886,
responseStart: 1432762409141,
responseEnd: 1432762409229,
domComplete: 1432762411136,
domLoading: 1432762409147,
domInteractive: 1432762410129,
domInteractive: 1432762410129,
domContentLoadedEventStart: 1432762410164,
domContentLoadedEventEnd: 1432762410263,
loadEventEnd: 1432762411140,
loadEventStart: 1432762411136

How to Use

All of the metrics exposed on the window.performance interface are available to your application via JavaScript. Here’s example code for gathering durations of the different phases of the main page load experience:

function onLoad() {
  if ('performance' in window && 'timing' in window.performance) {
    setTimeout(function() {
      var t = window.performance.timing;
      var ntData = {
        redirect: t.redirectEnd - t.redirectStart,
        dns: t.domainLookupEnd - t.domainLookupStart,
        connect: t.connectEnd - t.connectStart,
        ssl: t.secureConnectionStart ? (t.connectEnd - secureConnectionStart) : 0,
        request: t.responseStart - t.requestStart,
        response: t.responseEnd - t.responseStart,
        dom: t.loadEventStart - t.responseEnd,
        total: t.loadEventEnd - t.navigationStart
      };
    }, 0);
  }
}

With access to all of this performance data, you are free to do with it whatever you want. You could analyze it on the client, notifying you when there are problems. You could send 100% of the data to your back-end analytics server for later analysis. Or, you could hook the data into a DIY or commercial RUM solution that does this for you automatically.

Let’s explore all of these options:

DIY

There are many DIY / Open Source solutions out there that gather and analyze data exposed by NavigationTiming.

Here are some DIY ideas for what you can do with NavigationTiming:

  • Gather the performance.timing metrics on your own and alert you if they are over a certain threshold (warning: this could be noisy)
  • Gather the performance.timing metrics on your own and XHR every page-load’s metrics to your backend for analysis
  • Watch for any pages that resulted in one or more redirects via performance.navigation.redirectCount
  • Determine what percent of users go back-and-forth on your site via performance.navigation.type
  • Accurately monitor your app’s bootstrap time that runs in the body’s onload event via (loadEventEnd - loadEventStart)
  • Monitor the performance of your DNS servers
  • Measure DOM event timestamps without adding event listeners

Open-Source

There are some great projects out there that consume NavigationTiming information.

Boomerang, an open-source library developed by Philip Tellis, had a method for tracking performance metrics before NavigationTiming was supported in modern browsers. Today, it incorporates NavigationTiming data if available. It does all of the hard work of gathering various performance metrics, and lets you beacon (send) the data to a server of your choosing. (I am a contributor to the project).

To compliment Boomerang, there are a couple open-source servers that receive Boomerang data, such as Boomcatch and BoomerangExpress. In both cases, you’ll still be left to analyze the data on your own:

BoomerangExpress

To view NavigationTiming data for any site you visit, you can use this kaaes bookmarklet:

kaaes bookmarklet

SiteSpeed.io helps you track your site’s performance metrics and scores (such as PageSpeed and YSlow):

SiteSpeed.io

Finally, if you’re already using Piwik, there’s a plugin that gathers NavigationTiming data from your visitors:

"generation time" = responseEnd - requestStart

Piwik

Commercial Solutions

If you don’t want to build or manage a DIY / Open-Source solution to gather RUM metrics, there are many great commercial services available.

Disclaimer: I work at SOASTA, on mPulse and Boomerang

SOASTA mPulse, which gathers 100% of your visitor’s performance data:

SOASTA mPulse

Google Analytics Site Speed:

Google Analytics Site Speed

New Relic Browser:

New Relic Browser

NeuStar WPM:

NeuStar WPM

SpeedCurve:

SpeedCurve

There may be others as well — please leave a comment if you have experience using another service.

Availability

NavigationTiming is available in most modern browsers. According to caniuse.com 81% of world-wide browser market share supports NavigationTiming, as of May 2015. This includes Internet Explore 9+, Firefox 7+, Chrome 6+, Opera 15+, Android Browser 4+ and Mac Safari 8+.

CanIUse NavigationTiming

Notably absent? iOS Safari. NavigationTiming made a brief appearance in iOS 8.0, but was pulled from 8.1 for “performance reasons“. As of iOS 8.3, NavigationTiming is still missing in iOS Safari. This is highly unfortunate for mobile-focused websites, as it means owners cannot get accurate performance metrics of their visitors.

One thing that can help for mobile is Boomerang, which can capture performance metrics for all browsers, even those not supporting NavigationTiming, for everything beyond the first page view on your site. It does this by tracking timestamps in a cookie from page-to-page. However, there is no accurate way of tracking first-page network timings (beyond guess-timates).

I’m hoping Apple is hard at work bringing NavigationTiming back to Safari. You can check on the latest updates by viewing doesioshavenavigationtiming.com. Hah.

Tips

Some final tips to re-iterate if you want to use NavigationTiming data:

  • Use fetchStart instead of navigationStart, unless you’re interested in redirects, browser tab initialization time, etc.
  • loadEventEnd will be 0 until after the body’s onload event has finished (so you can’t measure it in the load event itself).
  • We don’t have an accurate way to measure the “request time”, as requestEnd is invisible to us (the server sees it).
  • secureConnectionStart isn’t available in Internet Explorer, and will be 0 in other browsers unless on a HTTPS link.
  • If your site is the home-page for a user, you may see some 0 timestamps. Timestamps up through the responseEnd event may be 0 duration because some browsers speculatively pre-fetch home pages (and don’t report the correct timings).
  • Some browsers report 0 for timestamps that should always be filled in, such as domainLookup*, connect* and requestStart. This is a bug, but you may need to detect and work around this if you’re analyzing data on your own.
  • If you’re going to be beaconing data to your back-end for analysis, if possible, send the data in the body’s onload event versus waiting for onbeforeunload. onbeforeunload isn’t 100% reliable, and may not fire in some browsers (such as iOS Safari).
  • Single-Page Apps: You’ll need a different solution for “soft” or “in-page” navigations (SOASTA has developed a Boomerang plugin for SPAs).

NavigationTiming2

Currently just a draft, NavigationTiming2 builds on top of NavigationTiming, adding:

Head to the mailing list or view the spec on Github for more information.

Conclusion

NavigationTiming exposes valuable and accurate performance metrics that were not previously available to your web app. If you’re interested in measuring and monitoring the performance of your web app, NavigationTiming data is the first place you should look.

Next up: Interested in capturing the same network timings for all of the sub-resources on your page, such as images, JavaScript, and CSS? ResourceTiming is what you want.

Other articles in this series:

More resources: