I’m developing a performance measurement tool and took this code to record time to DOMReady. Testing on a real site I’ve discovered that many records come with zero time to DOMReady. Filtering the records by browser I’ve got that only IE’s records were zero.
This zero time means that the event fired right after it was defined. This means that in practice on IE we could run the code bound to DOMReady before actually DOM Tree is ready.
There are similar issues described here.

After investigation I could reproduce it and found out that one of third-party advertisements caused the bug. It was inside IFRAME and it had alot of Javascript that was doing document.write() plugging next Javascript etc.

How to fix it?
There could be different solutions:

• if possible, move the code to the bottom of the page instead of using DOMReady event
• move the code into window.onload (execution will be delayed)
• use another implementation of DOMReady for IE, there is solution that utilizes “doScroll()” method in order to check whether the content is ready yet

Note that it might be not so easy to find this bug. Your Javascript application that starts in DOMReady will work perfectly with your browser on your development server, but on production it may fail because of advertisements enabled and in worse scenario you’ll get geo-targeted advertisements and the bug will appear only for several countries.

How to catch it?
Measure the time in Javascript and send alert if the code runs right after event binding:

var page_start = new Date().getTime();
function bootstrap()
{
  if (0 == (new Date().getTime() - page_start))
    new Image().src="/developer_alert.php"; // send alert
...
...
}
bind_domready(bootstrap);


Basically you provide a page URL, then the service loads all styles of the page (both internal and external), finds all background images and encodes them into base64 (for data:URI format). As output, it provides a tar.gz file with HTML file and CSS files with encoded images in textual format.

The HTML contains the original pages with built-in code that checks browser version and plugs the styles with encoded images. There are several version of the styles for different browsers:

• MHTML – for IE version < 8 (only IE8 supports data:URI )
• data:URI – for all the other browsers
• original styles – for the case when Javascript is disabled and for IE7 under Vista (there is a bug related to security issue with cached MHTML background images)

The advantages are:

• all background images will be loaded in one HTTP request
• it’s possible to quickly rebuild CSS sprites after making changes

This is a new tool that’s in beta stage so I would not use it in production.
The tool written in Java and the authors are going to opensource it as soon as release-candidate is ready.
(via Habr article in Russian)

I remind that besides Data:URI Sprites we have classic Image Sprites that involve joining images into another big image (opposite to base64 supposed by Data:URI). There are number of tools exist for Image Sprites and most advanced that I know is SmartSprites.

The advantage of Image Sprites over Data:URI Sprites is that it’s supported by almost all browsers including very old versions. The disadvantage is that Image Sprites composition cannot be fully automated and every time should be controlled by web developer.

I’ve prepared a sample page: http://webscalingblog.com/waterfall/1/, let’s look how we can optimize it.

Waterfall diagram of loading this page using IE7 (clickable):

We see that page contains 3 CSS components, 3 Javascript and 7 images.
There are two orange spot on the diagram – the browser opened two connections and was ready to download two components in parallel.
Green part of bars – time to first byte – it’s the HTTP request overhead. The overhead here is 150-200ms on each HTTP request. We see much of green color on the diagram.

Look at 2.css, 2.js and 3.js – the browser spent most of time sending request and waiting rather than downloading.

Especially important is Time to Start Render which is shown as green vertical line. We see that it took 2 seconds to start render this pretty simple page.
2 seconds the browser downloaded HTML, CSS and JS components.

As first step I’ve joined the CSS files and the JS files into 1 CSS file and 1 JS file:
http://webscalingblog.com/waterfall/2/

We see it gets better but browser still spending time on the Javascript, rendering cannot be started until browser downloaded and executed Javascript. And Javascript is blocking downloading of next components.

Next example shows what we get after moving Javascript to the bottom of page (let’s suppose that in this case we can do it).

http://webscalingblog.com/waterfall/3/

It becomes pretty much better, Time to Start Rendering was improved. Javascript is loading after the all other components loaded.
But still, the browser has to download CSS, there is green bar (wait time) after HTML downloading is done and the CSS component consumes one downloading slot.

Now let’s try to embed CSS into HTML.

http://webscalingblog.com/waterfall/4/

Finally, Time to Start Render was reduced dramatically. Now it’s 0.8s instead of 2s we had at the beginning.

Note that after embeding CSS we increased download volume for cached page views, because browser now has to download CSS within HTML. To avoid this, we can use the technique with cookie to guess cache state. It was described in the previous post.

Going from waterfall3 to waterfall4 we didn’t improve Full Load Time. So if we would use some benchmark to measure Full Load Time we probably would not notice any difference. While from end-user point of view there was significant improvement in response time.

At the moment I don’t know any benchmark (besides AOL Pagetest) that’s able to measure Time to Start Render. There is Firebug 1.3 Beta for Firefox which is able to do waterfall diagrams and which shows time to DOMContentReady and most time we can use it, but actually it isn’t the same, I noticed that moving Javascript to the bottom doesn’t improve DOMContentReady time while actually browser starts rendering earlier (I tested it with a big Javascript file)

And last step, if we want to improve Full Load Time and if we can use CSS Image Sprites here, we can join images into one bigger image.

http://webscalingblog.com/waterfall/5/
(Sorry, I’ve simply included the file and didn’t use CSS positioning)

It reduced Full Load Time from 2.4s to 2s. And we had only 7 images – usually sites contain much more and improvement will be more significant.
CSS Sprites were described in one of previous posts.

2. If we have several Stylesheets we join them into one bigger CSS file. As described in the previous post about Image Sprites, we want to reduce amount of downloaded components because of HTTP request overhead. And if in case of images there is just a delay before loading next image, in case of Stylesheet it’s a worse case: visitors see blank page until all Stylesheets loaded. One single file loads faster because there will be minimal overhead and usually it compresses better too.

3. We can go even further: if user comes with empty cache we can avoid additional request by including Stylesheet required for the page into HTML. So there will be no requests for CSS at all during loading of the page.
We can preload Stylesheet after the page loaded so for primed cache page views user already has the Stylesheet cached. This way we reduce download size and avoid HTTP requests.
More detailed:
Set a cookie when visitor comes for first time. Check the cookie on every request to determine whether the visitor comes first time or not. This way we can guess cache state of the visitor. Don’t inline Stylesheet component if visitor comes with primed cache, use an external definition instead.
Postload the external Stylesheet component when visitor comes first time (without the cookie). See example below.
Cookie named “jsl” will be set when user sees this page for first time and next requests will use external Stylesheet component.
You can set such cookie in PHP:

$is_first = !isset($_COOKIE['jsl']);
if ($is_first)
setcookie('jsl',1,time()+3600*24,'/','.site.com',false,true);

We’ve supposed that browser cache data will be removed in 24 hours and cookie expiration time set to 24 hours.


<?php if ($is_first): ?>
<style>
<?php readfile('style.css') ?>
</style>
<script>
function preload_components()
{
var ste = document.createElement('link');
ste.rel = 'stylesheet';
ste.type = 'text/css';
ste.href = 'http://www.site.com/style.css';
document.getElementsByTagName('head')[0].appendChild(ste);
}
</script>
<?php else: ?>
<link rel="stylesheet" type="text/css" href="style.css">
<?php endif ?>


The block above will output inline Stylesheet in case user comes for first time otherwise it will output link to the static Stylesheet components. You should place this code inside document head.
It will work correctly even in case user comes with empty cache (for example if our cache got forced out because cache size limit).

Style.css – global CSS file. Probably in real application instead of readfile(’style.css’) you will include only part of the Stylesheet that’s needed to render current page.
To find out CSS that’s really required for current page, you can use a Firefox extension: Dust-Me Selectors (note that this tool may not count CSS rules for elements you assign dynamically in Javascript).

Replace existing opening tag
<body …>
with
<body … <?php if ($is_first) echo 'onload="preload_components()"'?>>
We’ve added document onload event handler that will create link DOM element if user comes with empty cache so the components will be already in cache for further page views. The external component will be preloaded only after whole page loaded and rendered so it won’t hurt performance.

The first header Cache-Control: must-revalidate means that browser must send validation request every time even if there is already cache entry exists for this object.
Browser receives the content and stores it in the cache along with the last modified value.
Next time browser will send additional header:
If-Modified-Since: 15 Sep 2008 17:43:00 GMT

This header means that browser has cache entry that was last changed 17:43.
Then server will compare the time with last modified time of actual content and if it was changed server will send the whole updated object along with new Last-Modified value.

If there were no changes since the previous request then there will be short empty-body answer:
HTTP/1.x 304 Not Modified

And browser will use the cached content.

What if server doesn’t send Cache-Control: must-revalidate?
Then modern browsers look at profile setting or decide on their own whether to send conditional request. So we better to send Cache-Control to make sure that browser sends conditional request.

Sample PHP code:

$last_modified_ts = floor(mktime()/30)*30;
if (
isset($_SERVER['HTTP_IF_MODIFIED_SINCE']) &&
strtotime($_SERVER['HTTP_IF_MODIFIED_SINCE']) >= $last_modified_ts
)
{
header('HTTP/1.1 304 Not Modified');
exit;
}
header('Cache-Control: must-revalidate');
header('Last-Modified: '.gmdate('d M Y H:i:s',$last_modified_ts).' GMT');
echo date('d M Y H:i:s');


This example outputs cached datetime that’s expiring every 30 seconds.

Last-Modified suits good in case we can easily calculate modification time of page content.

We must be careful with Last-Modified if we have page content that consists many fragments. We should calculate Last-Modified value of every fragment and get the latest one.
Note that if we have authentication and there is a page fragment that depends on authentication we have to reset Last-Modified value after login/logout – for every page that contains the fragment.
Also note that in case of several web servers we should make sure that Last-Modified value changes synchronous for all the servers.

ETag
This method suits for cases when it’s difficult to maintain Last-Modified value: when you have complicated application with many page fragments especially if there are third-party libraries. Or for the case with authentication, when page content depends on authentication info.
There is simple idea besides ETag:
ETag value depends on the content and must be different for different content and the same for the same content.

Sample usage of ETag header:
$content = floor(mktime()/30)*30;
$etag = md5($content);
if (isset($_SERVER['HTTP_IF_NONE_MATCH']) && $_SERVER['HTTP_IF_NONE_MATCH'] == $etag)
{
header('HTTP/1.1 304 Not Modified');
exit;
}
header('Cache-Control: must-revalidate');
header('ETag: '.$etag);
echo $content;
echo '
Request time: '.date('d M Y H:i:s');


In this example content changes every 30 seconds and browsers will download only if the content was changed.

Static Content and Unnecessary ETag Header
For static content it’s recommended to send Cache-Control: max-age=… header with higher max-age value. In this case browser won’t send any request on normal page views.
So for static content there is no use of ETag header.
The worse case is in case of web servers cluster as ETag value differs for the file on different servers.
For Lighttpd server you can disable Etag using
static-file.etags = 'disable'
in lighttpd.conf

Disabling ETag in Apache:
Header unset ETag
FileETag None

Note that we still want Last-Modified header for static files. If user presses Refresh button, then browser will send conditional request and server will respond “304 Not Modified”.
And if you disable both Last-Modified and ETag browser will have to download the whole content again when user presses Refresh.

Lighttpd and Apache will send Last-Modified if you have configured mod_expires.

Let’s review the process step by step:
Step1.
Replace image entries <img src="/img/anyimage.gif"> with <div class="i_anyimage"></div>.

Add new css entry:
.i_anyimage
{
width: 17px;
height: 17px;
background-repeat: no-repeat;
background-image: url('/img/anyimage.gif');
}
At the end you should get your pages fully functional and identical to the originals.

Step2.
Now you should join all the images into one bigger file. You can use an image editor or create automated script, etc.

Once the image is ready you start to change the css entries so they use the sprite image instead of original image:
.i_anyimage
{
width: 17px;
height: 17px;
background-repeat: no-repeat;
background-image: url('/img/sprite.png');
background-position: -XXpx -YYpx;
}

XX, YY are coordinates of the original image within the sprite.

What can be useful to know here:

CSS Sprite Problems

For Lighttpd there is mod_expire module. Enable it in server.modules section:
server.modules = (
...
"mod_expire",
...
)
Then add following directives for directories with static files:
$HTTP["url"] =~ "^/images/" {
expire.url = ( "" => "access 30 days" )
}


Max-age for Nginx server can be enabled using ngx_http_headers_module:
expires max;

Now web server sends the caching header for static files:
Cache-Control: max-age=2592000

In case of design change we should prevent using outdated content that browsers have in their caches. This can be done by adding file versions to filenames:
script.js -> script1.js -> script2.js -> ... etc

Cache-control: max-age can be useful also when we output HTML. Imagine pages generated by PHP that changed not so often, once per day or even longer. But browsers still have to download HTML every page view.
We can improve it by sending max-age value in PHP.
header('Cache-Control: max-age=28800');

This way we set desirable cache lifetime to 8 hours. Now if someone is clicking a link for second time within 8 hours period he gets the page instantly.

Max-age also helps to make proxy servers more efficient. We can easily organize transparent server-side caching by adding proxy server to web frontend.

Note that there is not easy case if pages have content that changes often and that’s relevant.
For example, there can be difficulties in caching pages with login form that transforms into some box with «Hello username» after user login or if there are user comments, the user who posted commentary will not see it. Because we cannot ask browser to destroy cache entry, it will still get the old page from cache.
The solution can be using Javascript to generate login box (requires enabled Javascript). If we set a cookie after user logged in, we can check it on client-side and generate suitable content for the logged in user. This way the content will be the same from server side view and can be cached.

For Apache2:
compression done by mod_deflate
AddOutputFilterByType DEFLATE text/html text/plain text/css text/javascript application/x-javascript text/xml


For Lighttpd:
enable mod_compress in lighttpd.conf and add configuration directives:
compress.cache-dir = "/tmp/"
compress.filetype = ("text/plain", "text/html", "application/x-javascript", "text/css", "text/javascript", "text/xml")


Nginx configuration directives:
gzip on;
gzip_min_length 1000;
gzip_types text/html text/plain text/css text/javascript application/x-javascript text/xml;
(See ngx_gzip_module documentation for details)

Besides compression there is minification (removing comments and unnecessary whitespaces) that can further reduce size of Javascript.
Minified Prototype.js becomes 92KB size uncompressed and 23KB compressed, this is 20% less than compressed non-minified script.
There are number of tools for minification exists, I’m using JSMin (http://crockford.com/javascript/jsmin).