Web Pages that Kinda Work™

We started this journey seriously in 1997 (time-line and site history) having played around with a web presence since 1995. We slavishly tried to make our web pages work exactly the same way on every browser. We had only about 150 pages on the site and there were really only two browsers in those days - Netscape 4.x and MSIE both of which were wildly different. We were able to write cross browser pages that displayed well on both major families by keeping things simple. In 1999 we decided - a risky decision then - to push the envelope - we decided that page navigation was important and that pop-up/pop-down/fly-out menus were the best way to simplify user navigation. Further, as the site grew, CSS would become increasingly important. Those two apparently simple decisions - which we still consider to be right - really got us into the cross browser business and forced us to make some technical decisions - trade-offs - that have consequences for certain users of this site.

In 2007 and again in 2011 we stood back to look at what has happened since those early decisions. Could we get to a single page content that will render acceptably on at least the browsers we care about, a single standards compliant'ish web?

Fall 2015 Mobile Site Update: We now use a minor variant of this technique to generate pages for mobile devices based on detecting one of two strings in the UA. This works for smartphones - feature phones we essentially ignore.
Bad News (2010'ish): The great HTML5 betrayal. We had always planned to progessively move to XHTML (perhaps 10% of our pages were indeed XHTML 1.0 though we never dared to be strict) in line with W3C recommendations, religiously closing unterminated tags (img, br etc.) in anticipation. Always carefully using the W3C validator and other good stuff. And then - boom. A cabal had decided that XHTML was no longer the way forward and that the perfect internet world will be rendered in HTML5. With some collateral damage. Like all our carefully terminated open tags generating warnings and errors. So if this can happen once it can happen again. We changed every doc type to:
```
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"  "http://www.w3.org/TR/html4/loose.dtd">
```
and don't plan to validate any pages ever again. Instead, we simply check the site irregularly with FF, MSIE (8 and 9), Chrome (which should also cover Safari) and Opera. We'll do the absolute minimum and keep a weather eye open on the progress of HTML5 (or whatever else it is suceeded by).

Fall 2013 Update: We've started to use the W3C page validator again (for HTML 4.01 transitional and HTML5) and are slowly removing the major roadblocks to HTML 5 validation. We are in no rush since we are still waiting for the other shoe to fall off and daily await the announcement that SGML 75.3 is the THE way forward.
Good News: CSS compliance is now generally very good accross all major browsers. Doubtless there are quirks (or even quarks) remaining but we are not pushing the boundaries enough for them to bother us. Since around 2013 we only support CSS drop-down/fly-out menus (JS only for the three remaining MSIE 6 users).

We suspect it will be another 5 or so years before we can use a single page format. And during the next 5 years, in this business, anything can, and usually does, happen. XHTML 2.0 anyone?

Our Priorities

We consider our priorities for this site to be:

Site maintenance and publication speed: with over 4,500 pages - still growing - we need to be able to build and change pages and layouts fast. We also want to be able to change site and page design quickly. For us this implies extensive use of CSS. In 2003 we moved to a CSS liquid layout which was surprisingly simple and extremely effective. We no longer worry about screens sizes getting, our pages will always use what you have - instead we now worry about them getting smaller (mobile thingies).
User Experience: We consider the major useablity factor for big sites to be the ease with which visitors can get to the information and less the any glitzy effects. For us, this meant left and right menus and pop-up/pop-down/flyout menus. We built our first JS pop-up menus in 1999 and replaced with in 2003 with CSS versions.

We have been thinking a lot about whether there is a role for so-called Web 2.0 and/or AJAX techniques on the site. Our musings to date.
Visual Appearance: It might not show but we do consider the site look to be important. This encompasses both the aesthetic look and feel of the site and particularly how it is laid out on the increasing range of screens sizes - hence our move to liquid layouts back in 2003. Again CSS is the dominant component of this aspect of site design.

Our Technical Choices

We made the following choices:

We follow the W3C standards for HTML 4.01 and CSS2 and plan only minimal changes to stay renderable in HTML5. We have always supported browsers that support these standards even when there was no commercial justification for doing so. Frankly in the early days (2003) there was not enough Gecko/Opera traffic to justify the effort but that has now radically changed - our current access stats show 30% Firefox, 18% generic Mozilla, 16% chrome/Safari and 15% MSIE. MSIE has come a long way and while it has been part of the optimized class since IE7 we only removed any vestigal handling with the arrival of IE8.
We are decreasing as rapidly as practical our use of javascript for other than local processing. We see it as both a major source of cross-browser compatibility problems - the DOM and AJAX do not really change this - and an ever increasing source of security related problems. We cannot, currently, remove Javascript entirely. We have about 30 or so lines for each page - historically more for MSIE. There is a place for javascript but, in our view, page layout effects are not that place. We would love to see an extension to the pseudo-class selectors in CSS to allow for click triggered events. With :rclick and :lclick or similar selectors we could do all kinds of neat Web 2.0 type things and we would be left with 10 lines or so of benign, but essential, Javscript.

In mid 2007 we started looking at what, so called, Web 2.0 interfaces could do for our style of pages.
Other browsers - those which are neither W3C or MSIE compatible are increasingly becoming a problem since they comprise three distinct groups.
1. Text based browsers. Mostly, but not exclusively, used for accessibility reasons. There are, however, a surprising number of folks with slow connections or who dislike the glitziness of the current web who also use them. This group cannot be ignored.
2. Older, oddball or cloaked browsers. Here we care much less.
3. Mobile devices. The functionality of this group is rapidly increasing but with their generally small screens they represent a major obstacle to right and left hand menu design (not enough space) and tend to have javascript and CSS support ranging from non-existent, through pathetic, to full blooded W3C. 2013 Update: Most smartphones are using pretty stock W3C browsers (Chrome, Gecko, WebKit, MSIE variants) to the technical issue has largely disappeared. Screen sizes are also becoming bigger (tablets, notepad or phablets). Perhaps sloth was the best strategy after all.
We have no desire to replace one set of differences with another to cover this wide range of default users. We are trying to create a single page strategy that will work for all of them equally well (or should that be equally badly) and are still considering a strategy of omitting all right hand menus - because the screen size is too small and many browsers still have problems with CSS right positioning. We also display a message for non-W3C browsers indicating that the browser is getting - in our view - a less than optimal browsing experience - which is probably unbearably annoying and outrageously arrogant. Though if it happens when using a cloaked browser ID string you get what you deserve.

Our Implementation Choices

We made the following choices in 1999 which still largely govern our site organisation:

We do server side browser sniffing and generate appropriate page content. Depending on your browser the CSS and HTML may be different. Currently we have three versions - W3C compliant, MSIE compatible and a default version. With the advent of MSIE 7 we thought we saw a way to handle differences in MSIE6/7 with MS's conditional expressions - only MSIE 7 changed the parsing on these statements so backward compatibilty died on the spot (and left an ugly collateral effect for MSIE users that we took far too long to fix. Since the advent of IE9 (2010) we now use server side sniffing for two page types - Gecko/Opera/MSIE and the rest.
We do not care about consistency between browsers. If you look at this page in both Gecko (Mozilla et al) and MSIE (< IE7) you may see different page banners. It would have taken many hours to get them to look exactly the same due to early CSS quirks. In our view it does not matter. The look is consistent within the browser it does not need to be consistent between browsers. This approach has saved us many, many hours of work for what we consider would have been a trivial return.
We are increasingly removing Javascript for page layout effects whenever we can. We have a series of wireless calculators and a trivial regular expression tester which are written in javascript and think functions like this, as well as field validation, auto-completion, etc., are good uses of javascript - page layout is, IOHO, an abuse of Javascript. CSS should - and does in good implementations - provide all the page layout effects we currently require or even desire.
We worked hard to provide pop-up/pop-down, multi-layer menus for site navigation. But our motive for pop-ups is to simplify the users navigation task - and to give a very quick spatial tour of the site as a side benefit - not provide a unique method to get to a page. If a user's browser cannot or does not support pop-ups they can still get to any page via multiple page clicks. We would argue that our navigation degrades relatively gracefully if CSS or Javascript support is not present. We think that any other navigation method is irresponsible and plain bad design.
We have used right hand menus since 2002. They never contain essential page navigation. Rather they are provided as a convenience, typically containing resource links and perhaps some context based short-cuts. Eliminating them entirely for certain classes of browsers is not catastrophic - site navigation is still possible. We typically send the menu bar to all users but with a CSS property of hidden/invisible for selected browser classes. Thus, if a browser is capable of making sense of them (such as a text browser) it can do so - else it's an overhead - albeit a somewhat painful one for limited bandwidth users such as those with mobile devices.
Our pages are rigidly structured into top banner, central content, left menu, right menu and footer and we make extensive use of Apache's XSSI feature to simplify page structure and especially to enable fast site-wide changes. One of the side-effects of XSSI is that javscript and CSS is contained in the page and does not use <link rel= constructs so speeding page-load as well as minimizing server load. We abandoned graphic roll-over effects in 2001 in favor of CSS versions and have never regretted it. Between Text color, background and borders the CSS effects provide for endless hours of fun.

Implementation Consequences

There are consequences for our implementation choices:

USER_AGENT Strings: If a browser does not generate a sensible USER_AGENT string it can get a default page. There are no official USER_AGENT standards but there is a well established ad-hoc standard. If a browser does not follow the ad-hoc standard or a user decides to experiment with the browser string then they will get a default page. If the purpose of the experimentation is to defeat browser sniffing they will succeed - now what? Cloaked browser strings are a pointless exercise. However, such users will still be able to get to the final page - though they will require more clicks - a perfect reward in our view.
Cached pages: We have had over the years a number of complaints that poorly designed web caches which only use the target URL generate an incorrect page view. For example, if a MSIE user's access caused a remote web cache to be filled, then a subsequent Gecko user's request, which was satisfied from the same web cache, would deliver an MSIE formatted page. Generally this way round is harmless - such is the commercial power of MSIE - but the other way round can be quite disastrous.
Disabling CSS: Users can disable CSS in most browsers. We have no idea why a user would do that since there are no security issues involved - unlike javascript which we understand all too well, hence our decision to reduce/eliminate the use of javascript.

Outstanding Issues

We have a long way to go - this is part of our very long to do list:

Accessibility: We think we should do an overhaul of CSS definitions to improve Accessibility. We have too many px and font-size directives in the CSS and think we need to rationalise the situation. Update We did a brief check at the end of 2005 on accessibility and were very disappointed with the very small amount of good advice out there on this important topic. Tons of motherhood - but a little short on practical advice. As always there are exceptions such as webaim.org and the W3C WAI.Update 2007 We finally moved to define all central pane font sizes as em units to allow improved user control over rendering. We have no more fixed font sizes in this pane. Progressively we'll look at converting to em units in the right and left menu panes but here we are pretty space constrained.
Pop-Outs: We will be adding more depth and more selection in our pop-outs to provide faster access to data - probably going to four or even five-level pop-outs. We will only do this for CSS based menus.
Small Screens: We think we should do something about providing access to the increasing number of smaller screens - mostly mobile/PDA/Telephone thingies. Most Liquid designs handle a range of big screens well but are hopeless for smaller screens - ours is no exception. Paradoxically we suspect that the solution to this problem is do more work on our default page format, that is, provide no features. But it also has serious implications - we think - for right/left menu usage - the space is not there for dual menus. Good site search features may be the best way forward for this class of user.
Languages: Next year (2006) we will be starting to provide some of our pages in multiple languages. We want this to be transparent to all users. We will use Apache's multiview features to provide this based on the browsers language priorities.

Web 2.0 - Theory and Practice

So just what is Web 2.0? If the term Web 2.0 is meant to describe a rich user experience then we would argue - through the use of pop-up/down menus and deep linking - we have been using Web 2.0 techniques since 1999. If you limit it to AJAX (javascript) techniques then we have been, and will continue, going in the opposite direction!

Some thoughts about our site and its pages:

We provide a lot of text based pages, we use a minimal number of images, few and very simple forms and currently no audio or multi-media content - though this may change over the next year or so. We are perhaps not ideal AJAX users in this respect.
We use a lot of links on our pages. Both internal and external. We have been thinking about using AJAX to suck in site based linked page content into an overlay div - rather than require a full page load. We currently have a simple visual indicator that a link will open a new page (grey background) or not. The new scheme would require another visual clue. Alternatively we could add the linked pages to the page content and use a trivial js function make it visible.
It's always tough when writing explanatory material of any kind to decide what subsidiary material should be added to emphasize points or provide background explantions. We tend to always add supporting material because we feel that the moment the reader is lost the rest of the material is, frankly, useless. Even the positioning of such material can be a difficult decision to avoid breaking reader concentration with irrelevancies - especially for experienced readers who have familiarity with the subject matter. We have been experimenting with hidden notes and using javascript and/or css to pop-up the additional material. This also involves a different style of writing - so the challenge is as much literary as technical.
We have never been fans of glitz. While we can look at the iMac and its bubble graphic menu effects and be awe struck - our second thought is - what else could we do with that CPU power? The only time such feature are useful is to allow a lot of, essentially unreadable material, to be packed into a very small space, but trivial CSS techniques can be used to expand them on :hover events.
We have never used photographic or movie based material - even in places where it would be very useful. The reasons are historic. Such techniques were either not available and/or viable when we started. That has now changed. We need to start experimenting with this kind of material.

Some thoughts on AJAX, Javascript libraries and related issues:

Size: The size of most javascript libraries offends us. The smallest we could find is 70K and the biggest weighs in at over 200K. Please. These libaries should contain tools to analyse page use and generate minimal js files or as a minimum, section the libray for content such as base, form validation, graphics, effects etc..
security: The thought of the security implications of random XMLHttpRequest requests hitting our server makes us wince a little. We need to think this one through a tad before implementing a live version.

Our Conclusions

We're getting periously close to having a single page strategy for all users - bar mobile ones. A scary thought but we may be back to where we were in 1997.
From our web statistics we judge that we provide a good browsing experience to about 97% of our human visitors. It's not as good as we want but perhaps that's as good as it gets.

We encourage comments on our approach, differing implementation experiences or alternate strategies and trade-offs. We're not interested in rants, diatribes or stream-of-consciousness stuff. But welcome comment from grunts, like us, who live and work in a practical world, solving practical problems with finite resources and genuinely trying to provide the best experience they can for site visitors.

Problems, comments, suggestions, corrections (including broken links) or something to add? Please take the time from a busy life to 'mail us' (at top of screen), the webmaster (below) or info-support at zytrax. You will have a warm inner glow for the rest of the day.