The Results
It should not come as a surprise to learn that the overall accessibility of government agency websites is abysmal.
This chapter attempts to sum up the findings. I've chosen to use the Web Content Accessibility Guidelines version 1.0 as the template, and therefore divided it into 14 sections.
Guideline 1: Provide equivalent alternatives to auditory and visual content.
The use of alternative content to graphical elements are, in one word, horrid. Among the 92 sites tested, none - that is 0 per cent - has managed to give proper, alternate, content to all graphical elements.
Image maps - client and server side - are in little use overall, with client-side maps taking up 46%. Only 16% are server-side maps. Among either type there are few redundant links provided, and very little use of the ALT-attribute.
Multimedia presentations are even more rare than image maps, and only a few were found. 96% of the sites had none of them. The remaining 4% could, often, not be tested due to proprietary file formats.
Summary of results for Guideline 1
Guideline 2: Don't rely on color alone.
Very few of the sites - only 19% - used colours as information carriers. Of these, only 7% (6 sites), had neglected to make sure that the same content was available with colour negated due to such things as colour blindness, grayscale monitors and printing.
Contrast between colours is, however, slightly more of a problem. In 25% of the sites, the contrast could be better, 75% has been judged as having sufficient contrast between background and text. Images has not been tested. [1]
Summary of results for Guideline 2
Guideline 3: Use markup and stylesheets and do so properly
Compliance with this guideline is, frankly, a mess. Spacer images are all over the place, and there is hardly a valid set of markup code anywhere: a full 100% of the sites have one or more invalid documents.
The use of absolute units - such as pt in setting font sizes - is abundant. All of the sites tested use pixel-based values in conjuction with table-based layout to, quite often, fix "page" widths.
Structure - i.e. the use of elements such as H1, P, UL, etc. to logically structure documents - is largely missing. 97% have either not used header tags at all or are using them for layout purposes, 74% have either not marked up lists or are using list elements for indentation and other purposes, and 53% have used quotation elements as a layout tool.
All in all, structure is heavily neglected.
Summary of results for Guideline 3
Guideline 4: Clarify natural language usage
Most of the documents in the test are, naturally, in Swedish. This is however not, in 98% of the cases, obvious: only 2% of the sites have specified which natural language the main content is written in. A speech browser might attempt to read English from most of these documents.
The problem becomes - if possible - worse when looking at documents with a mixed Swedish/other language content. Only 1% of the sites have any markup indicating a change of language. Most all - 93% - have some sections in English, with French, Spanish, German, Finnish, Arabic and Persian following. 10% of the sites have information, of various quality and scope, in more than 8 languages.
In general, the translation of content to other languages is poor. Many of these sites are aimed specifically at immigrants and one might expect them to have the main bulk of their content translated to a number of languages. 86% of the websites have content translated, but very little of this is properly marked up. None of the 92 sites has translated content equal to the Swedish version.
Indication of language is often done through a flag-symbol, a good if heavily critized technique. These flags are rarely given ALT-texts or language information such as through HREFLANG. Other sites use images to mimic languages such as Arabic or Chinese, again without ALT or HREFLANG.
Finally, one of the most useful structures - spoken as an immigrant to Sweden myself - is that of marking up abbreviations and acronyms. This has not been done on a single site. Neglecting this means that people with imperfect knowledge of Swedish culture and abbreviation idiosyncrasies lose valuable information.
Summary of results for Guideline 4
Guideline 5: Create tables that transform gracefully
In general it is difficult to say much about data tables in use on these sites. Only 48% has any data table at all. Among those, only 9% has correctly identified row- and column headers.
Multi-level tables, e.g. data table with more than one logical level, are even rarer: only 24% of the sites has any such, and of those none has logically structured them.
All in all, the data tables have very little structure imposed on them. None have abbreviations specified for header cells, only 1% has a summary.
Most of the tables are - as was expected - used for layout. Most, 99%, linearize well or fairly well. In one case only were the results so confusing that the documents were incomprehensible. Finally, 7% of the layout tables also used various structural markup for layout purposes, such as for instance TH for bold font.
Summary of results for Guideline 5
Guideline 6: Ensure that pages featuring new technologies transform gracefully.
The question of "new" technologies is a difficult one. Whilst, as the Web Accessibility Initiative says, developers are encouraged to use new technologies, it is important that they do not forget to ensure that the methods transform gracefully.
A majority of Swedish government agencies tested seems to have forgotten the principle of graceful transformation. 70% of the sites use one or more scripts - typically Javascript - without any alternate way to get at information. 12% has accessibility problems if stylesheets are not supported. [2]
Then there are the frames. 66% of the sites use them, very few [3] have acceptable NOFRAMES sections - many are empty or call for users to upgrade/change their browser.
Of the eventhandlers used, 85% are device dependent.
Summary of results for Guideline 6
Guideline 7: Ensure user control of time-sensitive content changes
One would expect government agencies to be careful and restrictive when it comes to animations and movement in web documents. However, 22% of the sites tested have one or more scripts/applets/images that animate or move content.
Reloading and redirecting is, very often, done through markup - that is the META element. 30% of the sites use this to reload the same document, 42% to redirect to somewhere else.
Summary of results for Guideline 7
Guideline 8: Ensure direct accessibility of embedded user interfaces
A majority of these sites have no interfaces that are independent of the user agent. However, of the 11% that do, none have made sure that these are accessible.
Summary of results for Guideline 8
Guideline 9: Design for device-independence
The assumption among these 92 sites is clear: access to a website is done by using a mouse; pointing and clicking. Whilst the majority of the image maps used - 46% - are client-side and not server-side, a mere 3% have bothered with tab indexing, and only 1% use access keys.
Of scripts, 85% are triggered by media or device dependent events such
as onMouseOver. Of the 16% server-side image maps, almost
none have redundant links.
Summary of results for Guideline 9
Guideline 10: Use interim solutions
Interim solutions are also a difficult topic: when should one stop using them ? [4]
For these 92 sites the answer seems to be: right now. 92% open links in new windows, for the most part without any indication of doing so. 92% also lack explicit labelling of form controls, 42% present data in columns without alternatives, 93% have form controls without place-keeping values, and 91% keep links adjacent to each other without any non-link, printable, characters separating them.
Summary of results for Guideline 10
Guideline 11: Use W3C technologies and guidelines
For sake of accessibility, or rather: for the sake of the "review process", the "built-in accessibility features", and the "industry consensus", authors should use W3C technologies. This is, mostly, not a bad idea.
However, it does reflect upon the rating of these websites. 92% use one, or more, technologies that are not created by the Consortium. For the most part this means GIF or JPEG images, as well as Flash movies.
99% uses deprecated technologies - typically the FONT tag, with a sprinkling of Microsoft- and Netscape-specific tags.
None of the sites tested make content available in multiple languages and types through standarized mechnisms. Finally, no alternative, accessible versions of these sites could be found.
Summary of results for Guideline 11
Guideline 12: Provide context and orientation information
This is yet another weak point. Of the 61 sites with one, or more, framed documents, 58 have no title set. Most have values for the NAME attribute, but these are rarely informative. No LONGDESC or description link could be found for any frameset.
Finally, form controls are - in 92% of the cases - not explicitly associated with labels.
However, the main sticking point is still structure: 92% of the government agency sites has not attempted to divide information into smaller, more logical, chunks.
Summary of results for Guideline 12
Guideline 13: Provide clear navigation mechanisms
Navigation is a complex issue. As the separate, user-interview based test showed, a full 25% of the sites were perceived to have no specific navigational aid at all: no sitemap, no A to Z index, no "most visited" collection of links.
82% of the sites err in making their link texts clear; there is a huge
number of click here
and equivalent links. Various forms of
meta-data exist, both in form of the META tag and
RDF, but only on 5%
of the sites.
51% have no sitemap, 28% change the methods of navigation one or more times
throughout the site, and 91% have no navigational bars
,
i.e. an indication of where on the site and
in which part of the navigation the user is. Finally, none at all
have grouped navigational links.
Search-functionality exist on 79% of the sites, but only 25% offer a variety of search methods depending on the skill level of the user. None have created document collections through, for instance, the HTML LINK mechanism.
Judging whether important information has been
front-loaded
is, in government writings, difficult. However, it would
seem that for the most part the authoring skills of the humans involved
are rather up to it. Only 11% of the sites were deemed non-compliant.
Luckily, 0% of the sites had any ASCII art to skip over.
Summary of results for Guideline 13
Guideline 14: Ensure that documents are clear and simple
Judging whether language is clear and simple is another challenge [5]. Of the 92 sites, adjusted for governmentaleese, 57% would seem to use language that is overly complex and difficult to read.
The sites all use a mix of text and images, but it is more often than not a case of decoration rather than content enhancement: 65% of the sites use graphical elements purely for show, and could very well look into more images as a help in understanding and comprehension.
Last, whilst most of the sites - 63% - maintain a consistent graphical presentation, 37% does not.
Summary of results for Guideline 14
Additional information
Difficult navigation
A fourth - 25 per cent - of the websites have no particular support for navigation at all. No sitemap, no A to Z index, no "most visited" collection of links. Of the 92 sites reviewed, none have utilized means of creating document collections such as HTML's LINK element.
All in all, navigating these websites is quite often a matter of looking through each section until one find what one is looking for. Several - around 18 per cent - has no search functionality.
The consequence of this is perhaps most apparent when a visitor is unaccustomed to using the Web for information gathering, or when the user has learning and/or memory difficulties. Link collections (sitemap, index, most-frequently-visited) and search fascilities would be helpful.
Links
One of the major difficulties is the way links are handled - both in terms of where they are opened, and how they are labeled. In the first case, 85 per cent of the sites use links that open unannounced in new windows. Only 17 per cent have labeled their links so that the texts make sense out of context.
No specific examination was made of whether underlines has been removed from links, but a quick search through the 37,000+ files mirrored reveals a staggering 17,588 cases of underline removal from links through the use of stylesheets.
Site-specific configuration
Only a few sites have chosen to use site-specific, often Javascript and cookie dependent, methods for configuration. 95% have opted for a solution where the user can change font sizes, colours, etc., through their user-agent configuration.
It is well worth noting, however, that quite a few sites - around 100 per cent - use absolute units for various layout purposes. In approximately 92% font sizes have been set in pixels. This leads to some users being unable, or hard put, to change the setting due to quirks in their browsers.
The final 5% all have different solutions for changing the appearance of the site. This was tested in particular, since such a method is confusing to users, and all to often dependent on specific technologies.
1
The siteSifter tool uses an algorithm for the testing of
colour contrast as outlined in a
paper by Ridpath, Treviranus and Weiss.
The various results have not been kept, but our highly non-systematic
results mirror those in the paper.
2
Typically when using an image to mimic text, making it transparent, and
setting a colour of the text
that depends on the background colour
set via CSS. When the said
background colour goes away, the text
becomes unreadable.
3
In this context, very few
means that approximately 10% of the sites
using frames have anything resembling acceptable noframes-content. The
exact number is rolled in with other statistics of checkpoint 6.5.
4
Personal view: if an interim solution
has benefits asides from being
interim and does no harm, keep using them.
5
siteSifter uses the Swedish
LIX algorithm for
determining how readable
a text is. This algorithm has shown itself
to work well in combination with a result table adjusted for content and
target group both in Swedish and other languages.
The original LIX result table is:
| Classification | Index | Example text |
|---|---|---|
| Very easy | 20-25 | Children's books |
| Easy | 31-35 | Popular magazines |
| Somewhat difficult | 40-45 | Newspapers |
| Difficult | 50-55 | Official letters and announcements |
| Very difficult | 60 | Scientific texts, articles of law, governmentaleese |
The result table has been adjusted upwards with 10% to cover for the
very often quite official language in use on government websites. Documents
with a value of 50 or more has been looked at in detail. For more
information regarding the algorithm, the paper
Kvantitativa läsbarhetsanalyser som metod för
datorstödd granskning by Magnus Cedergren can be
strongly recommended.