Taxonomy consultants:
what they do and where to find them

June, 2001

This article is based on interviews with Society members and the consultants they have worked with. It describes what taxonomy consultants do, what their deliverables look like, how to match their skills with your requirements, and where to find them.

Be sure to read the 2006 update at the end of this article.

What do taxonomy consultants do?
Taxonomy contractors can help you:

• add a Yahoo-like navigation scheme to your Web site;

• enhance your Web site search function to give users more relevant results;

• personalize information access so that users get only items of interest to them;

• index a book or classify a collection of documents by subject or other attributes;

• customize an auto-classification software tool.

Since taxonomy plays a supporting role in content management, we didn't find many "pure" taxonomy consulting firms. Instead, we found taxonomy specialists in larger firms as well as solo practitioners.

Deliverables
Typically, taxonomy contractors provide one or more of the following kinds of deliverables:

1. High level content management system designs and project plans. Typical time frame: 4 - 14 months. Cost: $150,000+ Example. Subtasks may include:

• description of user needs, obtained through group meetings and individual interviews;

• knowledge maps that identify and describe content characteristics, including format, ownership, security levels, etc;

• database specifications for maintaining controlled vocabularies (N.B., the Montague Institute software distributed in our courses is a working model that serves this purpose);

• "story boards" or web site mock-ups for illustrating Web site navigation paths and user options;

• recommended content management strategies.

Usually the deliverable is a printed report accompanied by a presentation and discussion. 

2. Taxonomy structures (e.g. outline of subject categories, controlled vocabulary, thesaurus). Typical time frame: 1 week - 3 months. Cost: $5,000+  Examples. 

The structure can be delivered as a printed report containing the terms in hierarchical structure, a tab- or comma-delimited text file suitable for importing into a content management system, or a proprietary thesaurus format, such as MultiTes (a popular thesaurus management program). 

3. Tagged or indexed content. Taxonomy structures (i.e. vocabularies and thesauri) are most useful when the terms are linked to the items you're trying to organize -- books, articles, reports, images, statues, or even document segments. Linking (or "tagging") can be done in a variety of ways, but all methods require input from humans, such as:

• selecting a sample collection of documents (the "training set") to allow a computer program to accurately categorize your content;

• developing rules that a program uses to assign terms to documents;

• adding meta tag labels and values to documents or entering field values into a database record that is linked to a document.

Since the linking process is dependent on the software tool, you'll need a contractor familiar with the system.

4. Business applications (e.g. functioning system for e-commerce, competitive intelligence, drug development, etc.). 

The deliverable is a fully functioning application, complete with installation, training, and technical support. Cost: $25,000+. 

Before you start looking
Before you look for a taxonomy contractor, make an assessment of what resources you already have available, for example:

1. Information professionals -- librarians, indexers, editors, webmasters, help desk staff, and even authors that know your company, its customers, and its products. Librarians know about taxonomy building blocks like controlled vocabularies and cross references. Even more important, they know what kinds of questions people ask when seeking information, and they may have already developed a taxonomy to organize books, newspaper clippings, and internal reports. Indexers, editors, and authors are familiar with the subject matter. They can contribute abstracts, glossaries, and index terms. Webmasters have a different take on user behavior that comes from web server logs.

Bottom line -- If you don't have information professionals available, you'll need a contractor to help with user needs assessment, training, planning, and technology assessment. If your content professionals and IT professionals have trouble communicating, you'll also want a contractor to bridge the gap.

2. Vocabularies, preferably in electronic format. Often you can leverage lists of terms and categories developed for a print publication, corporate library, or commercial information service. 

Bottom line -- If you don't have existing vocabularies, your contractor will have to spend time gathering and organizing terms from employees or customizing auto-classification programs.

3. Taxonomy budget. For commercial publishers, investments in taxonomies can easily be justified on the basis of cost savings or competitive advantage. For many corporations, though, the connection between taxonomies and the bottom line isn't so clear. Preparing a budget, getting sustained funding, reaching consensus on terminologies, developing standards, and training content owners is harder.

Bottom line -- If you need help developing and justifying a budget for taxonomy-related expenditures, you'll need a contractor with experience in preparing high level funding proposals for senior management. You'll also need a facilitator who can help your staff reach consensus on the size and scope of your taxonomy efforts.

Three things to look for
Almost everyone we interviewed agreed that it's important for taxonomy consultants to have:

• subject matter expertise (or, at minimum, the ability to elicit information from subject matter experts);

• technical knowledge -- how the taxonomy will work when integrated with a search engine, portal, or content management system;

• knowledge of user behavior -- how different kinds of users behave when seeking and using information.

Types of taxonomy contractors
Once you know what resources you already have, you can select a contractor to fill the gaps. We have identified five types, ranging from freelance specialists to full service IT and design firms. They are listed here in order of cost, with the least expensive first.

1. Freelance indexers, librarians, and thesaurus specialists. These people usually work solo, although they sometimes call on colleagues for help when the need arises. Typically, they create vocabularies and thesauri as well as evaluate the output of vendor pilot projects. Formats delivered include MS Word documents, Excel spreadsheets or Access databases, printed lists, or proprietary formats (e.g. MultiTes). Deliverables: controlled vocabularies, outlines of subject categories, thesauri.

» Although the independents all understand the basic principles of organizing information, they come from different backgrounds -- book indexing, database indexing, library book classification, or thesaurus creation. Look for contractors with Web indexing experience and a knowledge of how taxonomy structures interact with search engines.

2. Indexers plus software. We found three companies that combine human indexing skills with their own taxonomy software tool. These firms will perform human indexing services as well as license the software to enable you to maintain the taxonomy. Deliverables: controlled vocabularies, thesauri, computer-assisted taxonomy maintenance.

» The advantage is "one-stop-shopping." The craftsman comes with tools, installed on your server or hosted on his server. He or she also recruits and manages a stable of freelancers.

3. Vendors. These companies offer optional consulting services when you buy their content management software, license their content, or use their information services.

 In most cases, taxonomy consulting services are available only to customers.

» This option works well if you have already made a commitment to a software or information vendor. The staff's intimate knowledge of their content and/or technology can reduce development time and resolve compatibility issues. This strategy obviously doesn't work well if you're still in the planning or technology selection stage, although you can learn a lot through a pilot. Even if you're comfortable with your technology selection and get along well with the vendor's professional services staff, you may still want outside help with training in the intellectual and editorial aspects of content management.

4. Web site designers. Individuals and companies that will create a hierarchical list of categories as part of a Web site design or revamp. These contractors typically offer plans and specifications, not fully functioning systems. In other words, they do the "big picture" conceptual and intellectual work, leaving the technical work for others.

Deliverables: project plans, site designs.

» This is the way to go if you know your Web site doesn't work but you're not sure why, if you don't have on staff librarians with time to develop a taxonomy, or if you need help developing an RFP or RFI. These contractors will give you site designs and project plans but not a functioning system.

5. Systems integrators. IT firms that do the whole job, from concept to implementation.

Deliverables: functioning applications.

» If your librarians and IT staff lack the skill and/or time to design your intranet or implement your e-commerce project, you can outsource the job to these firms. One caveat -- their expertise in taxonomies may be thin. One of the indexers we interviewed had recently been engaged to conduct introductory training for the firm's staff.

Conclusions
If you need help with taxonomies, first round up your in-house resources. If necessary, provide training to get them up to speed and bridge the communication gaps among disciplines. Then look for the skills you don't have in house. If you have a strong, interdisciplinary team, contractors in categories 1, 2, and 3 are a likely choice. If you don't have librarians and web designers on staff (or they are too busy with other tasks), look for help in categories 4 and 5. 

TIP
If you need to create a taxonomy and have neither the knowledge nor the staff to do the job, then a consultant is your best bet. However, if you have an employee who can learn the basics and has enough time to create a prototype, consider a hands-on Web course, such as Creating and Using Business Taxonomies or Integrating Taxonomies.

Hands-on courses have the following benefits:

• Your staff knows how to maintain the taxonomy after the consultant leaves.
• The prototype serves as a demo that can be used to educate stakeholders and prepare a business case for better metadata management.
• Data you enter in the Web-based Lab is exported at the end of the course and available for importing into your applications.
• Hands-on learning leverages existing human resources and usually costs less.

Update (as of October 18, 2006)
Since this article was written, several things have happened that affect the practice of taxonomy consulting:

1. Industry consolidation. Many search engines and auto-classification tools have been absorbed into other products and services. If you hired a consultant to create a taxonomy specifically for one of these tools, you may now find yourself trying to extract terms and categories for use in a new application. To get around this (or at least mitigate its effects), we see a growing interest in enterprise architecture and metadata repositories. See "What is architecture?"

2. End-user development. Search engines have become easier to use. In many cases, it's possible to get a basic search up and running without a programmer. Moreover, collaboration products like Microsoft SharePoint make it easier for employees to create their own Web sites and "lite" document management systems. Increasingly, taxonomy development is being done by people who are not professional indexers or catalogers. As a result, local taxonomies are proliferating. For cross-functional collaboration, local taxonomies must be integrated with each other as well as with an "official" enterprise taxonomy. See "Get ready for end user development" and the Integrating taxonomies Web course.

3. Social tagging and "folk taxonomies." End users are not only creating their own categories but are sharing them with others. A person can even use the RSS syndication standard to subscribe to the tags and bookmarks of another user. Like traditional indexing, social tagging can help alleviate one of the Internet's most vexing problems — how to sort the wheat from the chaff. See "Roundtable highlights: Google & SharePoint, social bookmarks."

Architectures vs. taxonomies
As a result of these and other changes in the business environment, our most experienced members are thinking less about how to create taxonomies for specific applications and more about creating a flexible, open architecture that will accommodate local variations and rapid technological change. The three business drivers are compliance, cost savings, and innovation.

Creating and implementing a new enterprise architecture isn't easy. First, you have to sell a very abstract concept to senior executives. Second, you have to get stakeholders to sign off on the proposed design. Third, you need a cost-effective transition process that safeguards existing data and causes minimum disruption to existing operations. We have found that the best way to meet these three objectives is to select two disparate but related business processes (e.g. product development and product marketing) and design a working prototype for a metadata repository — the skeleton that expresses the architecture.

The prototype contains real data about people, organizations, documents, business records, categories, and the relationships among them. It includes some application interfaces (e.g. adding thesaurus terms to a search engine), so people can see how metadata can flow from one business function to another.

An alternative to taxonomy consultants
Most people hire taxonomy consultants because they either lack the time or the expertise to create an application-specific hierarchy of topics. There are two problems with this approach:

1. The consultant must usually spend time (and your money) learning your business.

2. Inevitably, the taxonomy will bump up against global architecture issues that need to be resolved.

As an alternative, consider creating a prototype metadata repository that will help get the resources for metadata management, engage stakeholders in productive discussions about how metadata is used in business functions, and provide practical guidance on architecture design. As an expert on your business data and processes, you already know more than you think about taxonomies. Instead of a consultant, consider engaging a coach, teacher, and editor.