This is a summary. The complete article (2377 words) contains screen shots, drawings, links, examples, specific products, and additional implementation details. To access the complete article, become a member.

In late 2003, the staff at Time Inc. Interactive (TII) realized that something had to be done about its in-house metadata tool. Editors needed new kinds of relationships, vocabularies needed to be standardized, and web sites needed a better way to share metadata. Instead of expanding the existing relational database structure, the staff decided to move to the Semantic Web.

Six years later, the conversion has been made, and much of the desired functionality has been implemented. But the process was a roller coaster ride of enthusiasm and high hopes followed by misunderstanding, frustration, and organizational change. Today, interest in the new tool is on the rise again as a new group of Time publications recognizes its potential.

Although TII was a Semantic Web early adopter, most of its problems came not from the technology but from issues common to any metadata project involving multiple disciplines, stakeholders, and applications. In this article, we look at why TII chose the Semantic Web, the development process it used, the challenges it encountered, and the tips that its staff offer to those thinking about a similar project. Thanks to Barbara McGlamery and Rachel Lovinger for providing material for the article.

Why TII chose the Semantic Web
Making magazine sites more attractive to visitors while minimizing costs means using metadata in new and creative ways. To get the desired results, TII staff had to:

• standardize keywords, or at least create a "same as" relationship between the variants (e.g. "living area" vs. "living room;"
• make inferences about data (i.e. use software to identify multiple roles for the same person, such as actor or musician);
• add polyhierarchical relationships (e.g. the ability to label "Will Smith" as both an actor and a musician);
• add multi-part relationships so that a resource (e.g. a movie rating) could contain both user and professional reviews;
• make the "direction" of relationships explicit so that the content management software would "know" that Jake Gyllenhaal was an actor in the movie Zodiac — not a movie in which Zodiac was the actor;
• make the data sufficiently generic to be used on multiple web sites but flexible enough to fit into each site's "editorial voice;"
• improve the performance of the system so that all magazine web sites could process metadata at the same time.

The new system, called TOPICS (Time Ontologies for Publishing, Inference, Classification, and Semantics), uses the Semantic Web RDF output format with metadata stored in a Sybase database. RDF's subject-predicate-object syntax is more compatible with the metadata and relationships needed by the new system and can be "read" by both computers and humans.

The development process
Creating the new system was a six-stage process consisting of the following steps:

1. Wireframes and functional specification
2. Business rules
3. Ontology specification
4. Cleanup
5. Migration plan
6. Execution

Challenges
As a Semantic Web early adopter, TII had to work with rather primitive tools. However, the technology itself was the least of their worries. Other challenges were:

• Tight schedule
• Lack of realistic sample data
• Hard-coded metadata
• Lack of a visualization tool
• Inadequate communication

Can't a search engine do it?
Part of the misunderstanding was due to a common misconception among both business people and IT staff — that full text search engines can support all the features that users want. While most of them can display topic hierarchies and allow users to refine search results by "facets" such as date or geographic region, search engines can't handle the complex relationships needed by commercial publishers like Time, nor are they suitable for mixing and matching metadata to create new functionality. After investigating this option, TII staff realized that a separate metadata repository was necessary.

Tips
With six years of Semantic Web experience behind them, Barbara and Rachel have the following tips for others considering a similar project:

• Get an executive sponsor. Check in often with that person and let him or her do your PR while you focus on the work.
• Get involved early in discussions of functionality to identify all the places where data integration should happen.
• Do a thorough business analysis and use realistic sample data in the design (functional specification) stage to make sure you've accounted for all contingencies.
• Make sure that everyone on the team is fully educated about the goals, the process, and the benefits.
• Don’t hardcode the taxonomy into a display template.
• Establish guidelines for shared metadata.
• Develop detailed technical standards so that all system components will work together.
• Determine standards for web site template developers so that the right metadata is captured in the right format.
• To speed up response time, reduce time spent processing complex queries to the metadata repository and solve scalability issues before data migration.
• Allow enough time and resources for development, especially design and testing of the interface that editors will use to view, add, and change metadata.
• Don’t give up functionality that you consider essential to the site.
• Provide a way for the design and quality assurance staff to visualize relationships.

This is good advice for any metadata project, not just those based on the Semantic Web.

What's next?
TII's initial Semantic Web project accomplished the goals of allowing multiple web sites to share data, creating the relationships necessary to add new features, and generating more web site traffic. Now a new group of magazines in the Time family has shown an interest in using TOPICS.

Pros & cons of the Semantic Web
Commercial publishing is one of three areas now using the Semantic Web in production mode to solve real world problems (the others are government/nonprofits and medicine/healthcare). Common to all these applications are competitive pressures or mandates, complex data relationships, and the need to share data among different organizations or organizational units.

Could TII have accomplished the same thing by using an enhanced relational database? Probably, but they wanted a forward-looking technology. Some believe that the Semantic Web will start the long, slow decline of relational database technology. For now, though, the Semantic Web has a steep learning curve, lacks a mature toolset, and requires a substantial time and money investment. What the TII experience shows us is that there's a lot we can do now to minimize those costs, such as:

• educate business managers and IT staff about the limitations of full text search engines;
• analyze applications to identify the metadata and relationship requirements;
• implement business processes to capture the necessary metadata and store it in a relational database;
• avoid hard coding topic hierarchies in content management systems and other proprietary applications.

In short, we should be paying as much attention to business applications and metadata management practices as we do to the specific implementation technology.

More... (members only) How to become a member

Created on 1/3/2010 l Updated on January 7, 2010