Posted to Web4Lib listserv on 10/19/2005 by Yuliya Lef

Question: We are considering purchasing a federated search system. As a result, I am wondering if someone has done a comparison analysis between various products or if there is one available online. If you would be willing to share your findings or provide recommendations, I would very much appreciate it.

Response 1: We are currently also going through a very comprehensive evaluation process of federated search engines and link resolvers at our university. Everything we did so far is available on our blog at http://fedsearch.blogspot.com

We first conducted a literature survey, from which we compiled a comparative overview - also available on our blog. From that we compiled an evaluation instrument (also on blog) with evaluation criteria. Currently we are in the process of interacting with the vendors. Each of the vendors gets an opportunity to demo their product via Interwise/WebFeat to 20+ evaluators (representatives from 3 large universities in South Africa). We then have a Q&A session, followed by a hands-on session.

Ina Smith, University of Pretoria

Response 2: At the risk of airing my dirty laundry in a very public space, I'd offer the following answer to Yuliya.

First, a brief preface: I've been involved with two federated search product selection processes at two different ARL libraries. I've also spent most of the last two years working with either Ex Libris' MetaLib or Endeavor's ENCompass, so feel I have more than enough stored experience and frustrations with them and with federated searching to offer a useful opinion.

Now the opinion: Federated searching is and, for the foreseeable future will remain, a deeply flawed enterprise. This is largely due to the differences between targets, but also due to the hash that some of the software products out there make of what can/should be a relatively simple thing. Despite the flaws, however, it's worth pursuing federated searching, if only because: a) it's going to get better with time, and b) even in its current state, you can do some cool things with it.

If you compare MetaLib and ENCompass, the outcome is laughable. MetaLib's central knowledge base makes it the obvious winner, and that's just the first reason that this is the case. Unless you're fond of writing your own connections for a wide variety of z39.50, XML gateways, HTTP screen scrapes, etc., don't pay a lot of money for any federated search product unless it sits on top of a large and well-maintained knowledge base. I'd declare MetaLib the general winner in that category, although I'm sure someone (a vendor, perhaps?) out there is itching to take me on for saying that.

Having just praised ML, however, I'll point out that it's interface is a royal pain in the keester. Having just returned from Access2005 in Edmonton, it's safe to say that x-server from Ex Libris, their attempt to offer an interface API to MetaLib, is not there quite yet. Even when it does get there, however, who will have the time and skills to build an interface essentially from scratch. We don't.

Perhaps a better route, and one that makes sense for institutions that aren't dripping cash, is to look around at some of the more interesting things happening in the open-source world. Check out DBWiz from Simon Fraser University, for example (http://dbwiz.lib.sfu.ca/dbwiz/). It's in production at both SFU and at UWinnipeg.

If you've got cash and want the most well-grounded tool, use MetaLib. If, however, you've got even a tiny amount of interest in trying to do something on your own using open-source software, then I'd suggest DBWiz or other such initiatives.

Dale Askey, Kansas State University

Response 3: We are building an interface from scratch here at Cal State San Marcos using the Metalib X-Server -- as are three or four other libraries.

But we are an early adopter of the X-Server. As soon as we finish our work here at San Marcos, we'll make our code open source (and I hope the other X-Server libraries will as well), so that new libraries who are interested in using the Metalib X-Server can start building their interfaces from . . . well, some point after scratch.

It's actually not that difficult to make an interface against an XML-based API. A little code to pass information back and forth and XSLT to make it pretty.

So I guess there's actually a middle point between commercial and open source metasearch system -- and that is to have a group of libraries sharing open source code for the interface to a system, while the vendor maintains the underlying application and knowledgebase.

I think that can actually work out really well. If I am going to spend my time working on a metasearch system and sharing code, I would much rather spend it on the interface (which is the most important part of the system) than building Z39.50 and other types of connections.

Dave Walker, Cal State San Marcos

Response 4: Actually a lot of vendors are already using this approach, letting MuseGlobal or WebFeat maintain the connections and then building an interface on top. Sirsi, Endeavor, Innovative, Ovid, etc have all done that with Muse; Dynix, etc. with WebFeat.

Jim Campbell, University of Virginia

Response 5: Not quite. Our consortium purchased WebFeat via Dynix and we are now implementing it. Dynix has not done any work on top of WebFeat nor does WebFeat provide any APIs or other mechanisms for us to build around its search engine in the fashion described with MetaLib X-Server.

John Weible, University of Illinois

Created on October 26, 2005 l Updated on October 30, 2005