Thunderstone Blog: Customized Search Engine & Search Software

When the U.S. General Services Administration looked to upgrade the searching capabilities on its GSA Auctions® website, it could have written a host-based search engine to run on the mainframe system. Instead, GSA opted to use "off-the-shelf" Thunderstone Search Appliances — enabling the implementation of an affordable search solution that reduces system load while providing sophisticated search features expected by today's savvy users.

The GSA Auctions® website (http://www.gsaauctions.gov) empowers people in the general public to bid electronically on excess and/or surplus Federal assets. It supports fully web-enabled auctions that permit registered participants to bid on a single item or multiple items (lots) within specified timeframes.

Auctioned items can include run-of-the-mill office equipment and furniture, as well as more exotic Federal assets such as scientific equipment, heavy machinery, airplanes, vessels, vehicles, etc. The website enables GSA to auction-off and dispose of a widely geographically dispersed inventory of products. Participants can bid on and purchase available assets, without worrying about the actual physical location of any particular item or buyer.

Interested individuals may browse products offered on the auction site, or they can choose to search for items and place bids. With flexible and robust search capabilities powered by the Thunderstone Search Appliance, GSA Auctions® takes advantage of Thunderstone's proven technological expertise in the simultaneous searching of both structured and unstructured data.

Prior to implementing Thunderstone's Appliance-based search solution, things shaped up very differently on the website — in terms of its data access and retrieval functionality. Back-ended by a COBOL application utilizing a cgi interface to the mainframe's web server, the site originally supported a basic full-text search that required parsing the complete database to search each active item.

It seemed appropriate to consider deploying new technology, because the GSA Auctions® site deserved a more feature-rich and less resource-intensive search tool.

Driving TCO Lower by Integrating Mainframe Systems with "Off-The-Shelf" Products

Thomas Schaefer serves as Systems Architect and consultant to the General Services Administration. He helps the GSA identify innovative ways to derive optimal value from its Unisys ClearPath Mainframe investment and to maximize the productive use of all related I.T. resources.

According to Schaefer, GSA could have written a host-based search engine to run on the system. But, they figured, "Why reinvent the wheel?" Instead, GSA rapidly deployed several off-the-shelf Thunderstone Search Appliances to affordably implement a state-of-the-art search solution with reduced load on the ClearPath system for each request.

"By using Thunderstone Search Appliances, GSA Auctions® has gained the rich search features users have come to expect from sites like Google. The broader point is that, while there is a move to consider total cost of ownership and migrate applications to the ClearPath environment, not every problem requires developing custom mainframe software. Some problems are better solved by integrating existing components, mainframe or otherwise, into composite systems," Schaefer said.

Thunderstone's DataLoad API, which allows data to be pushed into the Appliance, gets used a lot by the people who administer the GSA Auctions® site. And they appreciate the "plug-and-play" reliability of their Thunderstone Search Appliances -- because every added server brings with it an additional production cost. Who wants to maintain yet another server? "With Thunderstone," Schaefer said, "I haven't logged-on in eight months, and everything keeps running just fine."

GSA continues to work with Thunderstone to further enhance and refine its electronic auction offerings to the public, using the Thunderstone Search Appliance model as its standard. Schaefer explained, "We like the fact that we still have the sophistication of Texis, but in an appliance."

The current search solution for the GSAAuctions.gov site includes two load-balanced Thunderstone Search Appliances (Enterprise Edition) located in a Minnesota production environment. Each Appliance runs active-active. Two additional Thunderstone Search Appliances (one Enterprise Edition and one Small Business Edition) are installed at a GSA facility in Utah, for the purpose of ongoing new application development and testing.

Kevin J. Payne, Director of System Applications at GSA, recalled, "We considered several internal and external search alternatives. Tom [Schaefer] came up with the idea of using a search appliance, which basically meant using either Google or Thunderstone."

"Thunderstone demonstrated a real willingness to make changes that we wanted and to customize their standard appliances to meet our needs. The ability to customize was a big thing. Plus, we really like the way the search engine runs," Payne added.

Customization for the GSAAuctions.gov search enhancement project enabled GSA's Thunderstone Search Appliances to add as many as 50 additional data fields (well beyond the three definable fields that come with the standard Appliance) for auction-item searches based on bid amounts and different geographic locations.

Payne said the procurement process was fast and easy. And he expects other key GSA projects to also have their search capabilities powered by Thunderstone Search Appliances in the months ahead. As an example he cited the GovSales.gov (http://www.govsales.gov) website. Part of the Federal Asset Sales Presidential e-Government (E-GOV) initiative, it will very likely implement its own Thunderstone Search Appliance by Fall 2008.

"These are systems that bring in billions of dollars a year to the U.S. government," noted Payne.

Providing Multiple Ways of Searching Structured/Unstructured Data to Find Precise Results

The GSA Auctions® site offers a variety of ways to search for desired items:

• Global Search
  The upper right corner of every page on the GSA Auctions® website contains a global search box that can search the entire site for all assets available to bid on and purchase. Users may refine displayed global search results by entering another search term in the search box above their search results and then clicking the "Search Within Results" button.

• Category Search
  The home page has a category search box for searching only user-selected categories of items. Directly above the category list, the category search box begins with "All Categories" as the default selection from the "category dropdown" list. This list changes to reflect items available for selection as users browse the different product categories.

• State Search
  Choosing a state from the Browse States dropdown box on the site's main menu header will produce a list of items located in the chosen state and will allow a user to enter keywords for searching the full-text information associated with all the displayed items.

• Advanced Search
  Accessible via a hyperlink under the global search box, advanced search allows users to: Delimit keyword search types (all words, any words, exact match.) Specify a current winning bid range. Search within a state for an item.

The professionals who work at GSA sometimes tend to write in government-spec language that typical users do not use themselves. For instance, what GSA refers to as "vehicles" on their auction site does not correspond to terms that most people use when searching for vehicles. Rather, people look for "cars," "sedans," "autos," "automobiles," "SUVs," etc.

With built-in Metamorph concept-based searching capabilities, the Thunderstone Search Appliances saved GSA hundreds of thousands of dollars in development costs, According to Payne.

Today's organizations have a mix of structured information such as in databases and unstructured text in documents, but most of their tools only effectively use one or the other set of data -- leaving a lot of potential value unrecognized.

Thunderstone Software LLC (http://www.thunderstone.com), a long-time industry R&D leader in providing extremely robust search solutions to organizations ranging from internet giants such as eBay and HotJobs to small NGOs and sole-proprietors, will present a FREE September 3, 2008 webinar that reveals some remarkably affordable and easy-to-implement answers to the growing problem of accessing and retrieving all your important data.

"3 Powerful Advantages That SQL Brings To Structured And Unstructured Search"

The webinar highlights three major competitive advantages of an SQL-enabled search platform, including:

• The flexibility of Structured Query Language to express complex queries

• A large base of developers who already know how to write SQL queries

• SQL's ad-hoc nature that allows applications to meet new needs easily

For more details and information on how you can register for the FREE September 3, 2008 webinar exploring "3 Powerful Advantages That SQL Brings To Structured And Unstructured Search," email or call Peter Thusat at +1 216 820 2200.

About Thunderstone
Thunderstone Software LLC (http://www.thunderstone.com) pioneered simultaneous searching of both structured and unstructured data with the Texis relational database optimized for full-text search. Texis, the innovative development platform behind Thunderstone's entire line of enterprise search products, is the only fully-integrated SQL RDBMS that intelligently queries and manages databases containing natural language text, standard data types, geographic information, images, video, audio and other payload data. Thunderstone first released and began marketing the Texis RDBMS in 1992.

Since 1981 Thunderstone has continued to develop its global reputation as provider of the world's most powerful, scalable and flexible enterprise search solutions.

Sales contact: Fred Harmon

+1 216 820 2200 ext.105

Media contact: Peter Thusat

+1 216 820 2200 ext.118

July 2008 - Archive

CONTENTS

• Questions on Search
• News Happenings
• How Do You Use Search In Your Enterprise?
• Customer Quote of the Month
• Subscription/Unsubscription and Contacts

QUESTIONS ON SEARCH

Q. How are different disciplines such as enterprise search, ECM and business intelligence coming together, and what challenges or opportunities does this present to organizations in the management of their information?

A. The ever-increasing amount of information produced by and available to companies today provides sizable challenges and commensurate opportunities. To go along with that, there are a growing number of products that can help with specific aspects of handlling that information. Each of the disciplines (such as ECM, BI and search) is expanding in an attempt to provide an enterprise-wide view of the available information. Each is gaining capabilities from the other disciplines but still maintains its core view -- whether that be maintaining a central repository for all information, analyzing selected information in special ways or creating an efficient method to rapidly access and retrieve information.

Companies need to really know what information they have. And they must make intelligent decisions about how it can best be used within their workflows to provide a competitive advantage. In most organizations there are many opportunities to make better use of the available information. However, it can be a challenge to find the right tool with the features and flexibility needed.

NEWS HAPPENINGS

Free-lance writer Paul Korzeniowski interviewed, among others, Thunderstone Texis customer Derek Matthews (Lead Knowledge Architect of Arbia, Inc.) for an article to appear in the August issue of DestinationCRM magazine.

Eric Kavanagh (Host of DMRadio -- http://www.dmreview.com/dmradio) will interview John Turnbull (CEO of Thunderstone Software LLC) 21 August 2008 for a scheduled web radio show broadcast on "Enterprise Search."

How Do You Use Search In Your Enterprise?

Businesses, governments, NGOs and educational institutions use Thunderstone's enterprise search solutions in a wide range of applications. How about you? How many of the following search applications have you considered, or perhaps already implemented, at your organization?

• auctions
  Thunderstone has worked with well-known companies (and federal government customers) to deploy some of the most highly successful online auction sites in the world.

• classified advertising
  From real estate and automotive sales to personals and help wanted ads, Thunderstone's flexible search tools have made setting up online classified advertising a breeze.

   

• CRM and help lines
  When your people need to rapidly find precisely the right response to the pressing concerns of your customers, Thunderstone provides powerful search products for results you can trust.

   

• document 'findability'
  An intranet doesn't do you much good, if the file server contains documents your authorized users can't locate and retrieve as desired. Help them access data faster and securely with Thunderstone Software.

   

• eDiscovery and litigation support
  Lawyers and judges began using Thunderstone's Metamorph concept-based searching technology for litigation support more than 20 years ago and for eDiscovery applications as early as 1997.

   

• intelligence collection/analysis
  Customers employ Thunderstone's technology to create multiple independent search collections, each consisting of one or more websites. They can create special collections of web information for their users, or for competitive intelligence purposes, by indexing targeted third-party websites anywhere on the internet.

   

• internet publishing
  Thunderstone empowers viewers of your online publications to do "parametric searches" for relevant content on a particular subject, written by a specific author and published within a certain timeframe. The parameters (defined attributes) of "author" and "publish date" enable precise queries that quickly return only the exact matches your users want. Keyword, full-text search alone does not offer this capability.

   

• knowledge management
  Systems integrators can rely on Thunderstone Software for extremely robust and secure enterprise-wide information retrieval across multiple repositories that serve different types of users, who all want dynamic, context-sensitive views of defined content (structured data) with the ability to refine results through full-text searching (unstructured data.)

   

• multimedia searching
  Access/retrieval needs increasingly involve many varieties of multimedia data. Thunderstone's customers can also search for audio, graphics, photos and video files stored in databases where, thanks to Thunderstone, all the content becomes immediately searchable -- including newly added or frequently changed data.

   

• online product catalogs
  Thunderstone supports very large, high-volume online catalogs that contain unstructured text (product names, narrative descriptions, etc.) and structured content (styles/sizes, prices, current in-stock availability status, etc.,) giving online shoppers the ability to search by item description, to navigate by price range or to efficiently do both in combination.

   

• vertical portals
  Market niche specialists, non-profit organizations and other uniquely-focused subject matter experts have turned to Thunderstone for affordable, easy-to-implement search tools with the power, versatility and scalability demanded by their vertical portal sites.

   

• website searching
  Happy customers around the world have discovered that websites (public-facing or internal) with many pages and lots of potentially valuable information will enjoy substantial benefits by adding the search advantages of Texis, Webinator or Thunderstone Search Appliances.

   

If you would like to share your own story about one or more effective ways that your organization has deployed Thunderstone search technology, please call +1 216 820 2200 or send an email to

Next month's edition of this newsletter will include a look at how the U.S. General Services Administration uses Thunderstone Search Appliances in its Unisys ClearPath Mainframe environment to handle different types of searching on a popular GSA auction site for selling excess and surplus government furniture, office products, scientific equipment, heavy machinery, airplanes, vessels, vehicles, etc.

CUSTOMER QUOTE OF THE MONTH

“We use a Thunderstone Search Appliance to search our company intranet. All the customization options are great. It has been a very valuable product to us. Overall, the product is very nice. And with all the customization options, we were able to make the search engine perform the way we wanted it to. Great job on delivering a great product.”

Jeremy Townsend
Programmer/Internet Analyst
Perdue Farms Inc.

Feedback, suggestions and questions are welcome to

June 2008 - Archive

CONTENTS

• Questions on Search
• Thunderstone's Product Investment Protection Program
• Subscription/Unsubscription and Contacts

QUESTIONS ON SEARCH:
2008 Enterprise Search Summit Review

Thunderstone Software LLC, a Gold-Level Sponsor of the fifth annual Enterprise Search Summit in New York City, sent Peter Thusat (Chief Marketing Officer) and Fred Harmon (Chief Sales Officer) to meet with and hear from many of the search industry's leading experts, analysts, consultants, solution providers, resellers and end-users/customers representing a wide range of large, medium-sized and small organizations.

Although we witnessed a surprising level of overall collegiality and a definite feeling that "we're all in this together," by the end of the three days of intensive workshops, conference sessions, panel discussions, etc. -- some emerging areas of general concensus and stark disagreement among the active participants had become apparent.

AREAS OF GENERAL CONSENSUS

Keyword search alone does not work well enough to satisfy typical users in the enterprise.

Whether focusing on litigants engaged in e-discovery or employees looking for subject matter experts within their own globally dispersed teams, results improve dramatically when the deployed enterprise search solution can expand upon traditional keyword search capabilities with built-in tools that anticipate the data needs of users. For example, some of Thunderstone's enhanced product capabilities include concept-based search with a user-customizable thesaurus, parametric search with faceted or guided navigation based on targeted data fields/attributes/tags and real-time search of changing information with immediate data insertion that does not require any updating of the search index.

Mere novelty no longer translates into real-world sales in today's search marketplace.

Stephen Arnold (President of ArnoldIT.com) monitors approximately 300 companies that offer enterprise search solutions, and he said his colleague Susan Feldman (Research Vice President of IDC) tracks about 325 such companies. Arnold noted that these search companies have an average founding date of 1997, with newer entrants into the market fading away and going out of business at an ever-increasing pace. The lifespans of Thunderstone's search industry competitors continue to decline rapidly, as customers around the world eschew a dizzying array of cool new features with limited actual value to them in favor of well-proven solutions that can handle their actual needs for powerful, scalable and flexible information access/retrieval.

Most people cannot find important data that remains locked away in multiple silos.

Everybody seems to agree on the necessity of making ALL their data searchable, neglecting neither unstructured data nor structured data. Thunderstone pioneered the simultaneous searching of structured and unstructured data, and ongoing leadership related to this challenge continues as one of Thunderstone's core competencies. Also, personalization of search results has become recognized as an enterprise imperative -- both in terms of data security (allowing users to view only content which they have the proper credentials to see) and in terms of data findability (making the desired information searchable and usable in whatever manner best suits a person's particular work objectives at the moment.)

AREAS OF SHARP DISAGREEMENT

Taxonomists may or may not play an essential role for quite some time in enterprise search.

Speakers, panelists and audience members shared differing experiences and opinions on questions surrounding the issues of ontologies, user-generated metadata, social bookmarking, metadata standardization, automated tagging, the roles of taxonomists and other specialists, etc. The upshot? Don't look for absolute best practices where it may make more sense to identify the practice that simply works best today in your unique environment.

Should software facilitate or replace our need for expert assistance in finding good answers?

Some aspire to better enable users to quickly locate and contact an available expert to answer a specific query, because, they insist, people really want an accessible expert who can help them determine precisely what to do next. Others seek to eliminate as much wasted manual effort as possible and, instead, try to take maximum advantage of computing power to fully automate the way we solve information retrieval problems.

Don't blame search technology for mistakes due to understaffing.

Several presenters articulated their belief that an insufficient number of I.T. employees dedicated to managing/optimizing enterprise search deserves most of the blame for lingering user dissatisfaction. Administrators need help analyzing query logs, supporting users, conducting training, maintaining data security and more. Whether you do or don't see big staff additions on your team's horizon, either way -- Thunderstone's comprehensive maintenance and support program makes a huge difference.

To get your questions on search answered, please email .

Whenever It's Time To Upgrade Your Thunderstone Search Solution,
Thunderstone's Got Your Back With Product Investment Protection.

An important factor in determining your Total Cost of Ownership (TCO) involves looking ahead and trying to figure out how much it may cost your organization when you decide in the future that you want to increase the size of your license, to expand the capacity of your current system and, in general, to step up into a larger search solution than you originally purchased. What will happen at that time to the initial investment you made in your search product? How, exactly, do you expect your anticipated upgrade path to work?

Thunderstone's entire line of search solutions come with a one-time, perpetual license. And we've made upgrading so easy and affordable that you'll probably wonder why everybody doesn't do it the same way.

The Thunderstone Product Investment Protection Program

If you ever get close to outgrowing your enterprise search solution from Thunderstone Software LLC, just ask for a quote on the Thunderstone product that can best accommodate your new requirements -- and you'll quickly discover some exciting facts about Thunderstone's Product Investment Protection Program, including:

• Thunderstone will credit your total initial product investment towards the price of your new, larger Thunderstone license -- although there may be an additional cost to upgrade hardware, if required.
• All of Thunderstone's search solutions automatically qualify for the investment protection program, and it doesn't matter which product or combination of products figure into your desired upgrade.

Most organizations have a rapidly-expanding quantity of data they need to make searchable in order for targeted stakeholders to accomplish their immediate and long-term objectives. As your search requirements grow, so should your search solution options -- without you worrying about the investment you already made in the past.

Thunderstone's customers can enjoy all the performance advantages of Texis, Webinator, Thunderstone Search Appliance and Thunderstone Parametric Search Appliance, along with a comprehensive Maintenance & Support Program plus the very substantial resource-maximizing benefits of the Thunderstone Product Investment Protection Program.

When it's time to upgrade, Thunderstone's got your back.

To find out more, email or call Thunderstone Sales today at +1 216 820 2200.

Feedback, suggestions and questions are welcome to