Enterprise Search: Running Your Own Search Engine

Aug 18, 2008 • 7:10 pm | comments (1) by twitter | Filed Under Search Engine Strategies 2008 San Jose
 

The value of search as a true enterprise platform has been touted for years, yet few organizations have seriously embraced the opportunity. This session will provide an overview of enterprise search fundamentals and explore the enterprise search marketplace in depth. Current enterprise search tools, design issues, and real-world examples of effective enterprise search implementations will be discussed. Whether your organization has a well-established Intranet or an Intranet that has just been launched, you'll walk away from this session with a better understanding for the principles of a well-designed enterprise search solution.

Moderator:

Speakers:

Avi Rappoport

Scope:
  • Public-facing web site search
    • online stores
    • informational
    • hybrids
  • Intranet Search
    • Departmental
    • Extranets for partners
    • Research (narrow and deep)
    • Enterprise (wide and shallow)

Finding content

  • Robot crawlers
    • follow links
    • discover servers
    • same problems as webwide search robots
      • java, javascript, flash navigation links
      • unlinked content
      • redirects, infinite loops
  • Direct data source access
    • databases, CMS, document stores, data silos
    • web services, REST, and other APIs
    • Updates can trigger re-index
    • Avoid extra page content, navigation.

Un-optimized content

  • No title tag or duplicate titles
  • Invalid HTML
  • Modification dates unreliable
  • Non-HTML files
    • no file properties or duplicate properties
    • huge file sizes
    • multiple versions of the same file

Dealing with short queries

  • Average 1-3 words in a query
    • often ambiguous
    • multi-word queries are usually phrases
  • Smart query processing
    • Auto-complete for popular terms
    • Expand queries (synonyms, acronyms)
    • Match with Best Bets
    • Offer spelling suggestions (hard to do well).

Retrieval and relevance

  • Expanded queries find more match files
  • Relevance and perception
  • Complex algorithms rarely help. Users don’t care how many results there are, they usually just look at the first page. Even treating searches as AND or OR have similar results, best if you put the results with all words in phrase at top, however.
  • Page-rank not applicable
    • Most links are navigational, not meaningful
  • Simple term frequency ranking works
  • Heuristics: position and phrase matching
  • Personalization – small bang for big bucks. Might be useful in specific situations, but not usually.

No to Spam, yes to Metadata

  • Your own content is (mostly) reliable
  • Metadata adds value
    • search and show author
    • several more, missed them.

Search Analytics

  • Search Traffic
  • Top queries
    • navigation
    • topical
    • trends over time
  • No-match queries
    • add text, metadata, new content
    • improve no-matches page.
Enterprise search tips
  • Basic SEO is smart
  • Users rarely type long queries
  • Giving them what they really want is hard
  • Metadata can change the dynamic
  • Search analytics rock
  • This is more librarianship than marketing – expose your content rather than drive people to something.

www.searchtools.com consult8@searchtools.com

Bill French.

Enterprise search: definition?
Finding information…

  • inside the enterprise?
  • for employees?
  • for partners?
  • for clients?

His definition is Search Solutions that help to Achieve the Objectives of the Enterprise.

Observations:
Employees may (or may not) know what the yare looking for, or even know that they should be looking. One study said that workers may spend up to two hours a day searching.

Ideal Operational Efficiency
Information magically finds you in a context where it has the greatest value. Think less about the search function, and more about the software applications that employees are using. Those are the key moments where you can provide the greatest productivity to the user.

Findability: Three Key Observations

  1. Value directly proportional findability. If something cannot be found, it has no value.
  2. Findability is directly proportional to focus.
  3. Focus is dependent on two elements: the message and the presentation architecture.

Bill disagreed with Avi, and feels that you can’t always trust the internal metadata. Employees do want to be heard, they write things and want them to be found, and even the internal system can be gamed.

Andy Feit
Enterprise Search: Approaches to Choosing and Using Your Own Search Engine.
Three views: Knowledge Management, “Google for your intranet”, Application Centric.

Knowledge management is a range of practices used by organizations to identify, create, represent, distribute and enable adoption of what it knows and how it knows it (source: wikipedia). Can drive tremendous value, but not without the investment and commitment.

Cheap and Cheerful: If KM is so challenging, what options are out there? “Google for your intranet”

  • Could literally be Google, but there are plenty of other options for basic spidered search.
  • Generally works well when underlying content publishing process is in place. Problem can be if you have multiple versions of a document, and people finding wrong version.
  • Downside: like web search, it puts the onus on the user to do the work.
    • Worse because web relevancy models are not present.

Learning from the web…

  • Going vertical, or these days “micro-vertical”
  • Leverage focused content, engaged audience, and common goals
  • Often combined with Web 2.0-style interfaces:
    • User Generated Content
    • Tagging, Rankings, Comments, etc.

In the enterprise, silos are not always bad. We call “vertical role- and task-aware”. You searches can actually be quite relevant because it is specifically in your area of interest.

Examples of good silos and specific results.

  • Consulting -> Statements of Work Builder
  • Heathcare -> Patient Post-op Instructions. Instead of generic post-op instructions, take the existing content in your network (patient records) and making it specific for the user (patient). Would know exactly what procedures were done and have specific information for patient.
  • Market Research -> Dynamic Publishing
  • Legal Departments…
  • Marketing Teams…
  • Sales…

He showed screenshots of several specific vertical search applications.

Rebecca Thompson
“Using Search to Jump Start Collaboration”

Enterprise search today: screenshot of existing enterprise search for a company. Documents are organized by sources, such as Sharepoint, Documentum, intranet, etc. Another search type option is clusters and facets.

Enterprise search can be a hard sell to executives. You can present case that employees use a lot of time searching, but executives ask if people will be more productive with the time that they don’t search, or would they just mess around online. Missed good example of how to convince executives here. A compelling argument is employees can make use of existing internal knowledge. How do you deal with people that retire or leave? What happens to the data on the laptop after the person leaves? A lot of knowledge is now gone.

The Collaboration Starting Point. The natural starting point for collaboration is search. Why?

  • Search can access all content and data
    • Unlike standalone social tools, social search is bootstrapped by the data
  • Everyone in an organization can use search
    • It is simple and intuitive
  • Corporations have been struggling for years to organize data across departments and repositories – let users help!

Social Search in the Enterprise:

  • Social Tagging: voting. Users can actually do this to help clean up data.
  • Social Tagging: rating. You can either just show rating that user gave so others can see how useful it was, or you can put it in the backend too so that it can be used to increase relevancy of results.
  • Social Tagging: keywords. Users can tag documents with keywords. This can help by adding information that is not in the document itself. Can be freeform or from dropdown.
  • Social Tagging: annotations. Users can enrich search results by annotating existing results with their own thoughts and commentary. Almost like a mini blog post, or looks like comments on items in Friend Feed.
  • Social Bookmarking: Virtual Folders. Represented as a folder in the interface, but like a tag on the backend of the system.

Quickly went through at TV Network News Division search engine results.

Social Search

  • allows enterprises to tap into and make use of human knowledge within their organization
  • provides the opportunity to go beyond finding information to enriching it
  • reveals valuable insights into the collaborative intelligence of the organization.

Contributor: Keri Morgret

Previous story: Video Search Engine Optimization (VSEO)
 

Comments:

Sean

08/30/2008 09:13 pm

I like the idea of entrepreneurial search engines. Combine that with a specific library based on users, we could retain so much knowledge that has been lost in the written word. Although wiki's are based on user data the incentive to correct misinformation or give a peer review is not really there. For instance on Helium.com you can read others works and slowly build an army of "data crunchers" because they are being paid to write well and review works of other people. The nice thing about that is that anyone can do it..... More search engines like this, given some changes to user interaction, should have more relevance in today's online world.

blog comments powered by Disqus