Streamlining And Standardizing Open Source Search
Customer Overview
Customer Overview
A large not-for-profit research institutes online collection offers one-stop searching of 2 million records, including nearly a quarter of a million media files (images, media files, online journals, and other resources) distributed across dozens of archives, databases, museums, and libraries. Both institutional and public researchers use the online search service.
Business Issue
The research institute had developed a broad set of customized search capabilities using Solr, enabling users to search across a very diverse set of information. Recently, the IT staff noticed that while most of the searches were very fast, some were very slow. Over time, the complicated taxonomy they had built had become difficult to maintain; as new IT employees came on board, they faced a steeper learning curve for handling the customized code that had been developed. In addition, they needed to deploy a replicated environment to provide business continuity. Finally, some of the functions they had customized were now available out of the box from Solr.
To ensure they were making best use of resources, the institute asked
Lucid Imagination to review their Solr implementation and make best practices recommendations to ensure the implementation was consistent with best practices in the industry and in the marketplace.
Solution Description
The research institute engaged the following services from Lucid Imagination:
Search Health Check
Training
ExpertLink consulting services
Lucid Imagination consultants did an in-depth analysis, reviewing Solr caches, identifying better ways to set up configuration files, and removing inefficient unused code. The consultants reviewed the institutes custom code modules to identify opportunities for efficiency gains, specifically those functions that can now be handled by native Solr, with a view to configuring for best practices. The work included reviewing over a dozen customized Java modules, and making suggestions for more efficient database access. Recommendations were also made concerning the Solr server caching environment, including the placement of an external HTTP caching server to handle common queries and reduce the load on the Solr server. Most importantly, with the transition to standardized Solr capabilities, the institutes
search application is now set up for smoother adoption of improvements in future releases of Solr.
Lucid also provided training to internal IT staff. to bring them up to speed on the latest Solr capabilities, and establish better self-reliance for ongoing search application development and refinement. The training also included a focus on knowledge transfer, so that changes made during consulting on analysis and reconfiguration were well understood by the team at-large, and they could be take ownership of them and maintain them effectively going forward.
Lucid was also able to advise the institute on hierarchical facetingusing taxonomic information to create more detailed search results that are presented in a tree structure. Lucid also introduced the institute to search opportunities with geo-searching. While this capability is not yet in core Solr, Lucid was able to advise them on related projects and how best to make use of them.
Outcomes
All custom code developed by the institute's IT staff was thoroughly reviewed and unused modules were removed to improve efficiency and performance. A comprehensive recommendation was made for a framework to improve scalability. Because there had been no backup of the search index, Lucid provided the institute with strategy for how to set up a redundant environment with a replication server, and helped coach the implementationfor example, providing recommendations on update intervals to balance performance needs and hardware requirements.
Additional guidance was provided on a more efficient way to index new records as they come in, writing Java code that will transform the data into XML, associate descriptive tags with it, and then put it into the search database repository.
by: Lucid Imagination
Gadgets Gift The Power of Joomla and K2 Fast and easy photo editing!!! Apparel for the Serious Skater and Skateboarder HID conversion Kit – Single beam and dual beam Upright Brooms - Natural Fiber And Plastic Payday Money: Addition Funds To Overcome Sudden Urgencies Church Fundraiser Activities for Help unbiased opinion of chris brown and rihanna situation Sun Direct and Its Great Offers How Rock And Roll Are Coming To The Stage What Is Hypothyroidism And How Does It Affect You? Paper Shredders: Various Types And Uses