Comments on: Lucene and the Corporate Environment http://lucene.grantingersoll.com/2009/07/17/lucene-and-the-corporate-environment/ Thoughts on Apache Lucene, Mahout, Solr, Tika and Nutch Sat, 10 Sep 2011 20:15:34 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: An example of Open Source we use – Apache Lucene | elnblog.com http://lucene.grantingersoll.com/2009/07/17/lucene-and-the-corporate-environment/comment-page-1/#comment-6790 An example of Open Source we use – Apache Lucene | elnblog.com Mon, 20 Jul 2009 17:18:27 +0000 http://lucene.grantingersoll.com/?p=227#comment-6790 [...] This blog post includes the following very relevant thought: Apache Lucene/Solr is used in more companies than a large majority, if not all, of the commercial vendors out there combined [...] [...] This blog post includes the following very relevant thought: Apache Lucene/Solr is used in more companies than a large majority, if not all, of the commercial vendors out there combined [...]

]]> By: Robert Douglass http://lucene.grantingersoll.com/2009/07/17/lucene-and-the-corporate-environment/comment-page-1/#comment-6783 Robert Douglass Sun, 19 Jul 2009 21:50:05 +0000 http://lucene.grantingersoll.com/?p=227#comment-6783 This is a sentiment I see in other open source projects too. It somehow has to do with the fact that a product that the engineers can download and use for free often sneaks in the back door and becomes part of the landscape. A proprietary package has to go through a formal vetting process, and hardly ever arrives unnoticed. And yes, the open source products often deliver more value, albeit with less fanfare. You said something, though, that helped me put a finer point on why the Drupal integration with Solr is so cool: "The fact is, that modeling is required by all the vendors due to the nature of search." Yes! And in Drupal, you do the modeling while you build your site, and Drupal then conveys this model to Solr automatically, so in most cases, the site builder doesn't need to pick a special model for the search - it's already there as a result of deciding how to build the site. Thanks for stating it so succinctly. (the Drupal module: http://drupal.org/project/apachesolr ) This is a sentiment I see in other open source projects too. It somehow has to do with the fact that a product that the engineers can download and use for free often sneaks in the back door and becomes part of the landscape. A proprietary package has to go through a formal vetting process, and hardly ever arrives unnoticed. And yes, the open source products often deliver more value, albeit with less fanfare.

You said something, though, that helped me put a finer point on why the Drupal integration with Solr is so cool: “The fact is, that modeling is required by all the vendors due to the nature of search.” Yes! And in Drupal, you do the modeling while you build your site, and Drupal then conveys this model to Solr automatically, so in most cases, the site builder doesn’t need to pick a special model for the search – it’s already there as a result of deciding how to build the site. Thanks for stating it so succinctly.

(the Drupal module: http://drupal.org/project/apachesolr )

]]> By: grant_ingersoll http://lucene.grantingersoll.com/2009/07/17/lucene-and-the-corporate-environment/comment-page-1/#comment-6776 grant_ingersoll Sat, 18 Jul 2009 17:13:30 +0000 http://lucene.grantingersoll.com/?p=227#comment-6776 Sure, I get where the "packaging" thing comes from, especially in concerns to pure, Java Lucene, which is just a library, i.e. a bunch of APIs. However, the point I'm trying to make is these people who throw out the "packaging" argument make it sound like it is so hard to do and that you need to have a PHd in Information Retrieval to implement Lucene or Solr and that, simply isn't true, as is evidenced by the VERY LARGE number of people who have already done it. I've seen plenty of installations of "packaged" vendors now that I feel comfortable saying Solr, in most cases, requires roughly the same amount of work, if not less than any of them. Sure, I get where the “packaging” thing comes from, especially in concerns to pure, Java Lucene, which is just a library, i.e. a bunch of APIs. However, the point I’m trying to make is these people who throw out the “packaging” argument make it sound like it is so hard to do and that you need to have a PHd in Information Retrieval to implement Lucene or Solr and that, simply isn’t true, as is evidenced by the VERY LARGE number of people who have already done it.

I’ve seen plenty of installations of “packaged” vendors now that I feel comfortable saying Solr, in most cases, requires roughly the same amount of work, if not less than any of them.

]]> By: Jukka Zitting http://lucene.grantingersoll.com/2009/07/17/lucene-and-the-corporate-environment/comment-page-1/#comment-6775 Jukka Zitting Sat, 18 Jul 2009 16:31:42 +0000 http://lucene.grantingersoll.com/?p=227#comment-6775 > given all the people that have gotten past it Agreed, though as shown by the original statement and the trackback above, not everyone is in this position either for real or perceived reasons. And often those people can and want to work around the issue by throwing money at it. In such cases it's often more attractive to throw that money to an established vendor with a "packaged" solution, than to hire someone to set up a custom system based on open source project. Even when the latter would be technically superior and much cheaper! So yeah, while the notion of "packaging" may well be overrated, there's still a good point here. Instead of "really succeed" in the quote above, I'd say "better succeed". > given all the people that have gotten past it

Agreed, though as shown by the original statement and the trackback above, not everyone is in this position either for real or perceived reasons. And often those people can and want to work around the issue by throwing money at it.

In such cases it’s often more attractive to throw that money to an established vendor with a “packaged” solution, than to hire someone to set up a custom system based on open source project. Even when the latter would be technically superior and much cheaper!

So yeah, while the notion of “packaging” may well be overrated, there’s still a good point here. Instead of “really succeed” in the quote above, I’d say “better succeed”.

]]> By: grant_ingersoll http://lucene.grantingersoll.com/2009/07/17/lucene-and-the-corporate-environment/comment-page-1/#comment-6774 grant_ingersoll Sat, 18 Jul 2009 12:34:26 +0000 http://lucene.grantingersoll.com/?p=227#comment-6774 Yeah, Jukka, I know. My point is mostly that this notion of "packaging" seems to be a bit overrated, given all the people (and that number is very high) that somehow have gotten past it, and like me when I started on Lucene, they aren't necessarily IR experts by training. Lucene, of course, is a very different beast from Solr. Lucene is and will always be a Java library. Of course that takes more work. As for all the other bullet points, Lucid (http://www.lucidimagination.com) offers all of them. Yeah, Jukka, I know. My point is mostly that this notion of “packaging” seems to be a bit overrated, given all the people (and that number is very high) that somehow have gotten past it, and like me when I started on Lucene, they aren’t necessarily IR experts by training.

Lucene, of course, is a very different beast from Solr. Lucene is and will always be a Java library. Of course that takes more work.

As for all the other bullet points, Lucid (http://www.lucidimagination.com) offers all of them.

]]> By: Jukka Zitting http://lucene.grantingersoll.com/2009/07/17/lucene-and-the-corporate-environment/comment-page-1/#comment-6773 Jukka Zitting Sat, 18 Jul 2009 07:17:48 +0000 http://lucene.grantingersoll.com/?p=227#comment-6773 I would interpret "packaging" as being more than just the arrangement of bits and the wrapping around it. Deploying and maintaining a search engine is hardly a trivial operation, and having a proper deployment project followed by maintenance service tied up with the "purchase" of the product is really valuable in environments where people don't have the luxury of the time to really understand the technology or to hang around on mailing lists asking questions and picking up advice. So, here's how I would package an "enterprise search" product based on Apache Lucene: * The bits from official Apache Lucene releases * Set of useful extra tools and plugins (proprietary and/or open source) * Good integration for a smooth OOTB experience * Guides for getting started with, using and administering the search engine * List of supported platforms with recommended scale (backed by an organized QA process) * Option to do a deployment project (at cost X/day) with a (certified?) Lucene search expert * Option for a maintenance service (at costs per level) backed by an organized support team with a ticketing system That's quite a bit added value on top of the plain open source project, and I'm sure there are many places that are happy to pay for that value. As far as I can tell, Lucid seems to be on a good track to providing all of these items. When do we see a product coming out? I would interpret “packaging” as being more than just the arrangement of bits and the wrapping around it. Deploying and maintaining a search engine is hardly a trivial operation, and having a proper deployment project followed by maintenance service tied up with the “purchase” of the product is really valuable in environments where people don’t have the luxury of the time to really understand the technology or to hang around on mailing lists asking questions and picking up advice.

So, here’s how I would package an “enterprise search” product based on Apache Lucene:

* The bits from official Apache Lucene releases
* Set of useful extra tools and plugins (proprietary and/or open source)
* Good integration for a smooth OOTB experience
* Guides for getting started with, using and administering the search engine
* List of supported platforms with recommended scale (backed by an organized QA process)
* Option to do a deployment project (at cost X/day) with a (certified?) Lucene search expert
* Option for a maintenance service (at costs per level) backed by an organized support team with a ticketing system

That’s quite a bit added value on top of the plain open source project, and I’m sure there are many places that are happy to pay for that value.

As far as I can tell, Lucid seems to be on a good track to providing all of these items. When do we see a product coming out?

]]> By: Lucene / Solr still needs hackers to get up and running | Productification http://lucene.grantingersoll.com/2009/07/17/lucene-and-the-corporate-environment/comment-page-1/#comment-6770 Lucene / Solr still needs hackers to get up and running | Productification Sat, 18 Jul 2009 02:23:16 +0000 http://lucene.grantingersoll.com/?p=227#comment-6770 [...] read an article posted on the Lucene blog - “Lucene and the Corporate Environment” If the list of companies using Lucene are not “corporate” environments, then I don’t [...] [...] read an article posted on the Lucene blog – “Lucene and the Corporate Environment” If the list of companies using Lucene are not “corporate” environments, then I don’t [...]

]]>