Search is one of the most misunderstood functionalities in IT

by Xavier Comments: 0

There is a phrase I use all the time: “Search is one of the most misunderstood functionalities in IT”. And I think it is very accurate.

The problem lies in two different aspects:

  1. Developers don’t know how to use search engines. And it is ok, search engines can be hard to tune appropriately and it is a specialised niche. In some cases, there are some search engines which are awfully expensive.
  2. Developers are lazy. Let me explain this one.

Let’s say that I am setting up an application for selling cars. Potential customers always look for the same things, which are make, model, year, sort by price and so on and so forth. There is a set of meta data that is important and required to find what you are looking for. So what is the solution to this problem?

Use a database where each field is stored in a separate column and look for the fields accordingly, just like in the following image. It is a mistake or at least a UX horror. I hate database driven search, but that is just my personal opinion.

A typical database driven search input

The correct wat of doing it is by providing a single search box. How? Like this:

A proper search box

If you want to learn how, please click on the following link to my Pluralsight course to get started with enterprise search using Apache Solr!

pluralsight.com/training/courses/TableOfContents?courseName=enterprise-search-using-apache-solr

 

Installing Solr in Windows or Linux?

by Xavier Comments: 2

I have been a fan of Microsoft technologies all my life, probably because I’ve spent a lot of time working with .Net and related technologies. Eventually I became also an Apple fanboy as some people have called me.

But something that I haven’t been called a fan of is Linux. Don’t get me wrong, I think Linux is extremely important, but in my case I have not worked with it as much as I think I should have.

But now I am in a part of my life where I need to run Apache Solr in a production environment. What do I do? What comes naturally.

In a nutshell I set up a Windows machine in Amazon AWS, install Java, download Solr, java -jar start.jar, modify solrconfig.xml, modify schema.xml, turn around a few more knobs and test. Once I am happy I install Tomcat and voila, I have a single node for production. It is a small application with very few documents and a reasonable traffic, so it is all good. And besides, it is amazing how much a Solr instance in AWS can handle.

Anyway, my need keeps growing and I believe I need to set up a more resilient installation. Of course SolrCloud comes to mind, but I am thinking of how the pros install Solr.

So what do I do? Install Solr in a Linux AMI. Also, as I need monitoring now in place I set up SemaText. One downside of Windows is that at least when using SemaText, you can’t monitor on Windows, only Linux.

And there you go, that is my piece of advise. But not only from me, I’ve heard from many sources that Linux can be more performant and stable when running Apache Solr.

If you want to get more information on how to install Solr in a Linux instance, please follow the following link to the Apache Solr Reference Guide: https://cwiki.apache.org/confluence/display/solr/Taking+Solr+to+Production

Also, if you want to learn more about Getting Started with Enterprise Search with Apache Solr, please follow this link to my course on this subject:
www.pluralsight.com/courses/discussion/enterprise-search-using-apache-solr

Find out who is connecting to your database – lovely query!

by Xavier Comments: 0

So I needed to figure out which servers are connecting to which databases. That sounds like a complicated thing but it isn’t!

Just run this query (and make sure you have appropriate permissions)

SELECT
loginame, hostname, program_name, DB_Name(dbid), last_batch
FROM
sys.sysprocesses
where hostname <> ”
order by last_batch desc

Atlassian Summit Presentation Video – Collaboration is More than Communication

by Xavier Comments: 0

Collaboration is defined as “the action of working with someone to produce or create something.” Yet, many confuse communicating with collaborating. True collaboration gives you and your project an edge by aligning efforts towards a clear objective. I’ll show how teams can achieve true collaboration with JIRA Agile.

And here is my presentation from the Atlassian Summit 2014 on this topic


You can view in the Atlassian archives https://summit.atlassian.com/archives/2014/collaboration-teams/collaboration-is-more-than-communication-jira-agile

Hope you enjoy!

It is sooo easy to create bad usability… i.e. look at our friend

by Xavier Comments: 0

It is so easy to be part of the 90% of Sturgeon’s law! How hard can it be to create something with bad usability? Apparently it is extremely easy. Take a quick look at this example I am within the corporate Sharepoint. I want to upload a document. So I click Add document. assessment1 And now I get this popup.  My question is how hard can it be for Microsoft to create #1 a decent upload control to select location #2 that has enough space so that I can actually read the current location #3 and I clicked Add document within a specific folder. I think the majority of times I would want to add the document where I clicked. assessment2   How hard can all this be? Well, SharePoint is a great product but there are some areas for improvement and I would love if someone at the SharePoint team could take a stab at this.

I heard something about good UX a bit a ago that makes perfect sense. A UI is like a joke, if you have to explain it then it is not that good!

Solr Configuration

by Xavier Comments: 0

For those of you that landed directly on this page, I am working on a series of post that help people “Get Started with Enterprise Search using Apache Solr”. In this post I cover one specific module. If interested in watching the course please click here: http://pluralsight.com/training/courses/TableOfContents?courseName=enterprise-search-using-apache-solr . Click here to get to the starting point: https://www.xaviermorera.com/2014/06/getting-started-with-enterprise-search-using-apache-solr/ You’ve made it to module 3! Thanks a lot if this is the case. So far we started by understanding “Why Enterprise Search?” and “Why Solr?”, both very valid questions and fundamental if you want to go down the “search” path – which by the way if more people really understood search this could really save millions and millions of hours of frustrated users! And remember my promise: I will show you how to, in a few hours, build something that may cost thousands of dollars to develop and may take a long time to learn! That is pretty powerful if you ask me. After that, I walk you through architecture to show you where to place the search engine and also the internal architecture of Solr – don’t forget Lucene! And now we get to module 3, Solr Configuration. I start by showing you how to download solrinstaller.exe, click next/next/next and Solr is up and running! Yay…. Not quite Solr is not yet at that point. There is no Windows installer (yes, I am a Windows guy… no Linux) but it is extremely simple none the less. So in this module I show you

  • How to land on your feet when starting the wonderful search engine. I explain how you use Jetty server which can be started immediately. But this is only for dev purposes.
  • Then we move on to the areas of when you first open you search administration page, located in http://localhost:8983.
  • We move on to each one of the ares of the Admin UI. It is not 100% necessary to know them all, but there are some that you actually should. You may not care now, but if you run into errors then it is useful to know how Logging is where you should go.

logging

  • And then I do something that may save you a lot of time. I show you how to run Solr within Tomcat, which is suitable for production.

Let me tell you a quick story from a while ago so that you understand. I could be ashamed to say that it took me several hours to run Solr within Tomcat the first time I did it. I am not ashamed. I was clueless, some blog posts were technically accurate but confusing. In this training I show you how to do it in less than 7 minutes. This will save you time and frustration if you have never done it before and need to do so! And this is the end of the module. I prepare you for what’s to come, index data and the fun part: searching! I hope you enjoy and contact me if you have questions, concerns or most preferably criticism so that I can improve my trainings!

Architecture of an Enterprise Search Application

by Xavier Comments: 0

For those of you that landed directly on this page, I am working on a series of post that help people “Get Started with Enterprise Search using Apache Solr”. In this post I cover one specific module. If interested in watching the course please click here: http://pluralsight.com/training/courses/TableOfContents?courseName=enterprise-search-using-apache-solr . Click here to get to the starting point: https://www.xaviermorera.com/2014/06/getting-started-with-enterprise-search-using-apache-solr/ In the second module I start by trying to make my point on how

  • Beauty is in the eye of the beholder
  • Users usually don’t care about the application
  • They care about getting accurate and fast results.

Users don’t care about the how, engineers care about the how. The people that use it don’t. and that’s what counts! Then I focus for a few minutes on explaining architecture from two points of view:

  • Where, architecturally speaking, within an application does the search engine fit in

architecture

  • Solr’s architecture

Solr architecture


The third section of this module explains what makes Solr move and shake, mainly the search engine underneath: Lucene. And finally I explain to you the parts of a search application. This part may be a bit subjective depending on who looks at it, but most principles apply across the board. If you are interested on viewing this module you can start here:  enterprise-search-apache-solr-m2 Here is the table of contents:

You can also go back to the main page of this series by clicking here: https://www.xaviermorera.com/2014/06/getting-started-with-enterprise-search-using-apache-solr/

Solr Configuration

by Xavier Comments: 0

For those of you that landed directly on this page, I am working on a series of post that help people “Get Started with Enterprise Search using Apache Solr”. In this post I cover one specific module. If interested in watching the course please click here: http://pluralsight.com/training/courses/TableOfContents?courseName=enterprise-search-using-apache-solr . Click here to get to the starting point: https://www.xaviermorera.com/2014/06/getting-started-with-enterprise-search-using-apache-solr/

The third module is made up of four sections:

2014-07-30_1821

Installation

In this section I will show you how to install Solr. It is not as simple as solrinstaller.exe, next, next next… but it not too complicated either.

Demo

Watch, learn and follow. You will be able to get Apache Solr running in a few minutes.

Admin UI

In this section I go over each one of the options in the Admin UI. I don’t spend too much time as there are many secitons, but enough for you to understand what they are for.

Tomcat

Solr comes with Jetty server out of the box. However Jetty is not for production purposes, instead it should be for development only. I will show you how to run Solr within Tomcat. BTW: if you do not know how to do this, it may take you a few hours of research to learn on your own. Watch this clip and you will learn in a few minutes!

If you are interested on viewing this module you can start here:  enterprise-search-apache-solr-m3 

This is the table of contents:

You can also go back to the main page of this series by clicking here: https://www.xaviermorera.com/2014/06/getting-started-with-enterprise-search-using-apache-solr/

How to Add Data to a Different Solr Core

by Xavier Comments: 0

Today a good friend asked while watching my Getting Started with Enterprise Search Using Apache Solr course in Pluralsight on how to index data in different cores in Solr. Solr Cores   Here is the answer.

  • Let’s say you want to feed data to Solr’s default collection, called collection1, using the update handler, then do this:

java -Durl=http://localhost:8983/solr/update  -Dtype=text/csv -jar post.jar “Two sample courses with author test.csv”

  • But if you have a different core, i.e. psdemo, then do this:

java -Durl=http://localhost:8983/solr/psdemo/update  -Dtype=text/csv -jar post.jar “Two sample courses with author test.csv”

Easy as pie! If interested in watching the course please click here: http://pluralsight.com/training/courses/TableOfContents?courseName=enterprise-search-using-apache-solr .

Why Solr & Enterprise Search

by Xavier Comments: 0

For those of you that landed directly on this page, I am working on a series of post that help people “Get Started with Enterprise Search using Apache Solr”. In this post I cover one specific module. If interested in watching the course please click here: http://pluralsight.com/training/courses/TableOfContents?courseName=enterprise-search-using-apache-solr .Click here to get to the starting point: https://www.xaviermorera.com/2014/06/getting-started-with-enterprise-search-using-apache-solr/

I start my Enterprise Search Apache Solr training course by conveying my beliefs to you. I truly believe that a good search implementation can help tremendously with the adoption of your application or providing huge savings by letting your users find what they are looking for. I can even say that a Search Engine is a very straightforward and easily understood name, but maybe sometimes I think that you should have a Find Engine!

Anyway, in this module I start with why. I really hope we both are in the same lane. Immediately I tell you what I promise you will get from this course. It is a pretty powerful promise, but believe me it is true.

I promise… That in the next couple of hours I will teach you to build things that might take you weeks to learn on your own and we will create together a search experience that could cost thousands of dollars to build

And they I take you in a few minutes into some interesting topics, for example Why Enterprise Search. This might seem normal to you, but most people have no idea about search. Search is one of the most misunderstood functionalities in IT!

And then of course we touch on a bit of history of search engines, famous Solr sites so that you can see with your own eyes how the big guys really are using Solr! If they are, why shouldn’t you?

Also, Apache Solr is not alone. I have a personal opinion at this point that it is the best, but others disagree. Let’s all agree to disagree then, but in this training I will focus solely on Apache Solr.

Are you ready to watch? If so you can watch it here after you log in to Pluralsight: http://pluralsight.com/training/Player?author=xavier-morera&name=enterprise-search-apache-solr-m1&mode=live&clip=0&course=enterprise-search-using-apache-solr

Here is the table of contents for this module:

You can also go back to the main page of this series by clicking here: https://www.xaviermorera.com/2014/06/getting-started-with-enterprise-search-using-apache-solr/