Easiest Way of Issuing Solr Commit Command

by Xavier Comments: 0

There are a couple of ways to trigger a commit command in Solr. The easiest way is via a URL:


(Replace localhost:8983 with your Solr url and )

But you can also commit using the Documents option from the Admin UI. Simply navigate to Documents, using this URL:


And select Solr Command (raw XML orJSON), adding the command


Solr Command







Submit Document! It just works. And if you are using SolrCloud, the command goes to everyone.

The NIH Syndrome in Action: Calling Solr’s REST API

by Xavier Comments: 0

As I mentioned in my previous post, I was having a chat with a search architect on C# and Solr and he was telling me how they were going to call Solr using its REST API, which IMHO is not the best way to go.

I recommended SolrNet and I stick with this recommendation because you do not want to reinvent the wheel. SolrNet has been built over the course of several years, there are plenty of people using it and they have had the time to understand the functionality that Solr provides and code it accordingly. The problem that you will face if you implement everything via REST calls is that you need to take into consideration all possible scenarios, which in a lot of cases does not happen, And that’s when exceptions start to occur.

In some cases it is not possible to use a library as there may be legal implications. But in other cases it is because of the NIH syndrome, or the not invented here syndrome.

Sometimes teams prefer to have everything created in house. In my opinion that may apply pretty well if you have a closed library as you are at the mercy of the company’s support and sometimes even their ability to profit and therefore continue to exist.

But if it is open source, like SolrNet, then don’t be afraid. You are in control, you can modify the code if need be – as with any open source, of course following the licensing terms – and definitively you can benefit from the experience of many others that have traveled the same road you are currently on.

But of course, be nice. If you make an improvement, please contribute back to the SolrNet community. Then we all mutually benefit.

Five ways meetings suck and how to make them rock!

by Xavier Comments: 2

Meetings are a double edge sword. On one hand they are very useful for brainstorming, communication within and between teams and in general when used appropriately it sparks collaboration; moving projects forward.

The problem lies when meetings are misused or abused which sadly sometimes tends to happen more often than not. And that’s why I want to tell you 5 reasons why meetings suck and how to make them rock!

#1 People use them to keep busy vs being productive: I’ve seen many times how people set up meetings because for them going to the meeting is the work. Even worse, a lot of meetings end with “let’s set up another meeting to continue the discussion”. This is very common among Project Managers or Product Managers. Just as a fun mental exercise, take a couple of minutes and think if you know a few. I do.

How to deal with it: Make sure that every meeting that you attend or control over has a real and very clear objective. Even more, make sure that it is important that this meeting takes place. If the objective is not important or required, then simply defer it until it is the right time.

#2 A 1 hour meeting is not a 1 hour meeting: Have you ever been in a 1 hour meeting with 15 other folks and wondered how much work you actually have to get done instead of just sit there waiting for the 1 hour mark or your 2 minute turn to talk. Also, they guy next to you seems to be thinking the same while playing Candy Crush or in the best case answering emails? Well it is much worse than that. A 1 hour meeting with 16 people is actually 2 man days of work lost forever, gone in time. I shiver just to think of how many dollars are wasted because of overcrowded meetings!

How to deal with it: Not everyone needs to be invited to every meeting. Make sure each person has a clear and known reason of why they are there and (friendly) kick out of the room anyone that doesn’t. Also, as the old adage says, “divide and conquer”. A very broad topic where too many people are involved can be broken down into smaller chunks with smaller groups. You only need one person to coordinate among the teams, and 1 is always much better than a committee. This is very common with Scrums that get abused and end up looking more like office parties. Bonus points if people bring food as that distracts even further!

#3 Meetings without agenda: Another reason why some meetings take much longer than expected is that there isn’t a clear agenda and objective. This leads to endless talking, going on tangents and again scheduling new meetings to continue the conversation. People also confuse a meeting agenda with just the subject of the meeting. For example, “Discussion on data” is not clear, it is broad and subjective. This scenario goes hand in hand when overall project vision is not very clear or there are many unknowns.

How to deal with it: Create a clear agenda with well defined points and if possible, use my favorite feature of Scrum: timebox. Parkinson’s law is the adage that “work expands so as to fill the time available for its completion” and this applies very well to meetings. If you don’t have a limit, people will use all time possible for a discussion. Timebox in a reasonable way and you may see wonderful results. And if the issue is overall project vision, try planning shorter term until there is a clearer view of the road ahead.

#4 Interruptions break flow for makers: If you work hard on improving yourself you are probably familiar with the concept of “flow”, but if you are not then interiorize this: “flow is the mental state of operation in which a person performing an activity is fully immersed in a feeling of energized focus, full involvement, and enjoyment in the process of the activity”. This is when you get a lot done. Developers usually experience this without knowing when “time flies”. But in reality what happened was that the brain was fully focused and engaged in a task with potentially wonderful results.

There is something else that you should be familiar with that is the “Maker’s Schedule, Manager’s Schedule”. Now let me explain. If you are a maker, i.e. programmer, you work most likely in large chunks of time, for example from 8 am to noon. During that time you focus on solving a problem and if you are interrupted, it may take up to 15 minutes to pick up where you left off. This means that if there are multiple meetings throughout the day, your maker’s schedule gets broken down into many small chunks that do not allow for focused full involvement into a task. I worked on multiple projects in Microsoft’s main campus in Seattle and it seemed like everyone in this particular division worked in 1 hour intervals which wreaked havoc on development work. Culprit: too many (project) managers in the same building. Real development took place somewhere else.

How to deal with the managers’s schedule:

Manager’s usually have smaller chunks of work that can be broken down into 1 hour slots, hence their impulse to “book meetings with the team to catch up”. They may have the best intentions, but this usually has a negative impact. Even worse, managers are higher in the food chain so let me ask you this, when was the last time that you simply cancelled a meeting with your boss because he was interrupting you?

How to deal with the maker’s schedule: The first step is for the manager to understand the impact he is having on his whole team with constant interruptions. This sometimes requires the maker to approach the manager and explain the implications of the constant interruptions. Then the manager should aim to schedule (required) meetings when they cause the least impact, which can be near start, noon or end of day. Also, use asynchronous communication. This sounds complex, but it is basically email, chat or even better, something like Jira to manage the makers progress.

#5 Another chance for talkers to take the stage, not doers: This is a very tricky one as it requires a leader that is aware of the capabilities and responsibilities of the team members attending the meeting. The problem usually lies in that human personalities are extremely different so some people tend to talk a lot while others prefer silence. And talking is not directly proportional to doing. Someone might talk a lot and do very little and the actual guy doing the work will just sit there and listen. This creates then an unbalanced and biased view as not all of the involved parties will communicate appropriately and the outcome of the meeting might not be the best.

How to deal with it: It is the meeting organizer’s responsibility to understand what is the involvement of each one of the parties invited to the meeting in respect to the meeting agenda. Then ask questions specifically on each one of the topics to get the correct information and help obtain the best possible outcome for the meeting.

In summary meetings can help your team leap forward. Just remember to schedule meetings only when absolutely required instead of as a way to keep busy, inviting only those that are definitively required, with a very clear agenda and objective, at times where they don’t interrupt the maker’s schedule and guiding the meeting to make sure that everyone contributes their share.

Any other recommendations?

Search is one of the most misunderstood functionalities in IT

by Xavier Comments: 0

There is a phrase I use all the time: “Search is one of the most misunderstood functionalities in IT”. And I think it is very accurate.

The problem lies in two different aspects:

  1. Developers don’t know how to use search engines. And it is ok, search engines can be hard to tune appropriately and it is a specialised niche. In some cases, there are some search engines which are awfully expensive.
  2. Developers are lazy. Let me explain this one.

Let’s say that I am setting up an application for selling cars. Potential customers always look for the same things, which are make, model, year, sort by price and so on and so forth. There is a set of meta data that is important and required to find what you are looking for. So what is the solution to this problem?

Use a database where each field is stored in a separate column and look for the fields accordingly, just like in the following image. It is a mistake or at least a UX horror. I hate database driven search, but that is just my personal opinion.

A typical database driven search input

The correct wat of doing it is by providing a single search box. How? Like this:

A proper search box

If you want to learn how, please click on the following link to my Pluralsight course to get started with enterprise search using Apache Solr!



Installing Solr in Windows or Linux?

by Xavier Comments: 2

I have been a fan of Microsoft technologies all my life, probably because I’ve spent a lot of time working with .Net and related technologies. Eventually I became also an Apple fanboy as some people have called me.

But something that I haven’t been called a fan of is Linux. Don’t get me wrong, I think Linux is extremely important, but in my case I have not worked with it as much as I think I should have.

But now I am in a part of my life where I need to run Apache Solr in a production environment. What do I do? What comes naturally.

In a nutshell I set up a Windows machine in Amazon AWS, install Java, download Solr, java -jar start.jar, modify solrconfig.xml, modify schema.xml, turn around a few more knobs and test. Once I am happy I install Tomcat and voila, I have a single node for production. It is a small application with very few documents and a reasonable traffic, so it is all good. And besides, it is amazing how much a Solr instance in AWS can handle.

Anyway, my need keeps growing and I believe I need to set up a more resilient installation. Of course SolrCloud comes to mind, but I am thinking of how the pros install Solr.

So what do I do? Install Solr in a Linux AMI. Also, as I need monitoring now in place I set up SemaText. One downside of Windows is that at least when using SemaText, you can’t monitor on Windows, only Linux.

And there you go, that is my piece of advise. But not only from me, I’ve heard from many sources that Linux can be more performant and stable when running Apache Solr.

If you want to get more information on how to install Solr in a Linux instance, please follow the following link to the Apache Solr Reference Guide: https://cwiki.apache.org/confluence/display/solr/Taking+Solr+to+Production

Also, if you want to learn more about Getting Started with Enterprise Search with Apache Solr, please follow this link to my course on this subject:

Find out who is connecting to your database – lovely query!

by Xavier Comments: 0

So I needed to figure out which servers are connecting to which databases. That sounds like a complicated thing but it isn’t!

Just run this query (and make sure you have appropriate permissions)

loginame, hostname, program_name, DB_Name(dbid), last_batch
where hostname <> ”
order by last_batch desc

Atlassian Summit Presentation Video – Collaboration is More than Communication

by Xavier Comments: 0

Collaboration is defined as “the action of working with someone to produce or create something.” Yet, many confuse communicating with collaborating. True collaboration gives you and your project an edge by aligning efforts towards a clear objective. I’ll show how teams can achieve true collaboration with JIRA Agile.

And here is my presentation from the Atlassian Summit 2014 on this topic

You can view in the Atlassian archives https://summit.atlassian.com/archives/2014/collaboration-teams/collaboration-is-more-than-communication-jira-agile

Hope you enjoy!

It is sooo easy to create bad usability… i.e. look at our friend

by Xavier Comments: 0

It is so easy to be part of the 90% of Sturgeon’s law! How hard can it be to create something with bad usability? Apparently it is extremely easy. Take a quick look at this example I am within the corporate Sharepoint. I want to upload a document. So I click Add document. assessment1 And now I get this popup.  My question is how hard can it be for Microsoft to create #1 a decent upload control to select location #2 that has enough space so that I can actually read the current location #3 and I clicked Add document within a specific folder. I think the majority of times I would want to add the document where I clicked. assessment2   How hard can all this be? Well, SharePoint is a great product but there are some areas for improvement and I would love if someone at the SharePoint team could take a stab at this.

I heard something about good UX a bit a ago that makes perfect sense. A UI is like a joke, if you have to explain it then it is not that good!

Solr Configuration

by Xavier Comments: 0

For those of you that landed directly on this page, I am working on a series of post that help people “Get Started with Enterprise Search using Apache Solr”. In this post I cover one specific module. If interested in watching the course please click here: http://pluralsight.com/training/courses/TableOfContents?courseName=enterprise-search-using-apache-solr . Click here to get to the starting point: https://www.xaviermorera.com/2014/06/getting-started-with-enterprise-search-using-apache-solr/ You’ve made it to module 3! Thanks a lot if this is the case. So far we started by understanding “Why Enterprise Search?” and “Why Solr?”, both very valid questions and fundamental if you want to go down the “search” path – which by the way if more people really understood search this could really save millions and millions of hours of frustrated users! And remember my promise: I will show you how to, in a few hours, build something that may cost thousands of dollars to develop and may take a long time to learn! That is pretty powerful if you ask me. After that, I walk you through architecture to show you where to place the search engine and also the internal architecture of Solr – don’t forget Lucene! And now we get to module 3, Solr Configuration. I start by showing you how to download solrinstaller.exe, click next/next/next and Solr is up and running! Yay…. Not quite Solr is not yet at that point. There is no Windows installer (yes, I am a Windows guy… no Linux) but it is extremely simple none the less. So in this module I show you

  • How to land on your feet when starting the wonderful search engine. I explain how you use Jetty server which can be started immediately. But this is only for dev purposes.
  • Then we move on to the areas of when you first open you search administration page, located in http://localhost:8983.
  • We move on to each one of the ares of the Admin UI. It is not 100% necessary to know them all, but there are some that you actually should. You may not care now, but if you run into errors then it is useful to know how Logging is where you should go.


  • And then I do something that may save you a lot of time. I show you how to run Solr within Tomcat, which is suitable for production.

Let me tell you a quick story from a while ago so that you understand. I could be ashamed to say that it took me several hours to run Solr within Tomcat the first time I did it. I am not ashamed. I was clueless, some blog posts were technically accurate but confusing. In this training I show you how to do it in less than 7 minutes. This will save you time and frustration if you have never done it before and need to do so! And this is the end of the module. I prepare you for what’s to come, index data and the fun part: searching! I hope you enjoy and contact me if you have questions, concerns or most preferably criticism so that I can improve my trainings!

Architecture of an Enterprise Search Application

by Xavier Comments: 0

For those of you that landed directly on this page, I am working on a series of post that help people “Get Started with Enterprise Search using Apache Solr”. In this post I cover one specific module. If interested in watching the course please click here: http://pluralsight.com/training/courses/TableOfContents?courseName=enterprise-search-using-apache-solr . Click here to get to the starting point: https://www.xaviermorera.com/2014/06/getting-started-with-enterprise-search-using-apache-solr/ In the second module I start by trying to make my point on how

  • Beauty is in the eye of the beholder
  • Users usually don’t care about the application
  • They care about getting accurate and fast results.

Users don’t care about the how, engineers care about the how. The people that use it don’t. and that’s what counts! Then I focus for a few minutes on explaining architecture from two points of view:

  • Where, architecturally speaking, within an application does the search engine fit in


  • Solr’s architecture

Solr architecture

The third section of this module explains what makes Solr move and shake, mainly the search engine underneath: Lucene. And finally I explain to you the parts of a search application. This part may be a bit subjective depending on who looks at it, but most principles apply across the board. If you are interested on viewing this module you can start here:  enterprise-search-apache-solr-m2 Here is the table of contents:

You can also go back to the main page of this series by clicking here: https://www.xaviermorera.com/2014/06/getting-started-with-enterprise-search-using-apache-solr/