Using Solr from C# Made Dead Easy with SolrNet

by Xavier Comments: 0

I was having a conversation with an solutions architect a few days ago about a specific Solr project with .NET. He was telling me some of the details of what they were going to build, how they were going to call Solr’s REST API for querying and ….

Stop the press… Stop right there… I said….

You are going to call Solr’s REST API? Really? The fact that Solr has a REST API is massively useful, but why do you want to reinvent the wheel.

If you want to use Solr from .NET, I explained, the best way is to use SolrNet(https://github.com/mausch/SolrNet). SolrNet is an Apache Solr client for .Net that allows for you to use Solr from .NET in a super easy and efficient way. It abstracts Solr in such an easy way that you basically just create a POCO object (Plain Old Clr Object – not to wrongly called Plain Old C# Object as many people do) and presto, you have functions to Add, Delete, Commit and more. Let me show you a small sample I built about a week ago in literally just a few minutes to index a few thousand mails from Pluralsight’s (the best online training resource available IMHO – and yes I am a bit biased here!) author distribution list.

To create this demo, where I indexed about 4k emails I did the following

– Download Solr, 4.10 at the time [takes a few minutes to download]

– Clone collection1 and rename it authorslist, don’t forget to change the collection name

– Change in schema.xml so that you have the following fields (and remember this is just a quick and dirty example that I wanted to show how provide a searchable index of all author mails inside a Wordpress site. Looks like this:

<field name=”itemid” type=”string” indexed=”true” stored=”true” required=”true” multiValued=”false” />
<field name=”subject” type=”text_general” indexed=”true” stored=”true”/>
<field name=”sent” type=”date” indexed=”true” stored=”true” omitNorms=”true”/>
<field name=”sendername” type=”string” indexed=”true” stored=”true” multiValued=”false”/>
<field name=”recipients” type=”string” indexed=”true” stored=”true” multiValued=”true”/>
<field name=”body” type=”text_general” indexed=”true” stored=”true”/>
<field name=”htmlbody” type=”text_general” indexed=”false” stored=”true”/>

– Make itemid uniquekey instead of id

– Modify solrconfig.xml to add a new requesthandler, so that I can play a bit with facets and weights, nothing out of the ordinary

– And now to index the mails I just wrote a few lines of code to read from my Outlook, using the Office Primary Interop Assemblies (known as PIAs) in a console application

– Add SolrNet to the console app

– Create a POCO object to represent whatever I added to the schema. It should look like this. Please notice the attributes, which is what tells SolrNet what it is representing in Solr. This is my POCO object:

public class MailSolr {

[SolrUniqueKey(“itemid”)]
public string ItemId { get; set; }

[SolrField(“subject”)]
public string Subject { get; set; }

[SolrField(“sent”)]
public DateTime Sent { get; set; }

[SolrField(“sendername”)]
public string SenderName { get; set; }

[SolrField(“recipients”)]
public ICollection Recipients { get; set; }

[SolrField(“body”)]
public string Body { get; set; }

[SolrField(“htmlbody”)]
public string HtmlBody { get; set; }
}

– And write the following lines to index

ISolrOperations solr = GetSolr();
int it = 0;
foreach (Microsoft.Office.Interop.Outlook.MailItem i in lM)
{
it++;
MailSolr m = new MailSolr();
m.ItemId = i.EntryID;
m.Subject = i.Subject;
m.Sent = i.SentOn;
m.SenderName = i.SenderName;
m.Body = i.Body;
m.HtmlBody = i.HTMLBody;

solr.Add(m);
}
solr.Commit();

As you can see adding a document with SolrNet is dead easy. Simply connect to Solr, create a new instance of your POCO object, solr.Add() and solr.Commit(). How much easier would you want it to be?

Next Tuesday I will talk about the NIH syndrome (not invented here) around SolrNet.

Developers: Think about consequences, implications, performance and related

by Xavier Comments: 0

Today I find myself removing a stored proc from our dbml because it is no longer in use. This stored proc updates the document information to be exported.

I check our source code and indeed no one uses it. So I go ahead and remove it. Is that the end of the story? Usually it is, but I decided to use a bit of common sense to find out if this is the last step or no. It wasn’t. Let me explain and get to the moral of the story quickly:

– We have a requirement to export the information of up to several tens of thousands of documents into Excel/Word/XML in a single go.

– We used to need to make sure that the document information is up to date, and given the source is in XML and XPath is rather slow, we built an intermediate SQL Server table to be able to have speedy exports.

– We made a change and now information is always up to date in the intermediate exports table, however the code that CHECKS if the information is up to date was not removed.

– So the info is not updated as it is already up to date, but we are still checking if we should ACTUALLY DO THE UPDATE. The code is still there.

What does this translate into? An unnecessary and performance degrading step. We need to check to determine if thousands of documents are up to date (and twice but this is another story) when they already are. This means there are two queries that we could easily avoid and save time as they add zero value but take several minutes to execute.


Moral of the story: when making a change don’t focus only on the change, focus also on the side effects of a change.

This should be a rule of common sense for all developers.

This configuration section cannot be used at this path. This happens when the section is locked at a parent level

by Xavier Comments: 0

Say what?

I got this error and was pretty much clueless. It is under Windows Server 2012 and it didn’t occur in 2008

HTTP Error 500.19 - Internal Server Error
The requested page cannot be accessed because the related configuration data for the page is invalid.

Detailed Error Information:
Module IIS Web Core
Notification BeginRequest
Handler Not yet determined
Error Code 0x80070021
Config Error This configuration section cannot be used at this path. This happens when the section is locked at a parent level. Locking is either by default (overrideModeDefault="Deny"), or set explicitly by a location tag with overrideMode="Deny" or the legacy allowOverride="false".
Config File \\?\C:\myapp\web.config
Requested URL https://localhost:443/api/v1/Login
Physical Path C:\myapp\api\v1\login
Logon Method Not yet determined
Logon User Not yet determined

Config Source:
90: < /modules >
91: < handlers >
92: < remove name="ExtensionlessUrlHandler-ISAPI-4.0_32bit" / >

More Information:
This error occurs when there is a problem reading the configuration file for the Web server or Web application. In some cases, the event logs may contain more information about what caused this error.
View more information »

How did I solve it?

Add Application Development from the Features! Remember to add the 4.5 stuff if you need it
appdev

And then do the following in a command prompt, if required. Update to Allow on the handlers and modules section in C:\Windows\System32\inetsrv\config\applicationHost.config and remember to open as administrator

< section name="handlers" overrideModeDefault="Allow" / >
< section name="modules" allowDefinition="MachineToApplication" overrideModeDefault="Allow" / >