How to force a Solr 5.0.0 field/column to be a certain type

Solr 5 uses a managed schema by default, while Solr 4 used the schema.xml file.  Solr 5 automatically creates the schema for you by guessing the type of the field.  Once the type is assigned to the field, you can't change it.  You have to set the type of the field before you add data to Solr 5.

To change the schema in Solr 5, you will want to use the Schema API, which is a REST interface. Go to https://cwiki.apache.org/confluence/display/solr/Schema+API

https://cwiki.apache.org/confluence/display/solr/Schemaless+Mode describes a little bit about the Schemaless Mode (i.e. managed schema).  That page states the following:

"You Can Still Be Explicit - Even if you want to use schemaless mode for most fields, you can still use the Schema API to pre-emptively create some fields, with explicit types, before you index documents that use them. ... Once a field has been added to the schema, its field type is fixed."

If you are using the quick start guide for Solr 5, here's what you have to do if you want to explicitly specify the field types:

After you enter the following command:

bin/solr start -e cloud -noprompt

Then enter a command like this:

curl -X POST -H 'Content-type:application/json' --data-binary '{    "add-field" : {        "name":"MYFIELDNAMEHERE",        "type":"tlong",        "stored":true}}' http://localhost:8983/solr/gettingstarted/schema

The previous command will force the MYFIELDNAMEHERE field to be a tlong.  Replace MYFIELDNAMEHERE with the field name that you want to be explicitly set, and change tlong to the Solr type that you want to use.

After doing that, then load your data as usual.

Solving "Error initializing QueryElevationComponent." with Solr

You may be getting the following error message when trying to start up Solr:

ERROR - 2014-08-11 16:49:49.554; org.apache.solr.core.CoreContainer; Unable to create core: collection1
org.apache.solr.common.SolrException: Error initializing QueryElevationComponent.
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:868)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:643)
        at org.apache.solr.core.CoreContainer.create(CoreContainer.java:556)
        at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:261)
        at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:253)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.common.SolrException: Error initializing QueryElevationComponent.
        at org.apache.solr.handler.component.QueryElevationComponent.inform(QueryElevationComponent.java:251)
        at org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:651)
        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:851)
        ... 10 more

If you're getting this error, you may have changed your primary index from a string to an int.

To fix this, edit your example/solr/collection1/conf/elevate.xml file, and change it to the following:

<?xml version="1.0" encoding="UTF-8" ?>

<!-- ... -->

<elevate>

 <query text="foo bar">

<!--

  <doc id="1" />

  <doc id="2" />

  <doc id="3" />

-->

 </query>

 <query text="ipod">

<!--   <doc id="MA147LL/A" />  

   <doc id="IW-02" exclude="true" />

-->

 </query>

</elevate>

Specifically, comment out the <doc ... /> tags.

How to get gzip working for Jetty 8 and Solr

If you're getting the following error message in the solr.log file:

java.lang.ClassNotFoundException: org.eclipse.jetty.servlets.GzipFilter

Try doing the following:

Edit /opt/solr/solr-webapp/webapp/WEB-INF/web.xml and put the following in it:

<web-app ... >

<filter>

      <filter-name>GzipFilter</filter-name>

      <filter-class>org.eclipse.jetty.servlets.GzipFilter</filter-class>

      <init-param>

        <param-name>mimeTypes</param-name>  

        <param-value>text/html,text/plain,text/xml,application/xhtml+xml,application/xml,text/css,application/javascript,image/svg+xml,application/json,application/xml; charset=UTF-8</param-value>

      </init-param>

    </filter>

    <filter-mapping>

      <filter-name>GzipFilter</filter-name>

      <url-pattern>/*</url-pattern>

    </filter-mapping>

If you don't have a jetty-servlets*.jar file, then download it from http://mvnrepository.com/artifact/org.eclipse.jetty/jetty-servlets and put it in /opt/solr/lib

Restart jetty:

sudo /sbin/service jetty restart

If solr doesn't start up, check the /opt/solr/logs/solr.log file.

How to require password authentication for Apache Solr 4 and Jetty

Some of the documentation in the Solr wiki is outdated, so here's what worked for me.  In the following files, you can change the highlighted fields.

Edit /opt/solr/etc/jetty.xml and add the following:

<Configure>

    <Call name="addBean">
      <Arg>
        <New class="org.eclipse.jetty.security.HashLoginService">
          <Set name="name">Test Realm</Set>
          <Set name="config"><SystemProperty name="jetty.home" default="."/>/etc/realm.properties</Set>
          <Set name="refreshInterval">0</Set>
        </New>
      </Arg>
    </Call>

...

</Configure>

Edit  /opt/solr/solr-webapp/webapp/WEB-INF/web.xml and add the following:

<web-app>
...
  <security-constraint>
    <web-resource-collection>
      <web-resource-name>Solr authenticated application</web-resource-name>
      <url-pattern>/*</url-pattern>
    </web-resource-collection>
    <auth-constraint>
      <role-name>core1-role</role-name>
    </auth-constraint>
  </security-constraint>

  <login-config>
    <auth-method>BASIC</auth-method>
    <realm-name>Test Realm</realm-name>

  </login-config>

...
</web-app>

Create a file in /opt/solr/etc/realm.properties and put the following in it:

admin: yourPasswordHere,core1-role

You can change "admin" to whatever username you want.

For the password, you can either use the literal password, or use an OBF/MD5/CRYPT hash.  To create a hash, you can do the following:

cd /opt/solr

java -cp /lib/jetty-util-8.1.10.v20130312.jar org.eclipse.jetty.util.security.Password admin yourPasswordHere

The above utility will print out the hash to the screen, and you can chose either the OBF, MD5, or CRYPT line.  Make sure that you copy the entire line, including the "OBF:..." part.  Copy this line to the /opt/solr/etc/realm.properties file; the result will look something like this:

admin: OBF:1x1v1xmk1w9b1pyh1oq31uum1xtv1zej1zer1xtn1uvk1or71pw51w8f1xmq1x0r

,core1-role

The above is all in one line.  No line break.  Make sure you put the comma before the "core1-role".

After you've changed everything, restart the solr server.  If you're using Fedora, do this:

sudo /sbin/service jetty.sh restart

Then open up your web browser to http://localhost:8983/solr/#/collection1/query and the web browser should ask you for a password.  Enter admin for the user name and yourPasswordHere for the password.

Hint: if you get an error on the Solr admin webpage, you can check the log for warning messages.  The log file is here: /opt/solr/logs/solr.log

More info:

http://wiki.apache.org/solr/SolrSecurity

http://wiki.eclipse.org/Jetty/Tutorial/Realms

http://acdha.tumblr.com/post/207608584/jetty-solr-password-cheatsheet

How to Install MongoDB on Fedora

If you are using the instructions at http://docs.mongodb.org/manual/tutorial/install-mongodb-on-red-hat-centos-or-fedora-linux/ they say to create a file called /etc/yum.repos.d/10gen.repo - but if you try that, you may be getting the following error when trying to start the mongod service.

It turns out that mongodb is built into fedora (via yum), so you don't need to add the 10gen.repo file.  Here's how to install mongodb on Fedora:

1) Uninstall mongodb:

yum erase mongodb
yum erase mongo-10gen

2) Remove the 10gen.repo file

rm /etc/yum.repos.d/10gen.repo

3) Install mongodb via default yum

yum install mongodb mongodb-server

4) Start monogodb service

/sbin/service mongod start

NoSQL Comparison

Good overview of NoSQL databases: http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis