Solr is a scalable, ready to deploy, search/storage engine optimized to search large volumes of text-centric data. Solr is enterprise-ready, fast and highly scalable.

Solr is an open-source search platform which is used to build search applications. It was built on top of Lucene (full text search engine). Solr is enterprise-ready, fast and highly scalable. The applications built using Solr are sophisticated and deliver high performance.

It was Yonik Seely who created Solr in 2004 in order to add search capabilities to the company website of CNET Networks. In Jan 2006, it was made an open-source project under Apache Software Foundation. Its latest version, Solr 6.0, was released in 2016 with support for execution of parallel SQL queries.

Solr can be used along with Hadoop. As Hadoop handles a large amount of data, Solr helps us in finding the required information from such a large source. Not only search, Solr can also be used for storage purpose. Like other NoSQL databases, it is a non-relational data storage and processing technology.

In short, Solr is a scalable, ready to deploy, search/storage engine optimized to search large volumes of text-centric data.

Features of Apache Solr

1. Restful APIs − To communicate with Solr, it is not mandatory to have Java programming skills. Instead you can use restful services to communicate with it. We enter documents in Solr in file formats like XML, JSON and .CSV and get results in the same file formats.
2. Full text search − Solr provides all the capabilities needed for a full text search such as tokens, phrases, spell check, wildcard, and auto-complete.
3. Enterprise ready − According to the need of the organization, Solr can be deployed in any kind of systems (big or small) such as standalone, distributed, cloud, etc.
4. Flexible and Extensible − By extending the Java classes and configuring accordingly, we can customize the components of Solr easily.
5. NoSQL database − Solr can also be used as big data scale NOSQL database where we can distribute the search tasks along a cluster.
6. Admin Interface − Solr provides an easy-to-use, user friendly, feature powered, user interface, using which we can perform all the possible tasks such as manage logs, add, delete, update and search documents.
7. Highly Scalable − While using Solr with Hadoop, we can scale its capacity by adding replicas.
8. Text-Centric and Sorted by Relevance − Solr is mostly used to search text documents and the results are delivered according to the relevance with the user’s query in order.
9. 
   

Lucene in Search Applications

Lucene is simple yet powerful Java-based search library. It can be used in any application to add search capability. Lucene is a scalable and high-performance library used to index and search virtually any kind of text. Lucene library provides the core operations which are required by any search application, such as Indexing and Searching.

How do Search Engines Work?

Any search application is required to perform some or all of the following operations.

SOLR URL in your browser.

http://localhost:8983/

If the installation process is successful, then you will get to see the dashboard of the Apache Solr user interface as shown below.

SOLR General Terminology

The following is a list of general terms that are used across all types of Solr setups −

1. Instance − Just like a tomcat instance or a jetty instance, this term refers to the application server, which runs inside a JVM. The home directory of Solr provides reference to each of these Solr instances, in which one or more cores can be configured to run in each instance.
2. Core − While running multiple indexes in your application, you can have multiple cores in each instance, instead of multiple instances each having one core.
3. Home − The term $SOLR_HOME refers to the home directory which has all the information regarding the cores and their indexes, configurations, and dependencies.
4. Shard − In distributed environments, the data is partitioned between multiple Solr instances, where each chunk of data can be called as a Shard. It contains a subset of the whole index.

SolrCloud Terminology

In an earlier chapter, we discussed how to install Apache Solr in standalone mode. Note that we can also install Solr in distributed mode (cloud environment) where Solr is installed in a master-slave pattern. In distributed mode, the index is created on the master server and it is replicated to one or more slave servers.

The key terms associated with Solr Cloud are as follows −

1. Node − In Solr cloud, each single instance of Solr is regarded as a node.
2. Cluster − All the nodes of the environment combined together make a cluster.
3. Collection − A cluster has a logical index that is known as a collection.
4. Shard − A shard is portion of the collection which has one or more replicas of the index.
5. Replica − In Solr Core, a copy of shard that runs in a node is known as a replica.
6. Leader − It is also a replica of shard, which distributes the requests of the Solr Cloud to the remaining replicas.
7. Zookeeper − It is an Apache project that Solr Cloud uses for centralized configuration and coordination, to manage the cluster and to elect a leader.

Configuration Files

The main configuration files in Apache Solr are as follows −

1. Solr.xml − It is the file in the $SOLR_HOME directory that contains Solr Cloud related information. To load the cores, Solr refers to this file, which helps in identifying them.
2. Solrconfig.xml − This file contains the definitions and core-specific configurations related to request handling and response formatting, along with indexing, configuring, managing memory and making commits.
3. Schema.xml − This file contains the whole schema along with the fields and field types.
4. Core.properties − This file contains the configurations specific to the core. It is referred for core discovery, as it contains the name of the core and path of the data directory. It can be used in any directory, which will then be treated as the core directory.

Starting Solr

After installing Solr, browse to the bin folder in Solr home directory and start Solr using the following command.

This command starts Solr in the background, listening on port 8983 by displaying the following message.

Starting Solr in foreground

If you start Solr using the start command, then Solr will start in the background. Instead, you can start Solr in the foreground using the –f option.

Starting Solr on another port

Using –p option of the start command, we can start Solr in another port, as shown in the following code block.

Stopping Solr

You can stop Solr using the stop command.

This command stops Solr, displaying a message as shown below.

Restarting Solr

The restart command of Solr stops Solr for 5 seconds and starts it again. You can restart Solr using the following command −

This command restarts Solr, displaying the following message −

Solr ─ help Command

The help command of Solr can be used to check the usage of the Solr prompt and its options.

Solr ─ status Command

This status command of Solr can be used to search and find out the running Solr instances on your computer. It can provide you information about a Solr instance such as its version, memory usage, etc.

You can check the status of a Solr instance, using the status command as follows −

On executing, the above command displays the status of Solr as follows −

Solr Core

A Solr Core is a running instance of a Lucene index that contains all the Solr configuration files required to use it. We need to create a Solr Core to perform operations like indexing and analyzing.

A Solr application may contain one or multiple cores. If necessary, two cores in a Solr application can communicate with each other.

Creating a Core

One way to create a core is to create a schema-less core using the create command, as shown below −

Here, we are trying to create a core named Solr_sample in Apache Solr. This command creates a core displaying the following message.

You can see a core selector where you can select the newly created core

Using create_core command

Alternatively, you can create a core using the create_core command. This command has the following options −

Let’s see how you can use the create_core command. Here, we will try to create a core named my_core.

On executing, the above command creates a core displaying the following message −

Solr Core Deletion

You can delete this core using the delete command by passing the name of the core to this command as follows −

On executing the above command, the specified core will be deleted displaying the following message.

Indexing in Apache Solr

In Apache Solr, we can index (add, delete, modify) various document formats such as xml, csv, pdf, etc. We can add data to Solr index in several ways.

In this chapter, we are going to discuss indexing −

1. Using the Solr Web Interface.
2. Using any of the client APIs like Java, Python, etc.
3. Using the post tool.

In this chapter, we will discuss how to add data to the index of Apache Solr using various interfaces (command line, web interface, and Java client API)

Adding Documents using Post Command

Solr has a post command in its bin/ directory. Using this command, you can index various formats of files such as JSON, XML, CSV in Apache Solr.

Browse through the bin directory of Apache Solr and execute the –h option of the post command, as shown in the following code block.

On executing the above command, you will get a list of options of the post command, as shown below.

Example

Suppose we have a file named sample.csv with the following content (in the bin directory).

Visit the homepage of Solr Web UI using the following URL −

http://localhost:8983/

Select the core Solr_sample. By default, the request handler is /select and the query is “:”. Without doing any modifications, click the ExecuteQuery button at the bottom of the page.

Adding Documents using the Solr Web Interface

You can also index documents using the web interface provided by Solr. Let us see how to index the following JSON document.

Step 1

Open Solr web interface using the following URL −

http://localhost:8983/

Step 2

Select the core Solr_sample. By default, the values of the fields Request Handler, Common Within, Overwrite, and Boost are /update, 1000, true, and 1.0 respectively, as shown in the following screenshot.

Now, choose the document format you want from JSON, CSV, XML, etc. Type the document to be indexed in the text area and click the Submit Document button, as shown in the following screenshot.

Updating the Document Using XML

Following is the XML file used to update a field in the existing document. Save this in a file with the name update.xml.

As you can observe, the XML file written to update data is just like the one which we use to add documents. But the only difference is we use the update attribute of the field.

In our example, we will use the above document and try to update the fields of the document with the id 001.

Suppose the XML document exists in the bin directory of Solr. Since we are updating the index which exists in the core named my_core, you can update using the post tool as follows −

On executing the above command, you will get the following output.

Verification

Visit the homepage of Apache Solr web interface and select the core as my_core. Try to retrieve all the documents by passing the query “:” in the text area q and execute the query. On executing, you can observe that the document is updated.

Deleting the Document

To delete documents from the index of Apache Solr, we need to specify the ID’s of the documents to be deleted between the <delete></delete> tags.

Here, this XML code is used to delete the documents with ID’s 003 and 005. Save this code in a file with the name delete.xml.

If you want to delete the documents from the index which belongs to the core named my_core, then you can post the delete.xml file using the post tool, as shown below.

On executing the above command, you will get the following output.

Verification

Visit the homepage of the of Apache Solr web interface and select the core as my_core. Try to retrieve all the documents by passing the query “:” in the text area q and execute the query. On executing, you can observe that the specified documents are deleted.

Deleting a Field

Sometimes we need to delete documents based on fields other than ID. For example, we may have to delete the documents where the city is Chennai.

In such cases, you need to specify the name and value of the field within the <query></query> tag pair.

Save it as delete_field.xml and perform the delete operation on the core named my_core using the post tool of Solr.

On executing the above command, it produces the following output.

Verification

Visit the homepage of the of Apache Solr web interface and select the core as my_core. Try to retrieve all the documents by passing the query “:” in the text area q and execute the query. On executing, you can observe that the documents containing the specified field value pair are deleted.

Deleting All Documents

Just like deleting a specific field, if you want to delete all the documents from an index, you just need to pass the symbol “:” between the tags <query></ query>, as shown below.

Save it as delete_all.xml and perform the delete operation on the core named my_core using the post tool of Solr.

On executing the above command, it produces the following output.

Verification

Visit the homepage of Apache Solr web interface and select the core as my_core. Try to retrieve all the documents by passing the query “:” in the text area q and execute the query. On executing, you can observe that the documents containing the specified field value pair are deleted.

Retrieving Data

In this chapter, we will discuss how to retrieve data using Java Client API. Suppose we have a .csv document named sample.csv with the following content.

You can index this data under the core named sample_Solr using the post command.

Querying Data

n addition to storing data, Apache Solr also provides the facility of querying it back as and when required. Solr provides certain parameters using which we can query the data stored in it.

In the following table, we have listed down the various query parameters available in Apache Solr.

You can see all these parameters as options to query Apache Solr. Visit the homepage of Apache Solr. On the left-hand side of the page, click on the option Query. Here, you can see the fields for the parameters of a query.

Retrieving from the 2^nd record

We can retrieve the records from the second record by passing 2 as a value to the parameter start, as shown in the following screenshot.

List of the Fields

If we want to have particular fields in the resulted documents, we need to pass the list of the required fields, separated by commas, as a value to the property fl.

In the following example, we are trying to retrieve the fields − id, phone, and first_name.

Faceting in Apache Solr refers to the classification of the search results into various categories. In this chapter, we will discuss the types of faceting available in Apache Solr −

1. Query faceting − It returns the number of documents in the current search results that also match the given query.
2. Date faceting − It returns the number of documents that fall within certain date ranges.

Faceting commands are added to any normal Solr query request, and the faceting counts come back in the same query response.

Faceting Query Example

Using the field faceting, we can retrieve the counts for all terms, or just the top terms in any given field.

As an example, let us consider the following books.csv file that contains data about various books.

Let us post this file into Apache Solr using the post tool.

On executing the above command, all the documents mentioned in the given .csv file will be uploaded into Apache Solr.

Now let us execute a faceted query on the field author with 0 rows on the collection/core my_core.

Open the web UI of Apache Solr and on the left-hand side of the page, check the checkbox facet, as shown in the following screenshot.

On checking the checkbox, you will have three more text fields in order to pass the parameters of the facet search. Now, as parameters of the query, pass the following values.

On executing, it will produce the following result.

Referred from tutorials point