Fusion 4.x: How to Upload Configuration Files for Use by Solr


So you want to upload configuration files Solr can use to properly function and make your application a screaming success.

The cat is happy to hear this. There is nothing the cat likes more than to put things in boxes where they will probably never be found (due solely on their direction).

Of course, there is the old way of doing it:

  • Solr standalone: put the configuration files in the /conf folder of each collection
  • Solr Cloud: Solr automagically uploads the desired configset into Zookeeper on first start-up and you create your collections to leverage that default configuration

Want to know how to modify config files under the control of the all-powerful, all-knowing Zookeeper? You’re in luck. So does the cat.

Since Solr Classic (or Standalone, if you insist) allows you access to the files right on the file system you can do as you please with the only caveat being that you have to restart Solr (or reload the core in question) if you update the configuration of an existing collection.

Solr Cloud:
You can work from a terminal window, and go through the downloading/editing/uploading/editing/uploading/etc. cycle of one or more of the files until you get it right (which, as the cat will tell you, isn’t the best of use of your bathtime), or you can give up and hope one of your colleague/co-workers picks up the mantle you have so carelessly dropped out of sheer frustration.

And then along came Jones Fusion.

Fusion and the Solr Cloud Configuration Files

Fusion is a platform built on top of Solr to make various Solr tasks easier to accomplish. The cat is impatient so a full-blown picture show of Fusion’s capabilities will have to wait for the next Avenger’s movie where it will be the ending easter egg.

Assumption:

– You have installed Fusion 4.2.1 (the latest as of when this blog was posted)
– You have a burning desire to show off your awesome Solr skills without having to use a terminal

Need to download Fusion? Get it from here.

The Short Story

  • Start Fusion
  • Go to an existing Fusion app or create a new one
  • Go to System -> Solr Config and press Add+
  • Enter the file name, path to be use for the file in ZK , and the contents of the file

The Long Story

  • Start Fusion

Go to the installation folder and run

$FUSION/bin/fusion start
  • Go to your existing Fusion app or create a new one

If you have an existing app then good on ya. If you don’t have an app create one. For this example I created an app called add-solr-files. Open you app in either case.

  • Go to System -> Solr Config and press Add+

Go the Solr Config panel by hovering over the System icon and selecting Solr Config.
That will display the list of Solr configuration files (the configset) for your collection (in this case the collection is named add-solr-files).

Press Add and it will give you a new panel where you enter:
– the name of the file (including a path if appropriate)
– the path to the file (if appropriate)
– the contents of the file

  • Enter the file name, path to use in ZK for the file, and the contents of the file
    • The file name is a standard file name. Don’t get fancy. Enter a leading path if required (the Figure below shows the use of a path, but it is not required)
    • The path to the file is one of the paths available from the drop down list
    • the contents of the file can be cut-and-paste from the file if the file already exists
    • Press Save

Figure – Before Saving

Figure – After saving

The file is listed in both Fusion (shown above) and Solr (shown below).

Figure – The Zookeeper list of config files found on the Solr Admin page

Note: This panel will not upload a file to zookeeper. You can only create the file and enter its contents right here (from the UI).

Practical Uses

Let’s assume you want to have multiple synonym files for use by your Solr collection.
For this example, I have created 2 files:

  • synonym-list-1.txt
  • synonym-list-2.txt

You could add the two files from the Solr Config panel, and then edit the managed-schema file (not necessarily a good idea) to add them to either the index analyzer, the query analyzer, or both, of a given field type.

For example (look at the configuration for the query analyzer’s SynonymGraphFilterFactory:

<fieldType class="solr.TextField" multiValued="true" name="text_general" positionIncrementGap="100">
  <analyzer type="index">
    <tokenizer class="solr.StandardTokenizerFactory"/>
    <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt"/>
    <filter class="solr.LowerCaseFilterFactory"/>
  </analyzer>
  <analyzer type="query">
    <tokenizer class="solr.StandardTokenizerFactory"/>
    <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt"/>
    <filter class="solr.SynonymGraphFilterFactory" expand="true" ignoreCase="true" synonyms="synonym-list-1.txt,synonym-list-2.txt"/>
    <filter class="solr.LowerCaseFilterFactory"/>
  </analyzer>
</fieldType>

Bad File or Folder? This is how you can delete it!

Oh, no! You’ve added a file and now you don’t need it for some reason. Misspelled file name, incorrect data, whatever. As easy as it was to add the file through the UI, you cannot delete the file from the UI.

So how do you do it? Sadly, you will have to open up a terminal and do some typing.

Two steps:

  1. Go to the $FUSION/apps/solr-dist/server/scripts/cloud-scripts folder
  2. Run zkcli.sh (notice the command is all lower case) and execute the clear command on the file or folder you want removed
    • Notice that the path does not start with /config, but rather with lwfusion/4.2.1/solr/config (or whatever version of Fusion you happen to be running)

For example:

# cd $FUSION/apps/solr-dist/server/scripts/cloud-scripts
# ./zkcli.sh -zkhost localhost:9983 -cmd clear /lwfusion/[FUSION_VERSION_NUMBER]/solr/configs/[collection name]/[file or folder name]

> cd ./fusion-4.2.1/fusion/4.2.1/apps/solr-dist/server/scripts/cloud-scripts
> ./zkcli.sh -zkhost localhost:9983 -cmd clear /lwfusion/4.2.1/solr/configs/add-solr-files/home

Newsflash

We have released the latest Fusion! Version 4.2.1 which is available for a 30-day trial from Lucidworks

References

Configuring solrconfig.xml

Fusion Documentation

Solr 7.7 Reference Guide

Disclosures

Carlos Valcarcel is a full time employee of LucidWorks, but lives in New York as he prefers hurricanes to earthquakes. Having worked at IBM, Microsoft, and Fast Search and Transfer the only thing he is sure of is that the font editor he wrote on his Atari 800 was the coolest program he has ever written. While questions can be a drag he admits that answers will be harder to give without them.

The cat isn’t real, but then neither are you. Enjoy your search responsibly.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.