Indexer Class

Properties   Methods   Events   Config Settings   Errors  

The Indexer class enables the creation and modification of search indexes.

Syntax

ipworkssearch.Indexer

Remarks

The Indexer class provides a simple way to create and manage Lucene 4.8 search indexes. The class is able to perform indexing operations such as adding, updating, and deleting index documents. It supports indexing operations with a wide variety of analyzers to allow for flexible control over the indexing process.

Preparing the Index

The first step in using the class is to initialize and prepare the search index for modifications. Set the IndexPath property to specify the location of the search index. This path can point to an existing index, or a new directory where the index will be stored. Once the property is set, call the OpenIndex method to load the index.

If IndexPath does not point to a pre-existing search index, the class will create a new one at the specified location before loading it. component.IndexPath = "PATH\\TO\\INDEX"; // Specify the path to the search index component.OpenIndex(); // Load the index

It is important to note that the OpenIndex method will load a snapshot of the index at the time it was opened. Any external changes made to the index afterwards will not be visible to the class until the method is called again. OpenIndex can be called multiple times to reload the index and reflect its latest changes in indexing operations.

Creating Documents

Documents in the index are composed of fields that are managed through the Fields collection. Fields can be added to this collection via the AddDocumentField method. string name = "title"; // The identifier of the field bool store = true; // Determines if the field's contents should be stored in the index int type = (int)TFieldTypes.ftText; // The type of data the field contains string text = "Sample document field"; // The text content of the field int analyzerType = (int)TSearchIndexAnalyzerTypes.atStandard; // Controls the text processing performed on the field's text content component.AddDocumentField(name, store, type, text, analyzerType); // Add the field to the DocumentFields collection After populating the collection with the desired fields, the IndexDocument method can be called to create a new document from these fields and add it to the search index. // Add two fields component.AddDocumentField("field1", true, (int)TFieldTypes.ftText, "Sample content for field 1", (int)TSearchIndexAnalyzerTypes); component.AddDocumentField("field2", true, (int)TFieldTypes.ftText, "Sample content for field 2", (int)TSearchIndexAnalyzerTypes); component.IndexDocument(); // Add a document with these fields to the search index

Deleting Documents

Documents that contain a specific field can be deleted from the index via the Delete method. // Deletes every document with a field that has a name of "field1" and a value of "Sample content for field 1" component.Delete("field1", "Sample content for field 1");

To delete all of the documents from the search index, call the DeleteAll method. // Delete all of the documents from the index component.DeleteAll();

Saving the Index

After modifying the index, call the CloseIndex method to commit the changes that were made and save them to the disk. component.CloseIndex();

Property List

---

The following is the full list of the properties of the class with short descriptions. Click on the links for further details.

Method List

---

The following is the full list of the methods of the class with short descriptions. Click on the links for further details.

Event List

---

The following is the full list of the events fired by the class with short descriptions. Click on the links for further details.

Config Settings

---

The following is a list of config settings for the class with short descriptions. Click on the links for further details.

Analyzer Property (Indexer Class)

The global analyzer for the search index.

Syntax

public int getAnalyzer();


public void setAnalyzer(int analyzer);


Enumerated values:
  public final static int atStandard = 1;
  public final static int atStop = 2;
  public final static int atWhitespace = 3;
  public final static int atSimple = 4;
  public final static int atEmail = 5;

Default Value

1

Remarks

Specifies the global analyzer for the document fields in the search index. Individual document fields may specify their own local analyzer types to override this global one. Please refer to the AnalyzerType field of the DocumentField type for more details.

Possible values are:

Analyzers and Tokenization

Before text is stored in a search index, it goes through an analyzer that breaks the text down into smaller, searchable parts known as tokens. The rules that determine how text is broken down into distinct tokens are defined by the analyzer type.

The way tokens are created directly affects the precision and relevance of search results. For example, if an analyzer does not break down the terms "WORD" and "word" as the same value, searches for one may not bring up results for the other.

Stop Words

Words that are omitted from text processing because of their lack of significant meaning are known as stop words. When the analyzer type is set to atStandard, atStop, or atEmail, stop words will be completely ignored by the analyzer in to improve search performance and relevance.

The full list of stop words that are ignored by these analyzers can be found below:

a, an, and, are, as, at, be, but, by, for, if, in, into, is, it, no, not, of, on,
or, such, that, the, their, then, there, these, they, this, to, was, will, with

This property is not available at design time.

Fields Property (Indexer Class)

A collection of document fields for updating and creating documents.

Syntax

public FieldList getFields();


public void setFields(FieldList fields);

Remarks

Contains a list of document fields that make up the document that gets added to the search index when calling IndexDocument or Update. A document field can be added by accessing this property directly, or by calling the AddDocumentField method.

This property is not available at design time.

Please refer to the Field type for a complete list of fields.

IndexPath Property (Indexer Class)

The path to the search index.

Syntax

public String getIndexPath();


public void setIndexPath(String indexPath);

Default Value

""

Remarks

The filesystem path to the search index. This property must be specified before calling the OpenIndex method.

This value can either be a path to a pre-existing search index in the Lucene 4.8 index format, or a path to a directory where the index will be stored.

Loading or Creating a Search Index

If IndexPath points to a pre-existing search index, it will be loaded by the class when OpenIndex is called. Otherwise, a call to OpenIndex will cause the class to create a new search index and store it in the specified directory.

If the specified directory does not exist, the class will attempt to create a new one at the specified location before creating the search index.

Example: component.IndexPath = "PATH\\TO\\INDEX"; // Specify the path to the search index component.OpenIndex(); // Load the index // ... Perform operations on the index ... component.CloseIndex();

Relative and Absolute Paths

If the path value begins with a / or a drive letter such as D:/, it is considered an absolute path. The class will interpet any other value as a relative path to resolve in relation to the current directory.

This property is not available at design time.

AddDocumentField Method (Indexer Class)

Creates a document field.

Syntax

public void addDocumentField(String name, boolean store, int type, String value, int analyzerType);

Remarks

Creates a document field and adds it to the Fields collection.

Name specifies the name of the document field. This value acts as an identifier for the field and does not have to be unique.

Store determines if the field Text value will be stored in the index. Fields that are not stored in the index can be searched, but their full text contents will not be retrievable in search results. This value is only applicable when the field Type is set to ftText.

Type indicates the type of field that will be created. Its value determines how the field is indexed. Possible values are:

Text specifies the text contents of the field.

AnalyzerType specifies the analyzer for the field. This value determines how the field Text is analyzed and broken down into tokens when it is indexed. This parameter is only applicable when the field Type is set to ftText. Otherwise, this parameter is ignored.

Possible values are:

Analyzers and Tokenization

Before text is stored in a search index, it goes through an analyzer that breaks the text down into smaller, searchable parts known as tokens. The rules that determine how text is broken down into distinct tokens are defined by the analyzer type.

The way tokens are created directly affects the precision and relevance of search results. For example, if an analyzer does not break down the terms "WORD" and "word" as the same value, searches for one may not bring up results for the other.

Example (Create a new document with a single field): // text field component.AddDocumentField("company", true, (int)TFieldTypes.ftText, "nsoftware", (int)SearchIndexAnalyzerTypes.atStandard); // create a new document from the added fields and add it to the index component.IndexDocument();

Example (Create a new document with multiple fields): // text field component.AddDocumentField("author", true, (int)TFieldTypes.ftText, "Generic Design Patterns", (int)SearchIndexAnalyzerTypes.atStandard); // string field component.AddDocumentField("code", true, (int)TFieldTypes.ftString, "GDP2024", (int)SearchIndexAnalyzerTypes.atStandard); // int field component.AddDocumentField("edition", true, (int)TFieldTypes.ftInt32, "2", (int)SearchIndexAnalyzerTypes.atStandard); // create the document and add it to the index // the added document will be comprised of all of the fields in the DocumentFields collection component.IndexDocument();

CloseIndex Method (Indexer Class)

Commits and saves the changes made to the search index.

Syntax

public void closeIndex();

Remarks

Commits the changes made to the search index and saves them to the location specified by IndexPath.

Config Method (Indexer Class)

Sets or retrieves a configuration setting.

Syntax

public String config(String configurationString);

Remarks

Config is a generic method available in every class. It is used to set and retrieve configuration settings for the class.

These settings are similar in functionality to properties, but they are rarely used. In order to avoid "polluting" the property namespace of the class, access to these internal properties is provided through the Config method.

To set a configuration setting named PROPERTY, you must call Config("PROPERTY=VALUE"), where VALUE is the value of the setting expressed as a string. For boolean values, use the strings "True", "False", "0", "1", "Yes", or "No" (case does not matter).

To read (query) the value of a configuration setting, you must call Config("PROPERTY"). The value will be returned as a string.

Delete Method (Indexer Class)

Deletes documents from the search index.

Syntax

public void delete(String fieldName, String fieldValue);

Remarks

Deletes documents that contain a specific field from the search index.

FieldName specifies the name of the field that will be used to determine the documents that will be deleted.

FieldValue specifies the exact text content of the field that will be used to determine the documents that will be deleted. A document must have a field with this exact value and the name specified in the FieldName parameter for it to be affected by this method.

Example: // Delete every document in the index that has a field named "content" with a value of "Sample data" component.Delete("content", "Sample data");

DeleteAll Method (Indexer Class)

Deletes all of the documents from the search index.

Syntax

public void deleteAll();

Remarks

Deletes all of the documents from the search index.

IndexDocument Method (Indexer Class)

Creates a new document and adds it to the search index.

Syntax

public void indexDocument();

Remarks

Creates a new document and adds it to the search index. Its document fields will be the same as the ones in the Fields collection when this method is called.

Example (Create a new document with a single field): // text field component.AddDocumentField("company", true, (int)TFieldTypes.ftText, "nsoftware", (int)SearchIndexAnalyzerTypes.atStandard); // create a new document from the added fields and add it to the index component.IndexDocument();

Example (Create a new document with multiple fields): // text field component.AddDocumentField("author", true, (int)TFieldTypes.ftText, "Generic Design Patterns", (int)SearchIndexAnalyzerTypes.atStandard); // string field component.AddDocumentField("code", true, (int)TFieldTypes.ftString, "GDP2024", (int)SearchIndexAnalyzerTypes.atStandard); // int field component.AddDocumentField("edition", true, (int)TFieldTypes.ftInt32, "2", (int)SearchIndexAnalyzerTypes.atStandard); // create the document and add it to the index // the added document will be comprised of all of the fields in the DocumentFields collection component.IndexDocument();

OpenIndex Method (Indexer Class)

Opens an existing search index or creates a new one.

Syntax

public void openIndex();

Remarks

Loads the search index located in IndexPath and prepares it for modifications. This method should be called at least once before making changes to the search index via IndexDocument, Delete, DeleteAll, or Update. To save the changes and close the index, call CloseIndex instead.

Loading or Creating a Search Index

If IndexPath points to a pre-existing search index, it will be loaded by the class when OpenIndex is called. Otherwise, a call to OpenIndex will cause the class to create a new search index and store it in the specified directory.

If the specified directory does not exist, the class will attempt to create a new one at the specified location before creating the search index.

Example: component.IndexPath = "PATH\\TO\\INDEX"; // Specify the path to the search index component.OpenIndex(); // Load the index // ... Perform operations on the index ... component.CloseIndex();

Index Updates

When this method is called, the component will load a snapshot of the index at the time it was opened. Any external changes made to the index afterwards will not be reflected in subsequent indexing operations until this method is called again.

This method can be called multiple times to reload the index and reflect its latest changes in indexing operations.

Example:

component.OpenIndex(); // Load the search index // ... Perform operations on the index ... component.OpenIndex(); // Reload the index to reflect any new updates

Reset Method (Indexer Class)

Resets the class.

Syntax

public void reset();

Remarks

Resets the class's properties to their default values.

Update Method (Indexer Class)

Updates documents in the search index.

Syntax

public void update(String fieldName, String fieldValue);

Remarks

Updates documents in the search index. When this method is called, every document in the index that contains the specified field will be deleted. Once deleted, a new document with the fields in the Fields collection will be created and added to the index to replace the deleted documents. This is effectively the same as calling Delete followed by IndexDocument.

FieldName specifies the name of the field that will be used to identify the documents that will be updated.

FieldValue specifies the exact text content of the field that will be used to identify the documents that will be updated. A document must have a field with this exact value and the name specified in the FieldName parameter for it to be affected by this method.

Example (Update a document that has incorrect fields) // Assuming the index has a document with a field named "data" // and a value of "incorrect data" // Add the fields we want for the updated document component.AddDocumentField("data", true, 1, "corrected data"); component.AddDocumentField("company", "nsoftware"); // Delete the incorrect docuement and replace it with the new one // // If the index contains multiple documents with this field, they // will all be deleted, but only one document will be created to // replace them component.Update("data", "incorrect data");

Error Event (Indexer Class)

Fires to provide information about errors during indexing.

Syntax

public class DefaultIndexerEventListener implements IndexerEventListener {
  ...
  public void error(IndexerErrorEvent e) {}
  ...
}

public class IndexerErrorEvent {
  public int errorCode;
  public String description;
}

Remarks

Fires in case of exceptional conditions during indexing. Normally, the class will raise an exception instead.

ErrorCode contains an error code and Description contains a textual description of the error. For a list of valid error codes and their descriptions, please refer to the Error Codes section.

Log Event (Indexer Class)

This event fires once for each log message.

Syntax

public class DefaultIndexerEventListener implements IndexerEventListener {
  ...
  public void log(IndexerLogEvent e) {}
  ...
}

public class IndexerLogEvent {
  public int logLevel;
  public String message;
  public String logType;
}

Remarks

This event fires once for each log message generated by the class. The verbosity is controlled by the LogLevel setting.

LogLevel indicates the level of detail of the log message. Possible values are:

Message is the log entry.

LogType identifies the type of log entry.

Field Type

Holds information about a document field.

Remarks

This is used to access the data associated with a single document field.

Fields

Constructors

public Field();

public Field( name,  store,  type,  inputFile,  inputText,  inputStream,  analyzerType);

Config Settings (Indexer Class)

The class accepts one or more of the following configuration settings. Configuration settings are similar in functionality to properties, but they are rarely used. In order to avoid "polluting" the property namespace of the class, access to these internal properties is provided through the Config method.

Indexer Config Settings

Base Config Settings

Trappable Errors (Indexer Class)