mirror of
https://github.com/Alfresco/SearchServices.git
synced 2026-09-16 18:12:56 +00:00
Merge master into SEARCH-1599_ShardSettingsAtStartup.
This commit is contained in:
Tom Page
@@ -0,0 +1,165 @@
|
||||
GNU LESSER GENERAL PUBLIC LICENSE
|
||||
Version 3, 29 June 2007
|
||||
|
||||
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
|
||||
Everyone is permitted to copy and distribute verbatim copies
|
||||
of this license document, but changing it is not allowed.
|
||||
|
||||
|
||||
This version of the GNU Lesser General Public License incorporates
|
||||
the terms and conditions of version 3 of the GNU General Public
|
||||
License, supplemented by the additional permissions listed below.
|
||||
|
||||
0. Additional Definitions.
|
||||
|
||||
As used herein, "this License" refers to version 3 of the GNU Lesser
|
||||
General Public License, and the "GNU GPL" refers to version 3 of the GNU
|
||||
General Public License.
|
||||
|
||||
"The Library" refers to a covered work governed by this License,
|
||||
other than an Application or a Combined Work as defined below.
|
||||
|
||||
An "Application" is any work that makes use of an interface provided
|
||||
by the Library, but which is not otherwise based on the Library.
|
||||
Defining a subclass of a class defined by the Library is deemed a mode
|
||||
of using an interface provided by the Library.
|
||||
|
||||
A "Combined Work" is a work produced by combining or linking an
|
||||
Application with the Library. The particular version of the Library
|
||||
with which the Combined Work was made is also called the "Linked
|
||||
Version".
|
||||
|
||||
The "Minimal Corresponding Source" for a Combined Work means the
|
||||
Corresponding Source for the Combined Work, excluding any source code
|
||||
for portions of the Combined Work that, considered in isolation, are
|
||||
based on the Application, and not on the Linked Version.
|
||||
|
||||
The "Corresponding Application Code" for a Combined Work means the
|
||||
object code and/or source code for the Application, including any data
|
||||
and utility programs needed for reproducing the Combined Work from the
|
||||
Application, but excluding the System Libraries of the Combined Work.
|
||||
|
||||
1. Exception to Section 3 of the GNU GPL.
|
||||
|
||||
You may convey a covered work under sections 3 and 4 of this License
|
||||
without being bound by section 3 of the GNU GPL.
|
||||
|
||||
2. Conveying Modified Versions.
|
||||
|
||||
If you modify a copy of the Library, and, in your modifications, a
|
||||
facility refers to a function or data to be supplied by an Application
|
||||
that uses the facility (other than as an argument passed when the
|
||||
facility is invoked), then you may convey a copy of the modified
|
||||
version:
|
||||
|
||||
a) under this License, provided that you make a good faith effort to
|
||||
ensure that, in the event an Application does not supply the
|
||||
function or data, the facility still operates, and performs
|
||||
whatever part of its purpose remains meaningful, or
|
||||
|
||||
b) under the GNU GPL, with none of the additional permissions of
|
||||
this License applicable to that copy.
|
||||
|
||||
3. Object Code Incorporating Material from Library Header Files.
|
||||
|
||||
The object code form of an Application may incorporate material from
|
||||
a header file that is part of the Library. You may convey such object
|
||||
code under terms of your choice, provided that, if the incorporated
|
||||
material is not limited to numerical parameters, data structure
|
||||
layouts and accessors, or small macros, inline functions and templates
|
||||
(ten or fewer lines in length), you do both of the following:
|
||||
|
||||
a) Give prominent notice with each copy of the object code that the
|
||||
Library is used in it and that the Library and its use are
|
||||
covered by this License.
|
||||
|
||||
b) Accompany the object code with a copy of the GNU GPL and this license
|
||||
document.
|
||||
|
||||
4. Combined Works.
|
||||
|
||||
You may convey a Combined Work under terms of your choice that,
|
||||
taken together, effectively do not restrict modification of the
|
||||
portions of the Library contained in the Combined Work and reverse
|
||||
engineering for debugging such modifications, if you also do each of
|
||||
the following:
|
||||
|
||||
a) Give prominent notice with each copy of the Combined Work that
|
||||
the Library is used in it and that the Library and its use are
|
||||
covered by this License.
|
||||
|
||||
b) Accompany the Combined Work with a copy of the GNU GPL and this license
|
||||
document.
|
||||
|
||||
c) For a Combined Work that displays copyright notices during
|
||||
execution, include the copyright notice for the Library among
|
||||
these notices, as well as a reference directing the user to the
|
||||
copies of the GNU GPL and this license document.
|
||||
|
||||
d) Do one of the following:
|
||||
|
||||
0) Convey the Minimal Corresponding Source under the terms of this
|
||||
License, and the Corresponding Application Code in a form
|
||||
suitable for, and under terms that permit, the user to
|
||||
recombine or relink the Application with a modified version of
|
||||
the Linked Version to produce a modified Combined Work, in the
|
||||
manner specified by section 6 of the GNU GPL for conveying
|
||||
Corresponding Source.
|
||||
|
||||
1) Use a suitable shared library mechanism for linking with the
|
||||
Library. A suitable mechanism is one that (a) uses at run time
|
||||
a copy of the Library already present on the user's computer
|
||||
system, and (b) will operate properly with a modified version
|
||||
of the Library that is interface-compatible with the Linked
|
||||
Version.
|
||||
|
||||
e) Provide Installation Information, but only if you would otherwise
|
||||
be required to provide such information under section 6 of the
|
||||
GNU GPL, and only to the extent that such information is
|
||||
necessary to install and execute a modified version of the
|
||||
Combined Work produced by recombining or relinking the
|
||||
Application with a modified version of the Linked Version. (If
|
||||
you use option 4d0, the Installation Information must accompany
|
||||
the Minimal Corresponding Source and Corresponding Application
|
||||
Code. If you use option 4d1, you must provide the Installation
|
||||
Information in the manner specified by section 6 of the GNU GPL
|
||||
for conveying Corresponding Source.)
|
||||
|
||||
5. Combined Libraries.
|
||||
|
||||
You may place library facilities that are a work based on the
|
||||
Library side by side in a single library together with other library
|
||||
facilities that are not Applications and are not covered by this
|
||||
License, and convey such a combined library under terms of your
|
||||
choice, if you do both of the following:
|
||||
|
||||
a) Accompany the combined library with a copy of the same work based
|
||||
on the Library, uncombined with any other library facilities,
|
||||
conveyed under the terms of this License.
|
||||
|
||||
b) Give prominent notice with the combined library that part of it
|
||||
is a work based on the Library, and explaining where to find the
|
||||
accompanying uncombined form of the same work.
|
||||
|
||||
6. Revised Versions of the GNU Lesser General Public License.
|
||||
|
||||
The Free Software Foundation may publish revised and/or new versions
|
||||
of the GNU Lesser General Public License from time to time. Such new
|
||||
versions will be similar in spirit to the present version, but may
|
||||
differ in detail to address new problems or concerns.
|
||||
|
||||
Each version is given a distinguishing version number. If the
|
||||
Library as you received it specifies that a certain numbered version
|
||||
of the GNU Lesser General Public License "or any later version"
|
||||
applies to it, you have the option of following the terms and
|
||||
conditions either of that published version or of any later version
|
||||
published by the Free Software Foundation. If the Library as you
|
||||
received it does not specify a version number of the GNU Lesser
|
||||
General Public License, you may choose any version of the GNU Lesser
|
||||
General Public License ever published by the Free Software Foundation.
|
||||
|
||||
If the Library as you received it specifies that a proxy can decide
|
||||
whether future versions of the GNU Lesser General Public License shall
|
||||
apply, that proxy's public statement of acceptance of any version is
|
||||
permanent authorization for you to choose that version for the
|
||||
Library.
|
||||
@@ -0,0 +1,43 @@
|
||||
## Alfresco Search Services and Insight Engine
|
||||
|
||||
This repository includes the source code for Alfresco Search Services and Insight Engine products.
|
||||
|
||||
**Alfresco Search Services** provides searching capabilities to Alfresco Content Services by leveraging [Apache SOLR](https://lucene.apache.org/solr/) core features. This product is used for both Enterprise and Community releases of Alfresco Content Services and it lives in [GitHub](https://github.com/Alfresco/SearchServices).
|
||||
|
||||
The official documentation for this product can be found at [Alfresco Search Services](https://docs.alfresco.com/search-community/concepts/search-home.html).
|
||||
|
||||
**Insight Engine** works together with *Alfresco Search Services* to provide extended capabilities to Alfresco Content Services (like a JDBC connector and Zeppelin integration). This product is licensed only in Enterprise mode, so the source code lives in our private *GitLab* and it's only available for Enterprise customers.
|
||||
|
||||
The official documentation for this product can be found at [Alfresco Search and Insight Engine](https://docs.alfresco.com/sie/concepts/Search-Insight-Engine-overview.html).
|
||||
|
||||
|
||||
### Alfresco Search Services
|
||||
|
||||
Getting the code using a Git client.
|
||||
|
||||
```bash
|
||||
$ git clone https://github.com/Alfresco/SearchServices.git
|
||||
```
|
||||
|
||||
Build the project using Maven.
|
||||
|
||||
```bash
|
||||
$ cd SearchServices/search-services
|
||||
$ mvn clean install
|
||||
```
|
||||
|
||||
All the resources required to run Alfresco Search Services will be available under `search-services/packaging/target` folder, including the distribution ZIP for local installations.
|
||||
|
||||
The Docker Image source code is available at `search-services/packaging/src/docker`. The building for the Docker Image is available in the public repository of Alfresco at Docker Hub:
|
||||
|
||||
[https://hub.docker.com/r/alfresco/alfresco-search-services](https://hub.docker.com/r/alfresco/alfresco-search-services)
|
||||
|
||||
*Note* The root `pom.xml` living in this folder is used for packaging Search Services and Insight Engine together. This file includes URLs not available for Community users, but it allows the `search-services` module without accessing to these resources.
|
||||
|
||||
More details are available at [search-services](/search-services) folder.
|
||||
|
||||
### Insight Engine
|
||||
|
||||
**Following resources will not be available for Community users**
|
||||
|
||||
More details are available at [insight-engine](/insight-engine) folder.
|
||||
@@ -0,0 +1,262 @@
|
||||
## DocRouters
|
||||
|
||||
|
||||
|
||||
### Introduction
|
||||
When an index grows too large to be stored on a single search server, it can be distributed across multiple search servers. This is known as _sharding_. The distributed/sharded index can then be searched using Alfresco/Solr's distributed search capabilities.
|
||||
Search Services can use any of these four different methods for routing documents and ACLs to shards.
|
||||
|
||||
The following diagram illustrates the DocRouter class hierarchy; the DocRouter interface, which declares the main contract, and all concrete implementors.
|
||||
|
||||
|
||||
|
||||
The light gray color on the _DateQuarterRouter_ is because it is not currently used.
|
||||
|
||||
### How a DocRouter is choosen
|
||||
To use a specific sharding method, when creating a shard we must add the required configuration properties in the solrcore.properties.
|
||||
There's a first required property (called _"shard_method"_) which is the mnemonic code associated to the sharding method, then additional properties could be needed, depending on the method.
|
||||
|
||||
As you can see from the class diagram above, the _DocRouter_ instance creation is a responsibility of the _DocRouterFactory_.
|
||||
Since we are interested (see the _DocRouter_ interface) in distributing ACLs and Nodes, the _MetadataTracker_ and the _ACLTracker_ read the shard_method in the configuration and create/use the corresponding _DocRouter_ instance.
|
||||
If the sharding method doesn't correspond to a valid option (e.g. invalid or unknown mnemonic code) then the system will fallback to _DBID_ routing (_DBDIDRouter_).
|
||||
|
||||
### Available document routers
|
||||
|
||||
The following table lists the available sharding methods and the associated mnemonic codes (i.e. the value of the "shard_method" attribute we need to configure in solrcore.properties).
|
||||
|
||||
| Name | Code |
|
||||
| -----|---------|
|
||||
|ACL (MOD) |MOD_ACL_ID|
|
||||
|ACL (HASH) |ACL_ID|
|
||||
|DBID (HASH)|DB_ID|
|
||||
|DBID (RANGE)|DB_ID_RANGE|
|
||||
|Month|DATE|
|
||||
|Metadata|PROPERTY|
|
||||
|Last registered indexing Shard|LRIS or LAST_REGISTERED_INDEXING_SHARD|
|
||||
|Explicit Shard ID (fallback on DBID)|EXPLICIT_ID or EXPLICIT_ID_FALLBACK_DBID|
|
||||
|Explicit Shard ID (fallback on LRIS)|EXPLICIT_ID_FALLBACK_LRIS|
|
||||
|
||||
##### ACL (MOD_ACL_ID)
|
||||
Nodes and access control lists are grouped by their ACL identifier. This places the nodes together with all the access control information required to determine the access to a node in the same shard. Both the nodes and access control information are sharded.
|
||||
The overall index size will be smaller than other methods as the ACL index information is not duplicated in every shard. Also, the ACL count is usually much smaller than the node count.
|
||||
This method is beneficial if you have lots of ACLs and the documents are evenly distributed over those ACLs.
|
||||
> The node distribution may be uneven as it depends how many nodes share ACLs.
|
||||
|
||||
In order to use this method, the following properties are required:
|
||||
|
||||
```
|
||||
shard.method=MOD_ACL_ID
|
||||
shard.instance=<shard.instance>
|
||||
shard.count=<shard.count>
|
||||
```
|
||||
|
||||
where
|
||||
|
||||
* shard.method is the mnemonic code associated with this router
|
||||
* shard.instance is the shard identifier, which must be unique across the cluster
|
||||
* shard.count is the total number of the shards composing the cluster
|
||||
|
||||
##### ACL (ACL_ID)
|
||||
The sharding method is similar to the previous one, except that the murmur hash of the ACL ID is used in preference to its modulus.
|
||||
This gives better distribution of ACLs over shards. The distribution of documents over ACLs is not affected and can still be skewed.
|
||||
|
||||
Apart the different mnemonic code, this option has the same configuration attributes of the MOD_ACL_ID.
|
||||
|
||||
##### DBID (DB_ID)
|
||||
This is the default sharding method used in Solr 6. As mentioned above, this is also the method used in case an invalid or unknown value is detected in the shard.method property.
|
||||
Nodes are evenly distributed over the shards at random based on the murmur hash of the DBID. The access control information is duplicated in each shard. The distribution of nodes over each shard is very even and shards grow at the same rate.
|
||||
|
||||
In order to use this method, the following properties are required:
|
||||
|
||||
```
|
||||
shard.method=DB_ID
|
||||
shard.instance=<shard.instance>
|
||||
shard.count=<shard.count>
|
||||
```
|
||||
|
||||
where
|
||||
|
||||
* shard.method is the mnemonic code associated with this router
|
||||
* shard.instance is the shard identifier, which must be unique across the cluster
|
||||
* shard.count is the total number of the shards composing the cluster
|
||||
|
||||
|
||||
##### DBID Range (DB_ID_RANGE)
|
||||
This sharding method routes documents within specific DBID ranges to specific shards. It adds new shards to the cluster without requiring a reindex. As consequence of that, the total number of shards (i.e. the shard.count attribute) mustn't be known in advance.
|
||||
For each shard, you specify the range of DBIDs to be included. As your repository grows you can add shards.
|
||||
|
||||
In order to use this method, the following properties are required:
|
||||
|
||||
```
|
||||
shard.method=DB_ID_RANGE
|
||||
shard.instance=<shard.instance>
|
||||
shard.range=0-200000
|
||||
```
|
||||
|
||||
where
|
||||
|
||||
* shard.method is the mnemonic code associated with this router
|
||||
* shard.instance is the shard identifier, which must be unique across the cluster
|
||||
* shard.range is the range of DBID that must belong to the shard
|
||||
|
||||
|
||||
##### Month (DATE)
|
||||
This method assigns dates sequentially through shards based on the month.
|
||||
For example: If there are 12 shards, each month would be assigned sequentially to each shard, wrapping round and starting again for each year.
|
||||
The non-random assignment facilitates easier shard management - dropping shards or scaling out replication for some date range. Typical ageing strategies could be based on the created date or destruction date.
|
||||
Each shard contains copies of all the ACL information, so this information is replicated in each shard. However, if the property is not present on a node, sharding falls back to the DBID murmur hash to randomly distribute these nodes.
|
||||
|
||||
In order to use this method, the following properties are required:
|
||||
|
||||
```
|
||||
shard.method=DATE
|
||||
shard.key=cm:created
|
||||
shard.instance=<shard.instance>
|
||||
shard.count=<shard.count>
|
||||
(optional)shard.date.grouping=4
|
||||
```
|
||||
|
||||
where
|
||||
|
||||
* shard.method is the mnemonic code associated with this router
|
||||
* shard.key is the node property which contains the date used by this sharding method
|
||||
* shard.instance is the shard identifier, which must be unique across the cluster
|
||||
* shard.range is the range of DBID that must belong to the shard
|
||||
* shard.date.grouping is an optional property used for grouping months together
|
||||
* shard.count is the total number of the shards composing the cluster
|
||||
|
||||
##### Metadata (PROPERTY)
|
||||
This method hashes the value of a node property and it uses the hash for targeting a given shard. All nodes with the same property value will be assigned to the same shard. Each shard will duplicate all the ACL information.
|
||||
Only properties of type d:text, d:date and d:datetime can be used. For example, the recipient of an email, the creator of a node, some custom field set by a rule, or by the domain of an email recipient. The keys are randomly distributed over the shards using murmur hash.
|
||||
|
||||
In order to use this method, the following properties are required:
|
||||
|
||||
```
|
||||
shard.method=PROPERTY
|
||||
shard.key=cm:creator
|
||||
shard.instance=<shard.instance>
|
||||
shard.count=<shard.count>
|
||||
(optional)shard.regex=^\d{4}
|
||||
```
|
||||
|
||||
where
|
||||
|
||||
* shard.method is the mnemonic code associated with this router
|
||||
* shard.key is the node property whose value will be used for determining the target shard
|
||||
* shard.instance is the shard identifier, which must be unique across the cluster
|
||||
* shard.count is the total number of the shards composing the cluster
|
||||
* shard.regex is an optional property which applies a regex on the node property value, before hashing it.
|
||||
|
||||
|
||||
##### Last Registered Indexing Shard (LRIS)
|
||||
This method uses, as the name suggests, the last indexing shard which subscribed to the Shard Registry.
|
||||
Like the _DB_ID_RANGE_ strategy, it's possible to add new shards to the cluster without requiring a reindex. And as consequence of that, the total number of shards (i.e. the _shard.count_ attribute) isn't needed with this sharding strategy.
|
||||
At indexing time, when the _MetadataTracker_ periodically asks for transactions and nodes, Alfresco repository creates the nodes instances and, using the associated transaction timestamp, asks to the _ShardRegistry_ which is the target Shard which should index/own the node.
|
||||
This _DocRouter_ will then use that information in order to understand if the incoming node belongs to it or not.
|
||||
|
||||
In order to use this method, the following properties are required:
|
||||
|
||||
```
|
||||
shard.method=LRIS
|
||||
shard.instance=<shard.instance>
|
||||
```
|
||||
|
||||
where
|
||||
|
||||
* shard.method is the mnemonic code associated with this router
|
||||
* shard.instance is the shard identifier, which must be unique across the cluster
|
||||
|
||||
###### ShardRegistry and ShardSubscription
|
||||
On SearchServices side, the _MetadataTracker_ periodically communicates the hosting Shard state to Alfresco repository.
|
||||
Alfresco, on its side, maintains a registry which provides near-real-time information about the Solr cluster (e.g. shards, shards state, topology). The very first time a shard subscribes the ShardRegistry creates a subscription for it.
|
||||
A ShardSubscription is a simple entity which includes
|
||||
|
||||
* the shard id
|
||||
* the subscription timestamp
|
||||
* the owning core name (i.e. the name of the core the shards belongs to)
|
||||
|
||||
Shard subscriptions are
|
||||
|
||||
* collected and sorted by subscription timestamp descendent order (i.e. the first entry is the last subscriber)
|
||||
* persisted on the database
|
||||
|
||||
The persistence is needed because in case of restart Alfresco needs to restore the subscriptions map and the correct shard topology. If no persistence definition exists, the map is empty and the registry will wait for the first subscriber.
|
||||
|
||||
##### Explicit Shard ID with fallback on DBID (EXPLICIT_ID, EXPLICIT_ID_FALLBACK_DBID)
|
||||
Nodes are routed to shards accordingly to a value of a node property which is supposed to contain the target shard instance identifier.
|
||||
Note that this is different from the "PROPERTY" doc router seen above because no hashing occurs on the node property:
|
||||
|
||||
* it must be numeric (it can be also a string with a numeric content)
|
||||
* it must contain the target shard instance identifier
|
||||
|
||||
If the system is not able to determine a numeric id from the configured node property, then the doc router uses the DB_ID as a fallback strategy.
|
||||
Although the primary sharding method (the numeric node property) results in an "elastic sharding" approach (we don't need to know the total number of shards in advance), when this doc router is
|
||||
used, we still need to know the _shard.count_ attribute, because it is a required parameter for the fallback router (DB_ID)
|
||||
|
||||
|
||||
|
||||
In order to use this method, the following properties are required:
|
||||
|
||||
```
|
||||
shard.method=EXPLICIT_ID_FALLBACK_DBID
|
||||
shard.key=cm:targetShardInstance
|
||||
shard.instance=<shard.instance>
|
||||
shard.count=<shard.count>
|
||||
```
|
||||
|
||||
where
|
||||
|
||||
* shard.method is the mnemonic code associated with this router. Note that also _EXPLICIT_ID_ can be used
|
||||
* shard.key is the node property whose value will be used for determining the target shard
|
||||
* shard.instance is the shard identifier, which must be unique across the cluster
|
||||
* shard.count is the total number of the shards composing the cluster (required by the DBID fallback strategy)
|
||||
|
||||
##### Explicit Shard ID with fallback on LRIS (EXPLICIT_ID_FALLBACK_LRIS)
|
||||
This method still belongs to the "explicit" id family, where nodes are routed to shards accordingly to a value of a node property which is supposed to contain the target shard instance identifier.
|
||||
So it is very similar to the previous one. The important difference resides in the fallback strategy: in case the target shard cannot be determined used the supplied node property, the
|
||||
"Last Registered Indexing Shard" is used as fallback.
|
||||
Since both strategies (the primary and the fallback) are "elastic", this sharding method doesn't have to know the total number of shards: new shards can be added without any reindexing.
|
||||
|
||||
|
||||
|
||||
Let's see what happens when a member in the cluster (a Solr or Alfresco node) is restarted.
|
||||
In all the examples below we have:
|
||||
|
||||
* an Alfresco instance, which maintains the persisted subscriptions map
|
||||
* two shards (i.e. Solr nodes): S1 subscribed at t1, S2 subscribed at t2
|
||||
|
||||
**Example #1: One shard is restarted**
|
||||
|
||||
|
||||
|
||||
In order to better explain the system behaviour we'll enumerate two sub-scenarios:
|
||||
|
||||
* **Indexed data is not lost**: after restarting, the node will find itself already registered in the subscription map. If it is the last indexing registered shard Alfresco will continue to use it as target indexing shard for the new incoming data
|
||||
* **Indexed data is lost**: after restarting, the node will still find its subscription in the ShardRegistry. That means on Alfresco side, the Shard is still associated with the proper transaction range. It will request data (e.g. txn, nodes) from Alfresco and the doc router will accept only those nodes that belong to a transaction within that range.
|
||||
|
||||
**Example #2: Two shards are restarted**
|
||||
|
||||
|
||||
|
||||
Shard subscriptions are persisted on Alfresco side, so this scenario is similar to the previous one: regardless the shards have the indexed data or not after a restart, their corresponding subscription are still available on the ShardRegistry. That means the doc router will be aware about the range of transactions that must be (re)indexed in the two nodes.
|
||||
|
||||
**Example #3: Alfresco is restarted**
|
||||
|
||||
|
||||
|
||||
The Shard Subscriptions Map is persisted on the database. That means its whole definition can survive to a system crash.
|
||||
After restarting, subscriptions are restored from the database so each shard will be aware about the range of transactions it belongs to.
|
||||
|
||||
In order to use this method, the following properties are required:
|
||||
|
||||
```
|
||||
shard.method=EXPLICIT_ID_FALLBACK_DBID
|
||||
shard.key=cm:targetShardInstance
|
||||
shard.instance=<shard.instance>
|
||||
```
|
||||
|
||||
where
|
||||
|
||||
* shard.method is the mnemonic code associated with this router. Note that also _EXPLICIT_ID_ can be used
|
||||
* shard.key is the node property whose value will be used for determining the target shard
|
||||
* shard.instance is the shard identifier, which must be unique across the cluster
|
||||
BIN
Binary file not shown.
|
After Width: | Height: | Size: 214 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 185 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 31 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 20 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 207 KiB |
BIN
Binary file not shown.
|
After Width: | Height: | Size: 209 KiB |
@@ -87,7 +87,7 @@
|
||||
<dependency>
|
||||
<groupId>jaxen</groupId>
|
||||
<artifactId>jaxen</artifactId>
|
||||
<version>1.1.6</version>
|
||||
<version>1.2.0</version>
|
||||
</dependency>
|
||||
|
||||
<!-- Test dependencies -->
|
||||
|
||||
+7
-1
@@ -60,9 +60,15 @@ public class DocRouterFactory
|
||||
case PROPERTY:
|
||||
log.info("Sharding via PROPERTY");
|
||||
return new PropertyRouter(properties.getProperty("shard.regex", ""));
|
||||
case LAST_REGISTERED_INDEXING_SHARD:
|
||||
log.info("Sharding via LAST_REGISTERED_INDEXING_SHARD");
|
||||
return new LastRegisteredShardRouter();
|
||||
case EXPLICIT_ID_FALLBACK_LRIS:
|
||||
log.info("Sharding via EXPLICIT_ID_FALLBACK_LRIS");
|
||||
return new ExplicitRouter(new LastRegisteredShardRouter());
|
||||
case EXPLICIT_ID:
|
||||
log.info("Sharding via EXPLICIT_ID");
|
||||
return new ExplicitRouter();
|
||||
return new ExplicitRouter(new DBIDRouter());
|
||||
default:
|
||||
log.info("Sharding via DB_ID (default)");
|
||||
return new DBIDRouter();
|
||||
|
||||
+22
-8
@@ -1,3 +1,21 @@
|
||||
/*
|
||||
* Copyright (C) 2005-2019 Alfresco Software Limited.
|
||||
*
|
||||
* This file is part of Alfresco
|
||||
*
|
||||
* Alfresco is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Lesser General Public License as published by
|
||||
* the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* Alfresco is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Lesser General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Lesser General Public License
|
||||
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
|
||||
*/
|
||||
package org.alfresco.solr.tracker;
|
||||
|
||||
import org.alfresco.solr.client.Acl;
|
||||
@@ -11,9 +29,10 @@ import org.slf4j.LoggerFactory;
|
||||
public class ExplicitRouter implements DocRouter {
|
||||
|
||||
protected final static Logger log = LoggerFactory.getLogger(ExplicitRouter.class);
|
||||
private final DBIDRouter fallback = new DBIDRouter();
|
||||
private final DocRouter fallbackRouter;
|
||||
|
||||
public ExplicitRouter() {
|
||||
public ExplicitRouter(DocRouter fallbackRouter) {
|
||||
this.fallbackRouter = fallbackRouter;
|
||||
}
|
||||
|
||||
@Override
|
||||
@@ -25,11 +44,6 @@ public class ExplicitRouter implements DocRouter {
|
||||
@Override
|
||||
public boolean routeNode(int shardCount, int shardInstance, Node node) {
|
||||
|
||||
if(shardCount <= 1)
|
||||
{
|
||||
return true;
|
||||
}
|
||||
|
||||
String shardBy = node.getShardPropertyValue();
|
||||
|
||||
if (shardBy != null && !shardBy.isEmpty())
|
||||
@@ -59,6 +73,6 @@ public class ExplicitRouter implements DocRouter {
|
||||
{
|
||||
log.debug("Shard "+shardInstance+" falling back to DBID routing for node "+node.getNodeRef());
|
||||
}
|
||||
return fallback.routeNode(shardCount, shardInstance, node);
|
||||
return fallbackRouter.routeNode(shardCount, shardInstance, node);
|
||||
}
|
||||
}
|
||||
|
||||
+62
@@ -0,0 +1,62 @@
|
||||
/*
|
||||
* Copyright (C) 2005-2019 Alfresco Software Limited.
|
||||
*
|
||||
* This file is part of Alfresco
|
||||
*
|
||||
* Alfresco is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Lesser General Public License as published by
|
||||
* the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* Alfresco is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Lesser General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Lesser General Public License
|
||||
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
|
||||
*/
|
||||
package org.alfresco.solr.tracker;
|
||||
|
||||
import org.alfresco.solr.client.Acl;
|
||||
import org.alfresco.solr.client.Node;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
/**
|
||||
* Routes a document only if the explicitShardId matches the provided shardId
|
||||
*
|
||||
* @author Elia
|
||||
*/
|
||||
public class LastRegisteredShardRouter implements DocRouter
|
||||
{
|
||||
|
||||
protected final static Logger log = LoggerFactory.getLogger(ExplicitRouter.class);
|
||||
|
||||
public LastRegisteredShardRouter()
|
||||
{
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean routeAcl(int shardCount, int shardInstance, Acl acl)
|
||||
{
|
||||
//all acls go to all shards.
|
||||
return true;
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean routeNode(int shardCount, int shardInstance, Node node)
|
||||
{
|
||||
|
||||
Integer explicitShardId = node.getExplicitShardId();
|
||||
|
||||
if (explicitShardId == null)
|
||||
{
|
||||
log.error("explicitShardId is not set for node " + node.getNodeRef());
|
||||
return false;
|
||||
}
|
||||
|
||||
return explicitShardId.equals(shardInstance);
|
||||
|
||||
}
|
||||
}
|
||||
+9
@@ -20,6 +20,7 @@ package org.alfresco.solr.tracker;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.util.ArrayList;
|
||||
import java.util.HashMap;
|
||||
import java.util.HashSet;
|
||||
import java.util.LinkedHashSet;
|
||||
import java.util.List;
|
||||
@@ -221,6 +222,9 @@ public class MetadataTracker extends AbstractTracker implements Tracker
|
||||
.map(Tracker::getTrackerState)
|
||||
.orElse(transactionsTrackerState);
|
||||
|
||||
HashMap<String, String> propertyBag = new HashMap<>();
|
||||
propertyBag.put("coreName", coreName);
|
||||
|
||||
return ShardStateBuilder.shardState()
|
||||
.withMaster(isMaster)
|
||||
.withLastUpdated(System.currentTimeMillis())
|
||||
@@ -240,6 +244,7 @@ public class MetadataTracker extends AbstractTracker implements Tracker
|
||||
.withTemplate(shardTemplate)
|
||||
.withHasContent(transformContent)
|
||||
.withShardMethod(ShardMethodEnum.getShardMethod(shardMethod))
|
||||
.withPropertyBag(propertyBag)
|
||||
.endFloc()
|
||||
.endShard()
|
||||
.endShardInstance()
|
||||
@@ -359,6 +364,7 @@ public class MetadataTracker extends AbstractTracker implements Tracker
|
||||
gnp.setStoreProtocol(storeRef.getProtocol());
|
||||
gnp.setStoreIdentifier(storeRef.getIdentifier());
|
||||
gnp.setShardProperty(shardProperty);
|
||||
gnp.setCoreName(coreName);
|
||||
|
||||
List<Node> nodes = client.getNodes(gnp, (int) info.getUpdates());
|
||||
for (Node node : nodes)
|
||||
@@ -452,6 +458,7 @@ public class MetadataTracker extends AbstractTracker implements Tracker
|
||||
gnp.setTransactionIds(txs);
|
||||
gnp.setStoreProtocol(storeRef.getProtocol());
|
||||
gnp.setStoreIdentifier(storeRef.getIdentifier());
|
||||
gnp.setCoreName(coreName);
|
||||
List<Node> nodes = client.getNodes(gnp, (int) info.getUpdates());
|
||||
for (Node node : nodes)
|
||||
{
|
||||
@@ -877,6 +884,7 @@ public class MetadataTracker extends AbstractTracker implements Tracker
|
||||
gnp.setStoreProtocol(storeRef.getProtocol());
|
||||
gnp.setStoreIdentifier(storeRef.getIdentifier());
|
||||
gnp.setShardProperty(shardProperty);
|
||||
gnp.setCoreName(coreName);
|
||||
List<Node> nodes = client.getNodes(gnp, Integer.MAX_VALUE);
|
||||
|
||||
ArrayList<Node> nodeBatch = new ArrayList<>();
|
||||
@@ -1050,6 +1058,7 @@ public class MetadataTracker extends AbstractTracker implements Tracker
|
||||
gnp.setTransactionIds(txs);
|
||||
gnp.setStoreProtocol(storeRef.getProtocol());
|
||||
gnp.setStoreIdentifier(storeRef.getIdentifier());
|
||||
gnp.setCoreName(coreName);
|
||||
return client.getNodes(gnp, Integer.MAX_VALUE);
|
||||
}
|
||||
catch (IOException e)
|
||||
|
||||
+202
@@ -0,0 +1,202 @@
|
||||
/*
|
||||
* Copyright (C) 2005-2019 Alfresco Software Limited.
|
||||
*
|
||||
* This file is part of Alfresco
|
||||
*
|
||||
* Alfresco is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Lesser General Public License as published by
|
||||
* the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* Alfresco is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Lesser General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Lesser General Public License
|
||||
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
|
||||
*/
|
||||
package org.alfresco.solr.tracker;
|
||||
|
||||
import java.util.Properties;
|
||||
import org.alfresco.model.ContentModel;
|
||||
import org.alfresco.repo.index.shard.ShardMethodEnum;
|
||||
import org.alfresco.repo.search.adaptor.lucene.QueryConstants;
|
||||
import org.alfresco.solr.AbstractAlfrescoDistributedTest;
|
||||
import org.alfresco.solr.client.Acl;
|
||||
import org.alfresco.solr.client.AclChangeSet;
|
||||
import org.alfresco.solr.client.AclReaders;
|
||||
import org.alfresco.solr.client.Node;
|
||||
import org.alfresco.solr.client.NodeMetaData;
|
||||
import org.alfresco.solr.client.StringPropertyValue;
|
||||
import org.alfresco.solr.client.Transaction;
|
||||
import org.apache.lucene.index.Term;
|
||||
import org.apache.lucene.search.BooleanClause;
|
||||
import org.apache.lucene.search.BooleanQuery;
|
||||
import org.apache.lucene.search.LegacyNumericRangeQuery;
|
||||
import org.apache.lucene.search.Query;
|
||||
import org.apache.lucene.search.TermQuery;
|
||||
import org.apache.lucene.util.LuceneTestCase;
|
||||
import org.apache.solr.SolrTestCaseJ4;
|
||||
import org.junit.AfterClass;
|
||||
import org.junit.Before;
|
||||
import org.junit.Test;
|
||||
|
||||
import static java.util.Collections.singletonList;
|
||||
import static org.alfresco.solr.AlfrescoSolrUtils.getAcl;
|
||||
import static org.alfresco.solr.AlfrescoSolrUtils.getAclChangeSet;
|
||||
import static org.alfresco.solr.AlfrescoSolrUtils.getAclReaders;
|
||||
import static org.alfresco.solr.AlfrescoSolrUtils.getNode;
|
||||
import static org.alfresco.solr.AlfrescoSolrUtils.getNodeMetaData;
|
||||
import static org.alfresco.solr.AlfrescoSolrUtils.getTransaction;
|
||||
import static org.alfresco.solr.AlfrescoSolrUtils.indexAclChangeSet;
|
||||
import static org.carrot2.shaded.guava.common.collect.ImmutableList.of;
|
||||
|
||||
/**
|
||||
* Test Routes based on an last registered shard
|
||||
*
|
||||
* @author Elia
|
||||
*/
|
||||
@SolrTestCaseJ4.SuppressSSL
|
||||
@SolrTestCaseJ4.SuppressObjectReleaseTracker (bugUrl = "RAMDirectory")
|
||||
@LuceneTestCase.SuppressCodecs({"Appending","Lucene3x","Lucene40","Lucene41","Lucene42","Lucene43", "Lucene44", "Lucene45","Lucene46","Lucene47","Lucene48","Lucene49"})
|
||||
public class DistributedLastRegisteredShardRouterTest extends AbstractAlfrescoDistributedTest
|
||||
{
|
||||
private static long MAX_WAIT_TIME = 80000;
|
||||
private final int timeout = 100000;
|
||||
|
||||
@Before
|
||||
private void initData() throws Throwable
|
||||
{
|
||||
initSolrServers(2, "DistributedLastRegisteredShardRoutingTest", getProperties());
|
||||
indexData();
|
||||
}
|
||||
|
||||
@AfterClass
|
||||
private static void destroyData() throws Throwable
|
||||
{
|
||||
dismissSolrServers();
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* Setup, indexes and returns the ACL used within the tests.
|
||||
*
|
||||
* @return the ACL used within the test.
|
||||
*/
|
||||
private Acl getTestAcl() throws Exception
|
||||
{
|
||||
AclChangeSet aclChangeSet = getAclChangeSet(1);
|
||||
Acl acl = getAcl(aclChangeSet);
|
||||
AclReaders aclReaders = getAclReaders(aclChangeSet, acl, singletonList("joel"), singletonList("phil"), null);
|
||||
|
||||
indexAclChangeSet(aclChangeSet, singletonList(acl), singletonList(aclReaders));
|
||||
|
||||
//Check for the ACL state stamp.
|
||||
BooleanQuery.Builder builder =
|
||||
new BooleanQuery.Builder()
|
||||
.add(new BooleanClause(new TermQuery(new Term(QueryConstants.FIELD_SOLR4_ID, "TRACKER!STATE!ACLTX")), BooleanClause.Occur.MUST))
|
||||
.add(new BooleanClause(LegacyNumericRangeQuery.newLongRange(
|
||||
QueryConstants.FIELD_S_ACLTXID, aclChangeSet.getId(), aclChangeSet.getId() + 1, true, false), BooleanClause.Occur.MUST));
|
||||
|
||||
Query waitForQuery = builder.build();
|
||||
waitForDocCount(waitForQuery, 1, MAX_WAIT_TIME);
|
||||
return acl;
|
||||
}
|
||||
|
||||
/**
|
||||
* Default data is indexed in solr.
|
||||
* 1 folder node with 2 children nodes.
|
||||
* 1 Child is on the same shard of the parent folder (shard 0) while the other is on shard 1.
|
||||
*/
|
||||
private void indexData() throws Exception
|
||||
{
|
||||
AclChangeSet aclChangeSet = getAclChangeSet(1);
|
||||
|
||||
Acl acl = getAcl(aclChangeSet);
|
||||
AclReaders aclReaders = getAclReaders(aclChangeSet, acl, singletonList("joel"), singletonList("phil"), null);
|
||||
|
||||
indexAclChangeSet(aclChangeSet,
|
||||
of(acl),
|
||||
of(aclReaders));
|
||||
|
||||
indexTestData(acl);
|
||||
}
|
||||
|
||||
|
||||
public void indexTestData(Acl acl) throws Exception
|
||||
{
|
||||
Transaction txn = getTransaction(0, 3);
|
||||
|
||||
/*
|
||||
* Create node1 in the first shard
|
||||
*/
|
||||
Node node1 = getNode(0, txn, acl, Node.SolrApiNodeStatus.UPDATED);
|
||||
node1.setExplicitShardId(0);
|
||||
NodeMetaData nodeMetaData1 = getNodeMetaData(node1, txn, acl, "elia", null, false);
|
||||
nodeMetaData1.getProperties().put(ContentModel.PROP_NAME, new StringPropertyValue("first"));
|
||||
|
||||
/*
|
||||
* Create node2 in the second shard
|
||||
*/
|
||||
Node node2 = getNode(1, txn, acl, Node.SolrApiNodeStatus.UPDATED);
|
||||
node2.setExplicitShardId(1);
|
||||
NodeMetaData nodeMetaData2 = getNodeMetaData(node2, txn, acl, "elia", null, false);
|
||||
nodeMetaData2.getProperties().put(ContentModel.PROP_NAME, new StringPropertyValue("second"));
|
||||
|
||||
/*
|
||||
* Create node3 with no explicitShardId
|
||||
*/
|
||||
Node node3 = getNode(2, txn, acl, Node.SolrApiNodeStatus.UPDATED);
|
||||
NodeMetaData nodeMetaData3 = getNodeMetaData(node3, txn, acl, "elia", null, false);
|
||||
nodeMetaData3.getProperties().put(ContentModel.PROP_NAME, new StringPropertyValue("third"));
|
||||
|
||||
/*
|
||||
* Create node4 in second shard
|
||||
*/
|
||||
Node node4 = getNode(3, txn, acl, Node.SolrApiNodeStatus.UPDATED);
|
||||
node4.setExplicitShardId(1);
|
||||
NodeMetaData nodeMetaData4 = getNodeMetaData(node4, txn, acl, "elia", null, false);
|
||||
nodeMetaData4.getProperties().put(ContentModel.PROP_NAME, new StringPropertyValue("second"));
|
||||
nodeMetaData4.getProperties().put(ContentModel.PROP_NAME, new StringPropertyValue("forth"));
|
||||
|
||||
/*
|
||||
* Create node4 in shard 4 (which does not exist)
|
||||
*/
|
||||
Node node5 = getNode(4, txn, acl, Node.SolrApiNodeStatus.UPDATED);
|
||||
node5.setExplicitShardId(4);
|
||||
NodeMetaData nodeMetaData5 = getNodeMetaData(node5, txn, acl, "elia", null, false);
|
||||
nodeMetaData5.getProperties().put(ContentModel.PROP_NAME, new StringPropertyValue("fifth"));
|
||||
|
||||
indexTransaction(txn,
|
||||
of(node1, node2, node3, node4, node5),
|
||||
of(nodeMetaData1, nodeMetaData2, nodeMetaData3, nodeMetaData4, nodeMetaData5));
|
||||
|
||||
/*
|
||||
* Get sure the nodes are indexed correctly in the shards
|
||||
*/
|
||||
waitForShardsCount(params("q", "cm:content:world", "qt", "/afts"), 3, timeout, System.currentTimeMillis());
|
||||
}
|
||||
|
||||
@Test
|
||||
public void testNodesShouldBeIndexedInSpecifiedShard() throws Exception
|
||||
{
|
||||
assertShardCount(0, params("q", "cm:content:world", "qt", "/afts"), 1);
|
||||
assertShardCount(1, params("q", "cm:content:world", "qt", "/afts"), 2);
|
||||
|
||||
assertShardCount(0, params("q", "cm:name:first", "qt", "/afts", "shards", shards), 1);
|
||||
|
||||
// The third node shouldn't be found because no explicit shard is set
|
||||
assertShardCount(0, params("q", "cm:name:third", "qt", "/afts", "shards", shards), 0);
|
||||
|
||||
// The fifth node shouldn't be found because the explicit shard set is not running
|
||||
assertShardCount(0, params("q", "cm:name:fifth", "qt", "/afts", "shards", shards), 0);
|
||||
}
|
||||
|
||||
protected static Properties getProperties()
|
||||
{
|
||||
Properties prop = new Properties();
|
||||
prop.put("shard.method", ShardMethodEnum.LAST_REGISTERED_INDEXING_SHARD.toString());
|
||||
return prop;
|
||||
}
|
||||
}
|
||||
@@ -22,7 +22,8 @@
|
||||
</distributionManagement>
|
||||
|
||||
<properties>
|
||||
<dependency.alfresco-data-model.version>8.32</dependency.alfresco-data-model.version>
|
||||
<dependency.alfresco-data-model.version>8.34</dependency.alfresco-data-model.version>
|
||||
<dependency.jackson.version>2.9.9</dependency.jackson.version>
|
||||
</properties>
|
||||
|
||||
<dependencies>
|
||||
@@ -35,13 +36,13 @@
|
||||
<dependency>
|
||||
<groupId>com.fasterxml.jackson.core</groupId>
|
||||
<artifactId>jackson-core</artifactId>
|
||||
<version>2.9.8</version>
|
||||
<version>${dependency.jackson.version}</version>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>com.fasterxml.jackson.core</groupId>
|
||||
<artifactId>jackson-annotations</artifactId>
|
||||
<version>2.9.8</version>
|
||||
<version>${dependency.jackson.version}</version>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
|
||||
+51
-40
@@ -1,28 +1,28 @@
|
||||
/*
|
||||
* #%L
|
||||
* Alfresco Solr Client
|
||||
* %%
|
||||
* Copyright (C) 2005 - 2016 Alfresco Software Limited
|
||||
* %%
|
||||
* This file is part of the Alfresco software.
|
||||
* If the software was purchased under a paid Alfresco license, the terms of
|
||||
* the paid license agreement will prevail. Otherwise, the software is
|
||||
* provided under the following open source license terms:
|
||||
*
|
||||
* Alfresco is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Lesser General Public License as published by
|
||||
* the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* Alfresco is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Lesser General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Lesser General Public License
|
||||
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
|
||||
* #L%
|
||||
*/
|
||||
/*
|
||||
* #%L
|
||||
* Alfresco Solr Client
|
||||
* %%
|
||||
* Copyright (C) 2005 - 2016 Alfresco Software Limited
|
||||
* %%
|
||||
* This file is part of the Alfresco software.
|
||||
* If the software was purchased under a paid Alfresco license, the terms of
|
||||
* the paid license agreement will prevail. Otherwise, the software is
|
||||
* provided under the following open source license terms:
|
||||
*
|
||||
* Alfresco is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Lesser General Public License as published by
|
||||
* the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* Alfresco is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Lesser General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Lesser General Public License
|
||||
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
|
||||
* #L%
|
||||
*/
|
||||
package org.alfresco.solr.client;
|
||||
|
||||
import java.util.List;
|
||||
@@ -48,10 +48,11 @@ public class GetNodesParameters
|
||||
private Set<QName> excludeNodeTypes;
|
||||
|
||||
private Set<QName> includeAspects;
|
||||
private Set<QName> excludeAspects;
|
||||
|
||||
private QName shardProperty;
|
||||
private Set<QName> excludeAspects;
|
||||
|
||||
private QName shardProperty;
|
||||
private String coreName;
|
||||
|
||||
public boolean getStoreFilter()
|
||||
{
|
||||
return (storeProtocol != null || storeIdentifier != null);
|
||||
@@ -145,17 +146,27 @@ public class GetNodesParameters
|
||||
public void setExcludeAspects(Set<QName> excludeAspects)
|
||||
{
|
||||
this.excludeAspects = excludeAspects;
|
||||
}
|
||||
|
||||
public QName getShardProperty()
|
||||
{
|
||||
return this.shardProperty;
|
||||
}
|
||||
|
||||
public void setShardProperty(QName shardProperty)
|
||||
{
|
||||
this.shardProperty = shardProperty;
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
public QName getShardProperty()
|
||||
{
|
||||
return this.shardProperty;
|
||||
}
|
||||
|
||||
public void setShardProperty(QName shardProperty)
|
||||
{
|
||||
this.shardProperty = shardProperty;
|
||||
}
|
||||
|
||||
public String getCoreName()
|
||||
{
|
||||
return this.coreName;
|
||||
}
|
||||
|
||||
public void setCoreName(String coreName)
|
||||
{
|
||||
this.coreName = coreName;
|
||||
}
|
||||
|
||||
|
||||
}
|
||||
|
||||
+64
-51
@@ -1,28 +1,28 @@
|
||||
/*
|
||||
* #%L
|
||||
* Alfresco Solr Client
|
||||
* %%
|
||||
* Copyright (C) 2005 - 2016 Alfresco Software Limited
|
||||
* %%
|
||||
* This file is part of the Alfresco software.
|
||||
* If the software was purchased under a paid Alfresco license, the terms of
|
||||
* the paid license agreement will prevail. Otherwise, the software is
|
||||
* provided under the following open source license terms:
|
||||
*
|
||||
* Alfresco is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Lesser General Public License as published by
|
||||
* the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* Alfresco is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Lesser General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Lesser General Public License
|
||||
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
|
||||
* #L%
|
||||
*/
|
||||
/*
|
||||
* #%L
|
||||
* Alfresco Solr Client
|
||||
* %%
|
||||
* Copyright (C) 2005 - 2016 Alfresco Software Limited
|
||||
* %%
|
||||
* This file is part of the Alfresco software.
|
||||
* If the software was purchased under a paid Alfresco license, the terms of
|
||||
* the paid license agreement will prevail. Otherwise, the software is
|
||||
* provided under the following open source license terms:
|
||||
*
|
||||
* Alfresco is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Lesser General Public License as published by
|
||||
* the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* Alfresco is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Lesser General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Lesser General Public License
|
||||
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
|
||||
* #L%
|
||||
*/
|
||||
package org.alfresco.solr.client;
|
||||
|
||||
public class Node
|
||||
@@ -37,8 +37,9 @@ public class Node
|
||||
private long txnId;
|
||||
private SolrApiNodeStatus status;
|
||||
private String tenant;
|
||||
private long aclId;
|
||||
private long aclId;
|
||||
private String shardPropertyValue;
|
||||
private Integer explicitShardId;
|
||||
|
||||
public long getId()
|
||||
{
|
||||
@@ -99,32 +100,44 @@ public class Node
|
||||
public void setAclId(long aclId)
|
||||
{
|
||||
this.aclId = aclId;
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* The property value to use for sharding - as requested
|
||||
*
|
||||
* @return null - if the node does not have the property, the standard "String" value of the property if it is present on the node.
|
||||
* For dates and datetime properties this will be the ISO formatted datetime.
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* The property value to use for sharding - as requested
|
||||
*
|
||||
* @return null - if the node does not have the property, the standard "String" value of the property if it is present on the node.
|
||||
* For dates and datetime properties this will be the ISO formatted datetime.
|
||||
*/
|
||||
public String getShardPropertyValue()
|
||||
{
|
||||
return this.shardPropertyValue;
|
||||
}
|
||||
|
||||
public void setShardPropertyValue(String shardPropertyValue)
|
||||
{
|
||||
this.shardPropertyValue = shardPropertyValue;
|
||||
}
|
||||
@Override
|
||||
public String toString()
|
||||
{
|
||||
return "Node [id=" + this.id + ", nodeRef=" + this.nodeRef + ", txnId=" + this.txnId
|
||||
+ ", status=" + this.status + ", tenant=" + this.tenant + ", aclId="
|
||||
+ this.aclId + ", shardPropertyValue=" + this.shardPropertyValue + "]";
|
||||
}
|
||||
|
||||
public String getShardPropertyValue()
|
||||
{
|
||||
return this.shardPropertyValue;
|
||||
}
|
||||
|
||||
public void setShardPropertyValue(String shardPropertyValue)
|
||||
{
|
||||
this.shardPropertyValue = shardPropertyValue;
|
||||
}
|
||||
|
||||
public Integer getExplicitShardId()
|
||||
{
|
||||
return this.explicitShardId;
|
||||
}
|
||||
|
||||
public void setExplicitShardId(Integer explicitShardId)
|
||||
{
|
||||
this.explicitShardId = explicitShardId;
|
||||
}
|
||||
|
||||
@Override
|
||||
public String toString()
|
||||
{
|
||||
return "Node [id=" + this.id + ", nodeRef=" + this.nodeRef + ", txnId=" + this.txnId
|
||||
+ ", status=" + this.status + ", tenant=" + this.tenant + ", aclId="
|
||||
+ this.aclId + ", shardPropertyValue=" + this.shardPropertyValue
|
||||
+ ", explicitShardId=" + this.explicitShardId + "]";
|
||||
}
|
||||
|
||||
|
||||
|
||||
}
|
||||
|
||||
+10
@@ -610,6 +610,11 @@ public class SOLRAPIClient
|
||||
{
|
||||
body.put("shardProperty", parameters.getShardProperty().toString());
|
||||
}
|
||||
|
||||
if (parameters.getCoreName() != null){
|
||||
body.put("coreName", parameters.getCoreName());
|
||||
}
|
||||
|
||||
|
||||
PostRequest req = new PostRequest(url.toString(), body.toString(), "application/json");
|
||||
|
||||
@@ -668,6 +673,11 @@ public class SOLRAPIClient
|
||||
if(jsonNodeInfo.has("shardPropertyValue"))
|
||||
{
|
||||
nodeInfo.setShardPropertyValue(jsonNodeInfo.getString("shardPropertyValue"));
|
||||
}
|
||||
|
||||
if(jsonNodeInfo.has("explicitShardId"))
|
||||
{
|
||||
nodeInfo.setExplicitShardId(jsonNodeInfo.getInt("explicitShardId"));
|
||||
}
|
||||
|
||||
if(jsonNodeInfo.has("tenant"))
|
||||
|
||||
+1
-1
@@ -1,5 +1,5 @@
|
||||
/*
|
||||
$Id: LICENSE.txt 1128 2006-02-05 21:49:04Z elharo $
|
||||
$Id$
|
||||
|
||||
Copyright 2003-2006 The Werken Company. All Rights Reserved.
|
||||
|
||||
|
||||
@@ -20,7 +20,7 @@ asm-3.3.1.jar http://code.google.com/p/subethasmtp/
|
||||
antlr-runtime-3.5.2.jar http://www.antlr.org/
|
||||
ST4-4.0.8.jar https://www.stringtemplate.org/license.html
|
||||
antlr-3.5.2.jar http://www.antlr.org/
|
||||
jaxen-1.1.6.jar http://jaxen.codehaus.org/
|
||||
jaxen-1.2.0.jar http://www.cafeconleche.org/jaxen/
|
||||
|
||||
|
||||
=== Apache variant License ===
|
||||
@@ -55,8 +55,8 @@ mybatis-spring-1.2.5.jar http://www.mybatis.org/
|
||||
chemistry-opencmis-server-support-1.0.0.jar http://chemistry.apache.org/
|
||||
chemistry-opencmis-server-bindings-1.0.0.jar http://chemistry.apache.org/
|
||||
quartz-2.3.1.jar http://quartz-scheduler.org/
|
||||
jackson-core-2.9.8.jar http://jackson.codehaus.org/
|
||||
jackson-annotations-2.9.8.jar http://jackson.codehaus.org/
|
||||
jackson-core-2.9.9.jar https://github.com/FasterXML/jackson
|
||||
jackson-annotations-2.9.9.jar https://github.com/FasterXML/jackson
|
||||
commons-httpclient-3.1-HTTPCLIENT-1265.jar http://jakarta.apache.org/commons/
|
||||
spring-aop-5.1.6.RELEASE.jar http://projects.spring.io/spring-framework/
|
||||
spring-beans-5.1.6.RELEASE.jar http://projects.spring.io/spring-framework/
|
||||
|
||||
Reference in New Issue
Block a user