inteligr8/SearchServices
1
0
Fork 0
mirror of https://github.com/Alfresco/SearchServices.git synced 2026-09-16 18:12:56 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'master' of git.alfresco.com:search_discovery/insightengine

Browse Source
This commit is contained in:
Angel Borroy
2019-06-18 10:07:52 +02:00
parent 0050743673 945229a496
commit a5d5d99b44
28 changed files with 817 additions and 187 deletions
Download Patch File Download Diff File Expand all files Collapse all files
LICENSE
+165
View File
@@ -0,0 +1,165 @@
GNU LESSER GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
This version of the GNU Lesser General Public License incorporates
the terms and conditions of version 3 of the GNU General Public
License, supplemented by the additional permissions listed below.
0. Additional Definitions.
As used herein, "this License" refers to version 3 of the GNU Lesser
General Public License, and the "GNU GPL" refers to version 3 of the GNU
General Public License.
"The Library" refers to a covered work governed by this License,
other than an Application or a Combined Work as defined below.
An "Application" is any work that makes use of an interface provided
by the Library, but which is not otherwise based on the Library.
Defining a subclass of a class defined by the Library is deemed a mode
of using an interface provided by the Library.
A "Combined Work" is a work produced by combining or linking an
Application with the Library. The particular version of the Library
with which the Combined Work was made is also called the "Linked
Version".
The "Minimal Corresponding Source" for a Combined Work means the
Corresponding Source for the Combined Work, excluding any source code
for portions of the Combined Work that, considered in isolation, are
based on the Application, and not on the Linked Version.
The "Corresponding Application Code" for a Combined Work means the
object code and/or source code for the Application, including any data
and utility programs needed for reproducing the Combined Work from the
Application, but excluding the System Libraries of the Combined Work.
1. Exception to Section 3 of the GNU GPL.
You may convey a covered work under sections 3 and 4 of this License
without being bound by section 3 of the GNU GPL.
2. Conveying Modified Versions.
If you modify a copy of the Library, and, in your modifications, a
facility refers to a function or data to be supplied by an Application
that uses the facility (other than as an argument passed when the
facility is invoked), then you may convey a copy of the modified
version:
a) under this License, provided that you make a good faith effort to
ensure that, in the event an Application does not supply the
function or data, the facility still operates, and performs
whatever part of its purpose remains meaningful, or
b) under the GNU GPL, with none of the additional permissions of
this License applicable to that copy.
3. Object Code Incorporating Material from Library Header Files.
The object code form of an Application may incorporate material from
a header file that is part of the Library. You may convey such object
code under terms of your choice, provided that, if the incorporated
material is not limited to numerical parameters, data structure
layouts and accessors, or small macros, inline functions and templates
(ten or fewer lines in length), you do both of the following:
a) Give prominent notice with each copy of the object code that the
Library is used in it and that the Library and its use are
covered by this License.
b) Accompany the object code with a copy of the GNU GPL and this license
document.
4. Combined Works.
You may convey a Combined Work under terms of your choice that,
taken together, effectively do not restrict modification of the
portions of the Library contained in the Combined Work and reverse
engineering for debugging such modifications, if you also do each of
the following:
a) Give prominent notice with each copy of the Combined Work that
the Library is used in it and that the Library and its use are
covered by this License.
b) Accompany the Combined Work with a copy of the GNU GPL and this license
document.
c) For a Combined Work that displays copyright notices during
execution, include the copyright notice for the Library among
these notices, as well as a reference directing the user to the
copies of the GNU GPL and this license document.
d) Do one of the following:
0) Convey the Minimal Corresponding Source under the terms of this
License, and the Corresponding Application Code in a form
suitable for, and under terms that permit, the user to
recombine or relink the Application with a modified version of
the Linked Version to produce a modified Combined Work, in the
manner specified by section 6 of the GNU GPL for conveying
Corresponding Source.
1) Use a suitable shared library mechanism for linking with the
Library. A suitable mechanism is one that (a) uses at run time
a copy of the Library already present on the user's computer
system, and (b) will operate properly with a modified version
of the Library that is interface-compatible with the Linked
Version.
e) Provide Installation Information, but only if you would otherwise
be required to provide such information under section 6 of the
GNU GPL, and only to the extent that such information is
necessary to install and execute a modified version of the
Combined Work produced by recombining or relinking the
Application with a modified version of the Linked Version. (If
you use option 4d0, the Installation Information must accompany
the Minimal Corresponding Source and Corresponding Application
Code. If you use option 4d1, you must provide the Installation
Information in the manner specified by section 6 of the GNU GPL
for conveying Corresponding Source.)
5. Combined Libraries.
You may place library facilities that are a work based on the
Library side by side in a single library together with other library
facilities that are not Applications and are not covered by this
License, and convey such a combined library under terms of your
choice, if you do both of the following:
a) Accompany the combined library with a copy of the same work based
on the Library, uncombined with any other library facilities,
conveyed under the terms of this License.
b) Give prominent notice with the combined library that part of it
is a work based on the Library, and explaining where to find the
accompanying uncombined form of the same work.
6. Revised Versions of the GNU Lesser General Public License.
The Free Software Foundation may publish revised and/or new versions
of the GNU Lesser General Public License from time to time. Such new
versions will be similar in spirit to the present version, but may
differ in detail to address new problems or concerns.
Each version is given a distinguishing version number. If the
Library as you received it specifies that a certain numbered version
of the GNU Lesser General Public License "or any later version"
applies to it, you have the option of following the terms and
conditions either of that published version or of any later version
published by the Free Software Foundation. If the Library as you
received it does not specify a version number of the GNU Lesser
General Public License, you may choose any version of the GNU Lesser
General Public License ever published by the Free Software Foundation.
If the Library as you received it specifies that a proxy can decide
whether future versions of the GNU Lesser General Public License shall
apply, that proxy's public statement of acceptance of any version is
permanent authorization for you to choose that version for the
Library.
README.md
+43
View File
@@ -0,0 +1,43 @@
## Alfresco Search Services and Insight Engine
This repository includes the source code for Alfresco Search Services and Insight Engine products.
**Alfresco Search Services** provides searching capabilities to Alfresco Content Services by leveraging [Apache SOLR](https://lucene.apache.org/solr/) core features. This product is used for both Enterprise and Community releases of Alfresco Content Services and it lives in [GitHub](https://github.com/Alfresco/SearchServices).
The official documentation for this product can be found at [Alfresco Search Services](https://docs.alfresco.com/search-community/concepts/search-home.html).
**Insight Engine** works together with *Alfresco Search Services* to provide extended capabilities to Alfresco Content Services (like a JDBC connector and Zeppelin integration). This product is licensed only in Enterprise mode, so the source code lives in our private *GitLab* and it's only available for Enterprise customers.
The official documentation for this product can be found at [Alfresco Search and Insight Engine](https://docs.alfresco.com/sie/concepts/Search-Insight-Engine-overview.html).
### Alfresco Search Services
Getting the code using a Git client.
```bash
$ git clone https://github.com/Alfresco/SearchServices.git
```
Build the project using Maven.
```bash
$ cd SearchServices/search-services
$ mvn clean install
```
All the resources required to run Alfresco Search Services will be available under `search-services/packaging/target` folder, including the distribution ZIP for local installations.
The Docker Image source code is available at `search-services/packaging/src/docker`. The building for the Docker Image is available in the public repository of Alfresco at Docker Hub:
[https://hub.docker.com/r/alfresco/alfresco-search-services](https://hub.docker.com/r/alfresco/alfresco-search-services)
*Note* The root `pom.xml` living in this folder is used for packaging Search Services and Insight Engine together. This file includes URLs not available for Community users, but it allows the `search-services` module without accessing to these resources.
More details are available at [search-services](/search-services) folder.
### Insight Engine
**Following resources will not be available for Community users**
More details are available at [insight-engine](/insight-engine) folder.
search-services/README.md
+297 -26
View File
@@ -7,73 +7,343 @@ Alfresco Search Services using Alfresco and Apache Solr
Git:
```bash
git clone https://github.com/Alfresco/SearchServices.git
$ git clone https://github.com/Alfresco/SearchServices.git
```
### Use Maven
Build project:
Build the project:
```bash
mvn clean install
$ mvn clean install -DskipTests=true
```
All the resources needed for the docker image will be available under packaging/target/docker-resources/
All the resources required to run Alfresco Search Services will be available under `packaging/target` folder.
### Start Alfresco Search Services from source
To run Alfresco Search Services locally first build the zip file using:
To run Alfresco Search Services locally, building the ZIP distribution file is required.
```bash
mvn clean install
$ mvn clean install -DskipTests=true
```
Extract the zip file and launch Alfresco Search Services using:
After the project is successfully built, ZIP can be extracted.
```bash
cd packaging/target
unzip alfresco-search-services-*.zip
cd alfresco-search-services/solr
./bin/solr start -Dcreate.alfresco.defaults=alfresco,archive
$ cd packaging/target
$ unzip alfresco-search-services-*.zip
$ cd alfresco-search-services
```
If you also start an ACS instance then index will be populated. By default Alfresco Search Services runs on port 8983, but this can be set by supplying e.g. `-p 8083` to the "solr start" command.
From Alfresco *Search Services 1.3.0.3*, distribution ZIP is released with Mutual Authentication TLS (SSL) by default. So before starting the service, generating secure keys for SSL communication is required. You can find detailed information for this step at [Alfresco documentation](https://docs.alfresco.com/search-enterprise/tasks/generate-keys-ssl.html).
The `keystores` folder generated by the SSL Tool contains the keystores and truststores for SSL configuration. In the following steps, it's assumed that SSL Tool has been executed from `/tmp` or `C:\tmp` folder.
```bash
$ tree /tmp/keystores/
keystores/
├── alfresco
│   ├── keystore
│   ├── keystore-passwords.properties
│   ├── ssl-keystore-passwords.properties
│   ├── ssl-truststore-passwords.properties
│   ├── ssl.keystore
│   └── ssl.truststore
├── client
│   └── browser.p12
├── solr
│   ├── ssl-keystore-passwords.properties
│   ├── ssl-truststore-passwords.properties
│   ├── ssl.repo.client.keystore
│   └── ssl.repo.client.truststore
└── zeppelin
├── ssl.repo.client.keystore
└── ssl.repo.client.truststore
```
SOLR SSL configuration files are available in `/tmp/keystores/solr` folder.
These files must be copied to `rerank` configuration folder.
```
$ cp /tmp/keystores/solr/* solrhome/templates/rerank/conf
```
If you are running from a *Linux* or *Mac OS X* machine, add following lines to `solr.in.sh` file.
```
SOLR_SSL_KEY_STORE=/tmp/keystores/solr/ssl.repo.client.keystore
SOLR_SSL_KEY_STORE_PASSWORD=keystore
SOLR_SSL_KEY_STORE_TYPE=JCEKS
SOLR_SSL_TRUST_STORE=/tmp/keystores/solr/ssl.repo.client.truststore
SOLR_SSL_TRUST_STORE_PASSWORD=truststore
SOLR_SSL_TRUST_STORE_TYPE=JCEKS
SOLR_SSL_NEED_CLIENT_AUTH=true
SOLR_SSL_WANT_CLIENT_AUTH=false
```
If you are running from a *Windows* machine, add following lines to `solr.in.cmd` file.
```
set SOLR_SSL_KEY_STORE=C:\tmp\keystores\solr\ssl.repo.client.keystore
SOLR_SSL_KEY_STORE_PASSWORD=keystore
SOLR_SSL_KEY_STORE_TYPE=JCEKS
SOLR_SSL_TRUST_STORE=C:\tmp\keystores\solr\ssl.repo.client.truststore
SOLR_SSL_TRUST_STORE_PASSWORD=truststore
SOLR_SSL_TRUST_STORE_TYPE=JCEKS
SOLR_SSL_NEED_CLIENT_AUTH=true
SOLR_SSL_WANT_CLIENT_AUTH=false
```
Once this settings are ready, start SOLR service from command line:
```
$ ./solr/bin/solr start "-Dcreate.alfresco.defaults=alfresco,archive \
-Dsolr.ssl.checkPeerName=false \
-Dsolr.allow.unsafe.resourceloading=true" -f
```
SOLR will create Alfresco cores (`alfresco` and `archive`) when starting, and configuration from `rerank` template will be copied to each core.
If you also started an ACS instance running in [https://localhost:8443/alfresco](https://localhost:8443/alfresco) with the keystores provided by the SSL Tool (`keystores/alfresco` folder), then the index will be populated.
SOLR Web Console will be available at:
[https://localhost:8983/solr](https://localhost:8983/solr)
**Note** Client certificate `browser.p12`, generated by the SSL Tool, is required to be installed in your browser in order to access to this Web Console.
By default Alfresco Search Services runs on port 8983, but this can be set by supplying e.g. `-p 8083` to the "solr start" command.
To set up remote debugging (on port 5005) start Alfresco Search Services with the following command and then connect using your IDE:
```bash
./bin/solr start -a "-Dcreate.alfresco.defaults=alfresco,archive -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=5005"
$ ./solr/bin/solr start -a "-Dcreate.alfresco.defaults=alfresco,archive \
-Dsolr.ssl.checkPeerName=false \
-Dsolr.allow.unsafe.resourceloading=true \
-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=5005" -f
```
DBID based sharding can be set up from the command line. For example a core containing shards 0, 1, 6 and 7 from an
index with twelve shards can be set up by starting an instance of Alfresco Search Services with a command like:
```bash
./bin/solr start -a -Dcreate.alfresco.defaults=alfresco,archive -Dnum.shards=12 -Dshard.ids=0,1,6,7
```
Further instances should be set up to contain the other shards, and it is possible to adjust the distribution and
replication of shards to achieve the desired index performance and redundancy.
To stop Alfresco Search Services:
```bash
./bin/solr stop
$ ./solr/bin/solr stop
```
### Docker
To build the docker image:
**Using Plain HTTP**
If you want to use Plain HTTP for SOLR instead of Mutual Auth TLS (SSL), use following steps.
```bash
cd packaging/target/docker-resources/
docker build -t searchservices:develop .
$ mvn clean install -DskipTests=true
```
After the project is successfully built, ZIP can be extracted.
```bash
$ cd packaging/target
$ unzip alfresco-search-services-*.zip
$ cd alfresco-search-services
```
Change default Alfresco Communication protocol to `none`.
```bash
$ sed -i 's/alfresco.secureComms=https/alfresco.secureComms=none/' solrhome/templates/rerank/conf/solrcore.properties
```
*Note* Above line is written in GNU sed, you can use `gsed` from Mac OS X or just edit the file with a Text Editor.
Start SOLR service from command line:
```
$ ./solr/bin/solr start "-Dcreate.alfresco.defaults=alfresco,archive" -f
```
SOLR will create Alfresco cores (`alfresco` and `archive`) when starting, and configuration from `rerank` template will be copied to each core and if you also started an ACS instance running in [http://localhost:8080/alfresco](http://localhost:8080/alfresco) then the index will be populated.
SOLR Web Console will be available at:
[http://localhost:8983/solr](http://localhost:8983/solr)
### Use Alfresco Search Services Docker Image
Once the project has been built, the Docker image can be also built:
```bash
$ cd packaging/target/docker-resources/
$ docker build -t searchservices:develop .
```
*Search Services* Docker image is configured with with **Mutual Authentication TLS (SSL)** by default.
**Configuration**
To pass an environment variable, it can be used the "-e" argument:
```bash
$ docker run -e SOLR_JAVA_MEM="-Xms4g -Xmx4g" -p 8983:8983 searchservices:develop
```
To pass several environment variables (e.g. SOLR\_ALFRESCO\_HOST, SOLR\_ALFRESCO\_PORT, SOLR\_SOLR\_HOST, SOLR\_SOLR\_PORT, SOLR\_CREATE\_ALFRESCO\_DEFAULTS, SOLR\_HEAP, etc.), just include the "-e" argument as times as required:
```bash
$ docker run -e SOLR_ALFRESCO_HOST=localhost -e SOLR_ALFRESCO_PORT=8080 -p 8983:8983 searchservices:develop
```
**Using Mutual Auth TLS (SSL)**
This Docker image is exposing as VOLUME the folder `/opt/alfresco-search-services/keystores`, that can be used to mount `keystores` folder from host.
By default Docker image is using SSL, but an environment variable `ALFRESCO_SECURE_COMMS=https` can be also passed to the Docker container to declare explicitly the SSL mode.
Additionally, SOLR Jetty server must be configured to start in SSL Mode using `SOLR_SSL_*` environment variables and Search Services must be configured by using Java environment variables starting with `alfresco.encryption.ssl.*`
Following command will start Search Services with SSL using keystores located at `/tmp/keystores/solr`. Note that the internal folders are relative to `/opt/alfresco-search-services/keystores`, as this is the Docker container folder exposed to hold the keystores.
```bash
$ docker run -p 8983:8983 \
-v /tmp/keystores/solr:/opt/alfresco-search-services/keystores \
-e SOLR_CREATE_ALFRESCO_DEFAULTS=alfresco,archive \
-e SOLR_SSL_KEY_STORE=/opt/alfresco-search-services/keystores/ssl.repo.client.keystore \
-e SOLR_SSL_KEY_STORE_PASSWORD=keystore \
-e SOLR_SSL_KEY_STORE_TYPE=JCEKS \
-e SOLR_SSL_TRUST_STORE=/opt/alfresco-search-services/keystores/ssl.repo.client.truststore \
-e SOLR_SSL_TRUST_STORE_PASSWORD=truststore \
-e SOLR_SSL_TRUST_STORE_TYPE=JCEKS \
-e SOLR_SSL_NEED_CLIENT_AUTH=true \
-e SOLR_OPTS="-Dsolr.ssl.checkPeerName=false \
-Dsolr.allow.unsafe.resourceloading=true \
-Dalfresco.encryption.ssl.keystore.type=JCEKS
-Dalfresco.encryption.ssl.keystore.location=/opt/alfresco-search-services/keystores/ssl.repo.client.keystore
-Dalfresco.encryption.ssl.keystore.passwordFileLocation=/opt/alfresco-search-services/keystores/ssl-keystore-passwords.properties
-Dalfresco.encryption.ssl.truststore.type=JCEKS
-Dalfresco.encryption.ssl.truststore.location=/opt/alfresco-search-services/keystores/ssl.repo.client.truststore
-Dalfresco.encryption.ssl.truststore.passwordFileLocation=/opt/alfresco-search-services/keystores/ssl-truststore-passwords.properties
" \
searchservices:develop
```
SOLR Web Console will be available at:
[https://localhost:8983/solr](https://localhost:8983/solr)
*Note* You must install the `browser.p12` certificate in your browser in order to access to this URL.
**Using Plain HTTP**
By default Docker image is using SSL, so it's required to add an environment variable `ALFRESCO_SECURE_COMMS=none` to use SOLR in plain HTTP mode.
To run the docker image:
```bash
docker run -p 8983:8983 searchservices:develop
$ docker run -p 8983:8983 -e ALFRESCO_SECURE_COMMS=none -e SOLR_CREATE_ALFRESCO_DEFAULTS=alfresco,archive searchservices:develop
```
To pass an environment variable:
SOLR Web Console will be available at:
```bash
docker run -e SOLR_JAVA_MEM=“-Xms4g -Xmx4g” -p 8983:8983 searchservices:develop
[http://localhost:8983/solr](http://localhost:8983/solr)
**Public Docker repository**
This Docker Image is available at Alfresco Docker Hub:
[https://hub.docker.com/r/alfresco/alfresco-search-services](https://hub.docker.com/r/alfresco/alfresco-search-services)
To use the public image instead of the local one (`searchservices:develop`) just use `alfresco/alfresco-search-services:1.3.x.x` labels.
### Use Alfresco Search Services Docker Image with Docker Compose
Sample configuration in a Docker Compose file using **Plain HTTP** protocol to communicate with Alfresco Repository.
```
solr6:
image: searchservices:develop
mem_limit: 2500m
environment:
# Solr needs to know how to register itself with Alfresco
SOLR_ALFRESCO_HOST: "alfresco"
SOLR_ALFRESCO_PORT: "8080"
# Alfresco needs to know how to call solr
SOLR_SOLR_HOST: "solr6"
SOLR_SOLR_PORT: "8983"
# HTTP settings
ALFRESCO_SECURE_COMMS: "none"
#Create the default alfresco and archive cores
SOLR_CREATE_ALFRESCO_DEFAULTS: "alfresco,archive"
SOLR_JAVA_MEM: "-Xms2g -Xmx2g"
ports:
- 8083:8983 #Browser port
```
To pass several environment variables (e.g. SOLR\_ALFRESCO\_HOST, SOLR\_ALFRESCO\_PORT, SOLR\_SOLR\_HOST, SOLR\_SOLR\_PORT, SOLR\_CREATE\_ALFRESCO\_DEFAULTS, SOLR\_HEAP, etc.):
SOLR Web Console will be available at:
[http://localhost:8983/solr](http://localhost:8983/solr)
Sample configuration in a Docker Compose file using **Mutual Auth TLS (SSL)** protocol to communicate with Alfresco Repository.
```bash
docker run -e SOLR_ALFRESCO_HOST=localhost -e SOLR_ALFRESCO_PORT=8080 -p 8983:8983 searchservices:develop
```
solr6:
image: searchservices:develop
mem_limit: 2500m
environment:
# Solr needs to know how to register itself with Alfresco
SOLR_ALFRESCO_HOST: "alfresco"
SOLR_ALFRESCO_PORT: "8443"
# Alfresco needs to know how to call solr
SOLR_SOLR_HOST: "solr6"
SOLR_SOLR_PORT: "8983"
# SSL settings
ALFRESCO_SECURE_COMMS: "https"
SOLR_SSL_TRUST_STORE: "/opt/alfresco-search-services/keystore/ssl.repo.client.truststore"
SOLR_SSL_TRUST_STORE_PASSWORD: "truststore"
SOLR_SSL_TRUST_STORE_TYPE: "JCEKS"
SOLR_SSL_KEY_STORE: "/opt/alfresco-search-services/keystore/ssl.repo.client.keystore"
SOLR_SSL_KEY_STORE_PASSWORD: "keystore"
SOLR_SSL_KEY_STORE_TYPE: "JCEKS"
SOLR_SSL_NEED_CLIENT_AUTH: "true"
#Create the default alfresco and archive cores
SOLR_CREATE_ALFRESCO_DEFAULTS: "alfresco,archive"
SOLR_JAVA_MEM: "-Xms2g -Xmx2g"
SOLR_OPTS: "
-Dsolr.ssl.checkPeerName=false
-Dsolr.allow.unsafe.resourceloading=true
-Dalfresco.encryption.ssl.keystore.type=JCEKS
-Dalfresco.encryption.ssl.keystore.location=/opt/alfresco-search-services/keystores/ssl.repo.client.keystore
-Dalfresco.encryption.ssl.keystore.passwordFileLocation=/opt/alfresco-search-services/keystores/ssl-keystore-passwords.properties
-Dalfresco.encryption.ssl.truststore.type=JCEKS
-Dalfresco.encryption.ssl.truststore.location=/opt/alfresco-search-services/keystores/ssl.repo.client.truststore
-Dalfresco.encryption.ssl.truststore.passwordFileLocation=/opt/alfresco-search-services/keystores/ssl-truststore-passwords.properties
"
ports:
- 8083:8983 #Browser port
volumes:
- ./keystores/solr:/opt/alfresco-search-services/keystores
```
SOLR Web Console will be available at:
[https://localhost:8983/solr](https://localhost:8983/solr)
**Samples for development use only**
docker-compose files can be used to start up Search Services with Alfresco and Share. There are two docker-composes files available. Depending on the version you want to start either change to 5.x or 6.x. E.g.
@@ -87,12 +357,13 @@ This will start up Alfresco, Postgres, Share and SearchServices. You can access
* Alfresco: http://localhost:8081/alfresco
* Share: http://localhost:8082/share
* Solr: http://localhost:8083/solr
If you start version 5.x instead you can also access the API Explorer:
* API Explorer: http://localhost:8084/api-explorer
### License
Copyright (C) 2005 - 2017 Alfresco Software Limited
This file is part of the Alfresco software.
search-services/alfresco-search/pom.xml
+1 -1
View File
@@ -101,7 +101,7 @@
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<version>2.27.0</version>
<version>2.28.2</version>
<scope>test</scope>
</dependency>
search-services/alfresco-search/src/main/java/org/alfresco/solr/AlfrescoCoreAdminHandler.java
+43 -21
View File
@@ -19,8 +19,8 @@
package org.alfresco.solr;
import static java.util.Arrays.asList;
import static java.util.Optional.ofNullable;
import static org.alfresco.solr.HandlerOfResources.extractCustomProperties;
import static org.alfresco.solr.HandlerOfResources.getSafeBoolean;
import static org.alfresco.solr.HandlerOfResources.getSafeLong;
@@ -41,10 +41,20 @@ import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.lang.reflect.Method;
import java.util.*;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Properties;
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.TimeUnit;
import com.google.common.collect.ImmutableMap;
import org.alfresco.error.AlfrescoRuntimeException;
import org.alfresco.httpclient.AuthenticationException;
import org.alfresco.service.cmr.repository.StoreRef;
@@ -88,12 +98,17 @@ public class AlfrescoCoreAdminHandler extends CoreAdminHandler
private static final String ARG_QUERY = "query";
public static final String DATA_DIR_ROOT = "data.dir.root";
public static final String ALFRESCO_DEFAULTS = "create.alfresco.defaults";
public static final String NUM_SHARDS = "num.shards";
public static final String SHARD_IDS = "shard.ids";
public static final String DEFAULT_TEMPLATE = "rerank";
static final String ALFRESCO_CORE_NAME = "alfresco";
static final String ARCHIVE_CORE_NAME = "archive";
static final String VERSION_CORE_NAME = "version";
static final List<String> ALLOWED_CORE_NAMES = asList(ALFRESCO_CORE_NAME, ARCHIVE_CORE_NAME, VERSION_CORE_NAME);
static final Map<String, StoreRef> STORE_REF_MAP = ImmutableMap.of(
ALFRESCO_CORE_NAME, StoreRef.STORE_REF_WORKSPACE_SPACESSTORE,
ARCHIVE_CORE_NAME, StoreRef.STORE_REF_ARCHIVE_SPACESSTORE,
VERSION_CORE_NAME, new StoreRef("workspace", "version2Store"));
private SolrTrackerScheduler scheduler;
private TrackerRegistry trackerRegistry;
@@ -118,10 +133,12 @@ public class AlfrescoCoreAdminHandler extends CoreAdminHandler
LOGGER.info("Starting Alfresco core container services");
trackerRegistry = new TrackerRegistry();
informationServers = new ConcurrentHashMap<String, InformationServer>();
informationServers = new ConcurrentHashMap<>();
this.scheduler = new SolrTrackerScheduler(this);
String createDefaultCores = ConfigUtil.locateProperty(ALFRESCO_DEFAULTS, "");
int numShards = Integer.valueOf(ConfigUtil.locateProperty(NUM_SHARDS, "1"));
String shardIds = ConfigUtil.locateProperty(SHARD_IDS, null);
if (createDefaultCores != null && !createDefaultCores.isEmpty())
{
Runnable runnable = () ->
@@ -134,7 +151,7 @@ public class AlfrescoCoreAdminHandler extends CoreAdminHandler
{
//Don't care
}
setupNewDefaultCores(createDefaultCores);
setupNewDefaultCores(createDefaultCores, numShards, 1, 1, 1, shardIds);
};
Thread thread = new Thread(runnable);
@@ -149,7 +166,21 @@ public class AlfrescoCoreAdminHandler extends CoreAdminHandler
*/
void setupNewDefaultCores(String names)
{
setupNewDefaultCores(names, 1, 1, 1, 1, null);
}
/**
* Creates new default cores based on the "createDefaultCores" String passed in.
*
* @param names comma delimited list of core names that will be created.
* @param numShards The total number of shards.
* @param replicationFactor - Not sure why the core needs to know this.
* @param nodeInstance - Not sure why the core needs to know this.
* @param numNodes - Not sure why the core needs to know this.
* @param shardIds A comma separated list of shard ids for this core (or null).
*/
void setupNewDefaultCores(String names, int numShards, int replicationFactor, int nodeInstance, int numNodes, String shardIds)
{
try
{
List<String> coreNames =
@@ -163,23 +194,14 @@ public class AlfrescoCoreAdminHandler extends CoreAdminHandler
.map(String::trim)
.filter(coreName -> !coreName.isEmpty())
.forEach(coreName -> {
LOGGER.info("Attempting to create default alfresco core: "+coreName);
switch (coreName)
LOGGER.info("Attempting to create default alfresco core: {}", coreName);
if (!STORE_REF_MAP.keySet().contains(coreName))
{
case ARCHIVE_CORE_NAME:
newDefaultCore(ARCHIVE_CORE_NAME, StoreRef.STORE_REF_ARCHIVE_SPACESSTORE, DEFAULT_TEMPLATE, null, new SolrQueryResponse());
break;
case ALFRESCO_CORE_NAME:
newDefaultCore(ALFRESCO_CORE_NAME, StoreRef.STORE_REF_WORKSPACE_SPACESSTORE, DEFAULT_TEMPLATE, null, new SolrQueryResponse());
break;
case VERSION_CORE_NAME:
final String versionStoreProtocol = "workspace";
final String versionStoreId = "version2Store";
newDefaultCore(VERSION_CORE_NAME, new StoreRef(versionStoreProtocol, versionStoreId), DEFAULT_TEMPLATE, null, new SolrQueryResponse());
break;
default:
LOGGER.error("Invalid '" + ALFRESCO_DEFAULTS + "' permitted values are " + ALLOWED_CORE_NAMES);
throw new AlfrescoRuntimeException("Invalid '" + ALFRESCO_DEFAULTS + "' permitted values are " + STORE_REF_MAP.keySet());
}
StoreRef storeRef = STORE_REF_MAP.get(coreName);
newCore(coreName, numShards, storeRef, DEFAULT_TEMPLATE, replicationFactor, nodeInstance,
numNodes, shardIds, null, new SolrQueryResponse());
});
}
catch (Exception exception)
@@ -189,7 +211,7 @@ public class AlfrescoCoreAdminHandler extends CoreAdminHandler
}
/**
* Shutsdown services that exist outside of the core.
* Shut down services that exist outside of the core.
*/
public void shutdown()
{
search-services/alfresco-search/src/main/java/org/alfresco/solr/config/ConfigUtil.java
+1 -1
View File
@@ -52,7 +52,7 @@ public class ConfigUtil {
{
String propertyValue = null;
String propertyKey = propertyName.toLowerCase();
String propertyKey = propertyName;
String jndiKey = convertPropertyNameToJNDIPath(propertyKey);
String envVar = convertPropertyNameToEnvironmentParam(propertyKey);
search-services/alfresco-search/src/main/java/org/apache/solr/core/CoreDescriptorDecorator.java
+29 -5
View File
@@ -18,12 +18,14 @@
*/
package org.apache.solr.core;
import com.google.common.collect.ImmutableList;
import java.util.List;
import java.util.Properties;
import org.alfresco.solr.config.ConfigUtil;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import java.util.Properties;
import com.google.common.collect.ImmutableList;
/**
* This class was created solely for the purpose of exposing the coreProperties of the CoreDescriptor.
@@ -32,13 +34,16 @@ import java.util.Properties;
* The Substitutable Properties are defined in the substitutableProperties list.
* @author Ahmed Owian
* @author Gethin James
* @author aborroy
*/
public class CoreDescriptorDecorator
{
private static Log log = LogFactory.getLog(CoreDescriptorDecorator.class);
private final Properties properties = new Properties();
private static String SECURE_COMMS_PROPERTY = "alfresco.secureComms";
public static ImmutableList<String> substitutableProperties =
public static ImmutableList<String> substitutablePropertiesSecure =
ImmutableList.of(
"alfresco.host",
"alfresco.port",
@@ -54,18 +59,37 @@ public class CoreDescriptorDecorator
"alfresco.encryption.ssl.keystore.provider",
"alfresco.encryption.ssl.truststore.type");
public static ImmutableList<String> substitutablePropertiesNone =
ImmutableList.of(
"alfresco.host",
"alfresco.port",
"alfresco.baseUrl",
"alfresco.secureComms");
public CoreDescriptorDecorator(CoreDescriptor descriptor)
{
properties.putAll(descriptor.coreProperties);
List<String> coreProperties;
String comms = ConfigUtil.locateProperty(SECURE_COMMS_PROPERTY, "none");
if (comms.equals("https"))
{
coreProperties = substitutablePropertiesSecure;
}
else
{
coreProperties = substitutablePropertiesNone;
}
try
{
substitutableProperties.forEach(prop ->
coreProperties.forEach(prop ->
properties.put(prop, ConfigUtil.locateProperty(prop,properties.getProperty(prop)))
);
}
catch(Exception e)
{
log.warn("Unable to locate alfresco host|port|baseUrl|ssl properties");
log.warn("Unable to locate alfresco host|port|baseUrl|ssl properties", e);
}
}
search-services/alfresco-search/src/main/resources/solr/instance/templates/noRerank/conf/solrcore.properties
+3 -2
View File
@@ -33,9 +33,10 @@ alfresco.recordUnindexedNodes=false
# encryption
# none, https
alfresco.secureComms=none
alfresco.secureComms=https
# ssl
# ssl, default values
# keystore and trustore files are not provided by default
alfresco.encryption.ssl.keystore.type=JCEKS
alfresco.encryption.ssl.keystore.provider=
alfresco.encryption.ssl.keystore.location=ssl.repo.client.keystore
search-services/alfresco-search/src/main/resources/solr/instance/templates/noRerank/conf/ssl-keystore-passwords.properties
-7
View File
@@ -1,7 +0,0 @@
aliases=ssl.alfresco.ca,ssl.repo.client
# The ssl keystore password
keystore.password=kT9X6oe68t
# The password protecting the ssl repository key
ssl.repo.client.password=kT9X6oe68t
# The password protecting the ssl Alfresco CA key
ssl.alfresco.ca.password=kT9X6oe68t
search-services/alfresco-search/src/main/resources/solr/instance/templates/noRerank/conf/ssl-truststore-passwords.properties
-5
View File
@@ -1,5 +0,0 @@
aliases=alfresco.ca
# The ssl truststore password
keystore.password=kT9X6oe68t
# The password protecting the ssl Alfresco CA strust certificate
alfresco.ca.password=kT9X6oe68t
search-services/alfresco-search/src/main/resources/solr/instance/templates/noRerank/conf/ssl.repo.client.keystore
BIN
View File
Binary file not shown.
search-services/alfresco-search/src/main/resources/solr/instance/templates/noRerank/conf/ssl.repo.client.truststore
BIN
View File
Binary file not shown.
search-services/alfresco-search/src/main/resources/solr/instance/templates/rerank/conf/solrcore.properties
+3 -2
View File
@@ -33,9 +33,10 @@ alfresco.recordUnindexedNodes=false
# encryption
# none, https
alfresco.secureComms=none
alfresco.secureComms=https
# ssl
# ssl, default values
# keystore and trustore files are not provided by default
alfresco.encryption.ssl.keystore.type=JCEKS
alfresco.encryption.ssl.keystore.provider=
alfresco.encryption.ssl.keystore.location=ssl.repo.client.keystore
search-services/alfresco-search/src/main/resources/solr/instance/templates/rerank/conf/ssl-keystore-passwords.properties
-7
View File
@@ -1,7 +0,0 @@
aliases=ssl.alfresco.ca,ssl.repo.client
# The ssl keystore password
keystore.password=kT9X6oe68t
# The password protecting the ssl repository key
ssl.repo.client.password=kT9X6oe68t
# The password protecting the ssl Alfresco CA key
ssl.alfresco.ca.password=kT9X6oe68t
search-services/alfresco-search/src/main/resources/solr/instance/templates/rerank/conf/ssl-truststore-passwords.properties
-5
View File
@@ -1,5 +0,0 @@
aliases=alfresco.ca
# The ssl truststore password
keystore.password=kT9X6oe68t
# The password protecting the ssl Alfresco CA strust certificate
alfresco.ca.password=kT9X6oe68t
search-services/alfresco-search/src/main/resources/solr/instance/templates/rerank/conf/ssl.repo.client.keystore
BIN
View File
Binary file not shown.
search-services/alfresco-search/src/main/resources/solr/instance/templates/rerank/conf/ssl.repo.client.truststore
BIN
View File
Binary file not shown.
search-services/alfresco-search/src/test/java/org/alfresco/solr/AbstractAlfrescoDistributedTest.java
+61 -20
View File
@@ -142,6 +142,67 @@ public abstract class AbstractAlfrescoDistributedTest extends SolrTestInitialize
solrComparator.putHandleDefaults();
}
/**
* This method has the responsibility of checking that each shard has at least count documents
* @param query - query to execute
* @param count - min number of results each shard must satisfy
* @param waitMillis - total ms to wait
* @return
* @throws Exception
*/
public static boolean checkMinCountPerShard(Query query, int count, long waitMillis) throws SolrServerException,IOException
{
long begin = System.currentTimeMillis();
List<SolrClient> shardedClients = getShardedClients();
long timeout = begin + waitMillis;
boolean allShardCompliant = false;
for (SolrClient singleShard : shardedClients)
{
allShardCompliant = false;
int totalHits = 0;
int cycles = 1;
while ((new Date()).getTime() < timeout && (!allShardCompliant))
{
QueryResponse response = singleShard.query(luceneToSolrQuery(query));
totalHits = (int) response.getResults().getNumFound();
if ((long) totalHits >= count)
{
allShardCompliant = true;
}
try
{
Thread.sleep((long) (500 * cycles++));
}
catch (InterruptedException e)
{
continue;
}
}
}
return allShardCompliant;
}
public static void waitForShardsCount(ModifiableSolrParams query, int count, long waitMillis, long start) throws Exception
{
long timeOut = start+waitMillis;
int totalCount = 0;
query.set("shards", shards);
SolrClient clientShard = clientShards.get(0);
while (System.currentTimeMillis() < timeOut)
{
QueryResponse response = clientShard.query(query);
totalCount = (int) response.getResults().getNumFound();
if (totalCount == count)
{
return;
}
}
throw new Exception("Cluster:Wait error expected "+count+" found "+totalCount+" : "+query.toString());
}
/**
* Waits until all cores (including shards) reach a count.
*
@@ -304,26 +365,6 @@ public abstract class AbstractAlfrescoDistributedTest extends SolrTestInitialize
waitForShardsCount(solrQuery, count, waitMillis, start);
}
public static void waitForShardsCount(ModifiableSolrParams query, int count, long waitMillis, long start) throws Exception
{
long timeOut = start+waitMillis;
int totalCount = 0;
query.set("shards", shards);
SolrClient clientShard = clientShards.get(0);
while (System.currentTimeMillis() < timeOut)
{
QueryResponse response = clientShard.query(query);
totalCount = (int) response.getResults().getNumFound();
if (totalCount == count)
{
return;
}
}
throw new Exception("Cluster:Wait error expected "+count+" found "+totalCount+" : "+query.toString());
}
protected static void injectDocToShards(long txnId, long aclId, long dbId, String owner) throws Exception {
for(SolrClient clientShard : clientShards) {
SolrInputDocument doc = new SolrInputDocument();
search-services/alfresco-search/src/test/java/org/alfresco/solr/AlfrescoCoreAdminHandlerTest.java
+32 -22
View File
@@ -19,11 +19,25 @@
package org.alfresco.solr;
import static java.util.Arrays.asList;
import static org.alfresco.solr.AlfrescoCoreAdminHandler.ALFRESCO_CORE_NAME;
import static org.alfresco.solr.AlfrescoCoreAdminHandler.ARCHIVE_CORE_NAME;
import static org.alfresco.solr.AlfrescoCoreAdminHandler.ARG_TXID;
import static org.alfresco.solr.AlfrescoCoreAdminHandler.STORE_REF_MAP;
import static org.alfresco.solr.AlfrescoCoreAdminHandler.VERSION_CORE_NAME;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyInt;
import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.Mockito.*;
import static org.mockito.Mockito.never;
import static org.mockito.Mockito.reset;
import static org.mockito.Mockito.spy;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
import java.util.List;
import java.util.Properties;
import java.util.stream.Collectors;
import org.alfresco.service.cmr.repository.StoreRef;
import org.alfresco.solr.adapters.IOpenBitSet;
@@ -44,10 +58,6 @@ import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.junit.MockitoJUnitRunner;
import java.util.List;
import java.util.Properties;
import java.util.stream.Collectors;
/** Unit tests for {@link org.alfresco.solr.AlfrescoCoreAdminHandler}. */
@RunWith(MockitoJUnitRunner.class)
public class AlfrescoCoreAdminHandlerTest
@@ -181,13 +191,13 @@ public class AlfrescoCoreAdminHandlerTest
// First let's try a list of invalid names, one by one
List<String> invalidNames =
AlfrescoCoreAdminHandler.ALLOWED_CORE_NAMES.stream()
STORE_REF_MAP.keySet().stream()
.map(coreName -> coreName + System.currentTimeMillis())
.collect(Collectors.toList());
invalidNames.forEach(spy::setupNewDefaultCores);
verify(spy, never()).newDefaultCore(anyString(), any(), anyString(), any(), any());
verify(spy, never()).newCore(any(), anyInt(), any(), any(), anyInt(), anyInt(), anyInt(), any(), any(), any());
reset(spy);
@@ -195,14 +205,14 @@ public class AlfrescoCoreAdminHandlerTest
String commaSeparatedNames = String.join(",", invalidNames);
spy.setupNewDefaultCores(commaSeparatedNames);
verify(spy, never()).newDefaultCore(anyString(), any(), anyString(), any(), any());
verify(spy, never()).newCore(any(), anyInt(), any(), any(), anyInt(), anyInt(), anyInt(), any(), any(), any());
}
@Test
public void coreNamesAreTrimmed_oneCoreNameAtTime() {
AlfrescoCoreAdminHandler spy = spy(new AlfrescoCoreAdminHandler() {
@Override
protected boolean newDefaultCore(String coreName, StoreRef storeRef, String templateName, Properties extraProperties, SolrQueryResponse rsp)
protected boolean newCore(String coreName, int numShards, StoreRef storeRef, String templateName, int replicationFactor, int nodeInstance, int numNodes, String shardIds, Properties extraProperties, SolrQueryResponse rsp)
{
// Do nothing here otherwise we cannot spy it
return true;
@@ -212,23 +222,23 @@ public class AlfrescoCoreAdminHandlerTest
// First let's try a list of names, one by one
final List<String> coreNames =
asList(
AlfrescoCoreAdminHandler.ARCHIVE_CORE_NAME + " ", // whitespace char at the end
"\t " + AlfrescoCoreAdminHandler.ALFRESCO_CORE_NAME, // whitespace chars at the beginning
" " + AlfrescoCoreAdminHandler.VERSION_CORE_NAME + " \t", // beginning and end
ARCHIVE_CORE_NAME + " ", // whitespace char at the end
"\t " + ALFRESCO_CORE_NAME, // whitespace chars at the beginning
" " + VERSION_CORE_NAME + " \t", // beginning and end
" \t"); // empty name
coreNames.forEach(spy::setupNewDefaultCores);
verify(spy).newDefaultCore(eq(AlfrescoCoreAdminHandler.ARCHIVE_CORE_NAME), any(), anyString(), any(), any());
verify(spy).newDefaultCore(eq(AlfrescoCoreAdminHandler.ALFRESCO_CORE_NAME), any(), anyString(), any(), any());
verify(spy).newDefaultCore(eq(AlfrescoCoreAdminHandler.VERSION_CORE_NAME), any(), anyString(), any(), any());
verify(spy).newCore(eq(ARCHIVE_CORE_NAME), eq(1), eq(STORE_REF_MAP.get(ARCHIVE_CORE_NAME)), anyString(), eq(1), eq(1), eq(1), eq(null), eq(null), any());
verify(spy).newCore(eq(ALFRESCO_CORE_NAME), eq(1), eq(STORE_REF_MAP.get(ALFRESCO_CORE_NAME)), anyString(), eq(1), eq(1), eq(1), eq(null), eq(null), any());
verify(spy).newCore(eq(VERSION_CORE_NAME), eq(1), eq(STORE_REF_MAP.get(VERSION_CORE_NAME)), anyString(), eq(1), eq(1), eq(1), eq(null), eq(null), any());
}
@Test
public void validAndInvalidCoreNames() {
AlfrescoCoreAdminHandler spy = spy(new AlfrescoCoreAdminHandler() {
@Override
protected boolean newDefaultCore(String coreName, StoreRef storeRef, String templateName, Properties extraProperties, SolrQueryResponse rsp)
protected boolean newCore(String coreName, int numShards, StoreRef storeRef, String templateName, int replicationFactor, int nodeInstance, int numNodes, String shardIds, Properties extraProperties, SolrQueryResponse rsp)
{
// Do nothing here otherwise we cannot spy it
return true;
@@ -238,17 +248,17 @@ public class AlfrescoCoreAdminHandlerTest
// First let's try a list of names, one by one
final List<String> coreNames =
asList(
AlfrescoCoreAdminHandler.ARCHIVE_CORE_NAME + " ", // whitespace char at the end
"\t " + AlfrescoCoreAdminHandler.ALFRESCO_CORE_NAME, // whitespace chars at the beginning
" " + AlfrescoCoreAdminHandler.VERSION_CORE_NAME + " \t", // beginning and end
ARCHIVE_CORE_NAME + " ", // whitespace char at the end
"\t " + ALFRESCO_CORE_NAME, // whitespace chars at the beginning
" " + VERSION_CORE_NAME + " \t", // beginning and end
" \t"); // empty name
// Then, the same list as a single parameter (e.g. name1, name2, name3, etc)
String commaSeparatedNames = String.join(",", coreNames);
spy.setupNewDefaultCores(commaSeparatedNames);
verify(spy).newDefaultCore(eq(AlfrescoCoreAdminHandler.ARCHIVE_CORE_NAME), any(), anyString(), any(), any());
verify(spy).newDefaultCore(eq(AlfrescoCoreAdminHandler.ALFRESCO_CORE_NAME), any(), anyString(), any(), any());
verify(spy).newDefaultCore(eq(AlfrescoCoreAdminHandler.VERSION_CORE_NAME), any(), anyString(), any(), any());
verify(spy).newCore(eq(ARCHIVE_CORE_NAME), eq(1), eq(STORE_REF_MAP.get(ARCHIVE_CORE_NAME)), anyString(), eq(1), eq(1), eq(1), eq(null), eq(null), any());
verify(spy).newCore(eq(ALFRESCO_CORE_NAME), eq(1), eq(STORE_REF_MAP.get(ALFRESCO_CORE_NAME)), anyString(), eq(1), eq(1), eq(1), eq(null), eq(null), any());
verify(spy).newCore(eq(VERSION_CORE_NAME), eq(1), eq(STORE_REF_MAP.get(VERSION_CORE_NAME)), anyString(), eq(1), eq(1), eq(1), eq(null), eq(null), any());
}
}
search-services/alfresco-search/src/test/java/org/alfresco/solr/SolrTestInitializer.java
+13
View File
@@ -121,6 +121,7 @@ public abstract class SolrTestInitializer extends SolrTestCaseJ4
public static void initSolrServers(int numShards, String testClassName, Properties solrcoreProperties) throws Throwable
{
solrcoreProperties = addExplicitShardingProperty(solrcoreProperties);
clientShards = new ArrayList<>();
solrShards = new ArrayList<>();
solrCollectionNameToStandaloneClient = new HashMap<>();
@@ -137,6 +138,18 @@ public abstract class SolrTestInitializer extends SolrTestCaseJ4
System.setProperty("solr.solr.home", testDir.toPath().resolve(serverName).toString());
}
private static Properties addExplicitShardingProperty(Properties solrcoreProperties)
{
if(solrcoreProperties == null){
solrcoreProperties = new Properties();
}
if(solrcoreProperties.getProperty("shard.method")==null)
{
solrcoreProperties.put("shard.method", "EXPLICIT_ID");
}
return solrcoreProperties;
}
public static void initSingleSolrServer(String testClassName, Properties solrcoreProperties) throws Throwable {
initSolrServers(0,testClassName,solrcoreProperties);
search-services/alfresco-solrclient-lib/pom.xml
+2 -2
View File
@@ -22,7 +22,7 @@
</distributionManagement>
<properties>
<dependency.alfresco-data-model.version>8.34</dependency.alfresco-data-model.version>
<dependency.alfresco-data-model.version>8.39</dependency.alfresco-data-model.version>
<dependency.jackson.version>2.9.9</dependency.jackson.version>
</properties>
@@ -61,7 +61,7 @@
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<version>2.27.0</version>
<version>2.28.2</version>
<scope>test</scope>
</dependency>
<dependency>
search-services/alfresco-solrclient-lib/src/main/java/org/alfresco/solr/client/SOLRAPIClientFactory.java
+88 -31
View File
@@ -38,6 +38,7 @@ import org.alfresco.httpclient.HttpClientFactory;
import org.alfresco.httpclient.HttpClientFactory.SecureCommsType;
import org.alfresco.repo.dictionary.NamespaceDAO;
import org.alfresco.service.cmr.dictionary.DictionaryService;
import org.apache.commons.httpclient.params.DefaultHttpParams;
/**
* This factory encapsulates the creation of a SOLRAPIClient and the management of that resource.
@@ -53,10 +54,6 @@ public class SOLRAPIClientFactory
// encryption related parameters
private String secureCommsType; // "none", "https"
private String keyStoreType;
private String keyStoreProvider;
private String passwordFileLocation;
private String keyStoreLocation;
// ssl
private String sslKeyStoreType;
@@ -151,24 +148,25 @@ public class SOLRAPIClientFactory
if (client == null)
{
baseUrl = props.getProperty("alfresco.baseUrl", "/alfresco");
keyStoreType = props.getProperty("alfresco.encryption.keystore.type", "JCEKS");
keyStoreProvider = props.getProperty("alfresco.encryption.keystore.provider");
passwordFileLocation = props.getProperty("alfresco.encryption.keystore.passwordFileLocation");
keyStoreLocation = props.getProperty("alfresco.encryption.keystore.location");
sslKeyStoreType = props.getProperty("alfresco.encryption.ssl.keystore.type", "JCEKS");
sslKeyStoreProvider = props.getProperty("alfresco.encryption.ssl.keystore.provider", "");
sslKeyStoreLocation = props.getProperty("alfresco.encryption.ssl.keystore.location",
"ssl.repo.client.keystore");
sslKeyStorePasswordFileLocation = props.getProperty(
"alfresco.encryption.ssl.keystore.passwordFileLocation", "ssl-keystore-passwords.properties");
sslTrustStoreType = props.getProperty("alfresco.encryption.ssl.truststore.type", "JCEKS");
sslTrustStoreProvider = props.getProperty("alfresco.encryption.ssl.truststore.provider", "");
sslTrustStoreLocation = props.getProperty("alfresco.encryption.ssl.truststore.location",
"ssl.repo.client.truststore");
sslTrustStorePasswordFileLocation = props.getProperty(
"alfresco.encryption.ssl.truststore.passwordFileLocation",
"ssl-truststore-passwords.properties");
// Load SSL settings only when using HTTPs protocol
secureCommsType = props.getProperty("alfresco.secureComms", "none");
if (secureCommsType.equals("https"))
{
sslKeyStoreType = getProperty(props, "alfresco.encryption.ssl.keystore.type", "JCEKS");
sslKeyStoreProvider = getProperty(props, "alfresco.encryption.ssl.keystore.provider", "");
sslKeyStoreLocation = getProperty(props, "alfresco.encryption.ssl.keystore.location",
"ssl.repo.client.keystore");
sslKeyStorePasswordFileLocation = getProperty(props,
"alfresco.encryption.ssl.keystore.passwordFileLocation",
"ssl-keystore-passwords.properties");
sslTrustStoreType = getProperty(props, "alfresco.encryption.ssl.truststore.type", "JCEKS");
sslTrustStoreProvider = getProperty(props, "alfresco.encryption.ssl.truststore.provider", "");
sslTrustStoreLocation = getProperty(props, "alfresco.encryption.ssl.truststore.location",
"ssl.repo.client.truststore");
sslTrustStorePasswordFileLocation = getProperty(props,
"alfresco.encryption.ssl.truststore.passwordFileLocation",
"ssl-truststore-passwords.properties");
}
maxTotalConnections = Integer.parseInt(props.getProperty("alfresco.maxTotalConnections", "40"));
maxHostConnections = Integer.parseInt(props.getProperty("alfresco.maxHostConnections", "40"));
socketTimeout = Integer.parseInt(props.getProperty("alfresco.socketTimeout", "60000"));
@@ -182,20 +180,79 @@ public class SOLRAPIClientFactory
protected AlfrescoHttpClient getRepoClient(KeyResourceLoader keyResourceLoader)
{
// TODO i18n
KeyStoreParameters keyStoreParameters = new KeyStoreParameters("SSL Key Store", sslKeyStoreType,
sslKeyStoreProvider, sslKeyStorePasswordFileLocation, sslKeyStoreLocation);
KeyStoreParameters trustStoreParameters = new KeyStoreParameters("SSL Trust Store", sslTrustStoreType,
sslTrustStoreProvider, sslTrustStorePasswordFileLocation, sslTrustStoreLocation);
SSLEncryptionParameters sslEncryptionParameters = new SSLEncryptionParameters(keyStoreParameters,
trustStoreParameters);
HttpClientFactory httpClientFactory = new HttpClientFactory(SecureCommsType.getType(secureCommsType),
HttpClientFactory httpClientFactory = null;
if (secureCommsType.equals("https"))
{
KeyStoreParameters keyStoreParameters = new KeyStoreParameters("SSL Key Store", sslKeyStoreType,
sslKeyStoreProvider, sslKeyStorePasswordFileLocation, sslKeyStoreLocation);
KeyStoreParameters trustStoreParameters = new KeyStoreParameters("SSL Trust Store", sslTrustStoreType,
sslTrustStoreProvider, sslTrustStorePasswordFileLocation, sslTrustStoreLocation);
SSLEncryptionParameters sslEncryptionParameters = new SSLEncryptionParameters(keyStoreParameters,
trustStoreParameters);
httpClientFactory = new HttpClientFactory(SecureCommsType.getType(secureCommsType),
sslEncryptionParameters, keyResourceLoader, null, null, alfrescoHost, alfrescoPort,
alfrescoPortSSL, maxTotalConnections, maxHostConnections, socketTimeout);
// TODO need to make port configurable depending on secure comms, or just make redirects work
}
else
{
httpClientFactory = new PlainHttpClientFactory(alfrescoHost, alfrescoPort, maxTotalConnections, maxHostConnections);
}
AlfrescoHttpClient repoClient = httpClientFactory.getRepoClient(alfrescoHost, alfrescoPortSSL);
repoClient.setBaseUrl(baseUrl);
return repoClient;
}
/**
* Return property value from system (passed as -D argument).
* If the system property does not exists, return local value from solrcore.properties
* If the local property does not exists, return default value
*
* @param props Local properties file (solrcore.properties)
* @param key The property key
* @return The value
*/
private String getProperty(Properties props, String key, String defaultValue)
{
String value = System.getProperties().getProperty(key);
if (value == null)
{
value = props.getProperty(key);
}
if (value == null)
{
value = defaultValue;
}
return value;
}
/**
* Local class to avoid loading sslEntryptionParameters for plain http connections.
*
* @author aborroy
*
*/
class PlainHttpClientFactory extends HttpClientFactory
{
public PlainHttpClientFactory(String host, int port, int maxTotalConnections, int maxHostConnections)
{
setSecureCommsType("none");
setHost(host);
setPort(port);
setMaxTotalConnections(maxTotalConnections);
setMaxHostConnections(maxHostConnections);
init();
}
@Override
public void init()
{
DefaultHttpParams.setHttpParamsFactory(new NonBlockingHttpParamsFactory());
}
}
}
search-services/packaging/src/docker/6.x/.env
+2 -2
View File
@@ -1,5 +1,5 @@
# In order to start the community edition set the value of ALFRESCO_EDITION to -community
ALFRESCO_EDITION=
# In order to start the community edition set the value of ALFRESCO_IMAGE to alfresco/alfresco-content-repository-community
ALFRESCO_IMAGE=alfresco/alfresco-content-repository
ALFRESCO_TAG=6.1.0-EA3
SHARE_TAG=6.0
POSTGRES_TAG=10.1
search-services/packaging/src/docker/6.x/docker-compose.yml
+3 -1
View File
@@ -1,7 +1,7 @@
version: '3'
services:
alfresco:
image: alfresco/alfresco-content-repository${ALFRESCO_EDITION}:${ALFRESCO_TAG}
image: ${ALFRESCO_IMAGE}:${ALFRESCO_TAG}
environment:
JAVA_OPTS : "
-Ddb.driver=org.postgresql.Driver
@@ -45,6 +45,8 @@ services:
- SOLR_SOLR_PORT=8983
#Create the default alfresco and archive cores
- SOLR_CREATE_ALFRESCO_DEFAULTS=alfresco,archive
#HTTP by default
- ALFRESCO_SECURE_COMMS=none
ports:
- 8083:8983 #Browser port
activemq:
search-services/packaging/src/docker/Dockerfile
+2
View File
@@ -42,6 +42,8 @@ WORKDIR $DIST_DIR
VOLUME $DIST_DIR/data
VOLUME $DIST_DIR/solrhome
# Expose a folder to mount keystores in the host (required for Mutual TLS Auth)
VOLUME $DIST_DIR/keystores
EXPOSE 8983
USER ${USERNAME}
search-services/packaging/src/docker/search_config_setup.sh
+14
View File
@@ -18,4 +18,18 @@ if [[ ! -z "$SOLR_JAVA_MEM" ]]; then
sed -i -e "s/.*SOLR_JAVA_MEM=.*/SOLR_JAVA_MEM=\"$SOLR_JAVA_MEM\"/g" $SOLR_IN_FILE
fi
# By default Docker Image is using TLS Mutual Authentication (SSL) for communications with Repository
# Plain HTTP can be enabled by setting ALFRESCO_SECURE_COMMS to 'none'
if [[ "none" == "$ALFRESCO_SECURE_COMMS" ]]; then
sed -i 's/alfresco.secureComms=https/alfresco.secureComms=none/' ${PWD}/solrhome/templates/rerank/conf/solrcore.properties
sed -i 's/alfresco.secureComms=https/alfresco.secureComms=none/' ${PWD}/solrhome/templates/noRerank/conf/solrcore.properties
# Apply also the setting to existing SOLR cores property files when existing
if [[ -f ${PWD}/solrhome/alfresco/conf/solrcore.properties ]]; then
sed -i 's/alfresco.secureComms=https/alfresco.secureComms=none/' ${PWD}/solrhome/alfresco/conf/solrcore.properties
fi
if [[ -f ${PWD}/solrhome/archive/conf/solrcore.properties ]]; then
sed -i 's/alfresco.secureComms=https/alfresco.secureComms=none/' ${PWD}/solrhome/archive/conf/solrcore.properties
fi
fi
bash -c "$@"
search-services/packaging/src/main/resources/README.MD
+5 -17
View File
@@ -1,28 +1,16 @@
## Alfresco Search Services
Alfresco Search Services ${project.version} using Apache Solr ${solr.version}
## Solr 6
Solr 6 is now available as a drop in replacement for Solr 4 with Alfresco 5.2.
Find out more about Solr 6 and how to be installed at:
Solr 6 brings with it a number of new features:
* Community users: https://docs.alfresco.com/search-community/concepts/search-home.html
- SHARDING by DATE, PROPERTY & DBID
- FINGERPRINTS to find similar documents
- Indexing Multiple Document Versions
- Full SSL support with sharding
* Enterprise users: https://docs.alfresco.com/search-enterprise/concepts/search-home.html
Find out more about Solr 6 and how it differs here: http://docs.alfresco.com/5.2/concepts/solr6-home.html
### Install & Run
**Please read the [Installation Documentation](http://docs.alfresco.com/5.2/concepts/solr6-install-config.html).**
To run, unzip then
```
solr/bin/solr start
```
### License
Copyright (C) 2005 - 2017 Alfresco Software Limited
Copyright (C) 2005 - 2019 Alfresco Software Limited
This file is part of the Alfresco software.
If the software was purchased under a paid Alfresco license, the terms of
search-services/packaging/src/main/resources/licenses/notice.txt
+10 -10
View File
@@ -58,16 +58,16 @@ quartz-2.3.1.jar http://quartz-scheduler.org/
jackson-core-2.9.9.jar https://github.com/FasterXML/jackson
jackson-annotations-2.9.9.jar https://github.com/FasterXML/jackson
commons-httpclient-3.1-HTTPCLIENT-1265.jar http://jakarta.apache.org/commons/
spring-aop-5.1.6.RELEASE.jar http://projects.spring.io/spring-framework/
spring-beans-5.1.6.RELEASE.jar http://projects.spring.io/spring-framework/
spring-context-5.1.6.RELEASE.jar http://projects.spring.io/spring-framework/
spring-context-support-5.1.6.RELEASE.jar http://projects.spring.io/spring-framework/
spring-core-5.1.6.RELEASE.jar http://projects.spring.io/spring-framework/
spring-expression-5.1.6.RELEASE.jar http://projects.spring.io/spring-framework/
spring-jcl-5.1.6.RELEASE.jar http://projects.spring.io/spring-framework/
spring-jdbc-5.1.6.RELEASE.jar http://projects.spring.io/spring-framework/
spring-orm-5.1.6.RELEASE.jar http://projects.spring.io/spring-framework/
spring-tx-5.1.6.RELEASE.jar http://projects.spring.io/spring-framework/
spring-aop-5.1.8.RELEASE.jar http://projects.spring.io/spring-framework/
spring-beans-5.1.8.RELEASE.jar http://projects.spring.io/spring-framework/
spring-context-5.1.8.RELEASE.jar http://projects.spring.io/spring-framework/
spring-context-support-5.1.8.RELEASE.jar http://projects.spring.io/spring-framework/
spring-core-5.1.8.RELEASE.jar http://projects.spring.io/spring-framework/
spring-expression-5.1.8.RELEASE.jar http://projects.spring.io/spring-framework/
spring-jcl-5.1.8.RELEASE.jar http://projects.spring.io/spring-framework/
spring-jdbc-5.1.8.RELEASE.jar http://projects.spring.io/spring-framework/
spring-orm-5.1.8.RELEASE.jar http://projects.spring.io/spring-framework/
spring-tx-5.1.8.RELEASE.jar http://projects.spring.io/spring-framework/
spring-web-5.1.6.RELEASE.jar http://projects.spring.io/spring-framework/
xercesImpl-2.11.0-alfresco-patched-20180402.jar http://xerces.apache.org/xerces2-j
guessencoding-1.4.jar http://docs.codehaus.org/display/GUESSENC/