inteligr8/SearchServices
1
0
Fork 0
mirror of https://github.com/Alfresco/SearchServices.git synced 2026-09-16 18:12:56 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'master' of https://git.alfresco.com/search_discovery/insightengine into feature/SEARCH-1681_ShardingOnTheCITests

Browse Source
This commit is contained in:
Keerat
2020-02-03 13:04:04 +00:00
parent 713866b448 fa497a3778
commit 74fc4e4edd
91 changed files with 3107 additions and 1520 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CONTRIBUTING.md
+67
View File
@@ -0,0 +1,67 @@
# Contributing
Thanks for your interest in contributing to this project!
The following is a set of guidelines for contributing to this library. Most of them will
make the life of the reviewer easier and therefore decrease the time required for the
patch be included in the next version.
Alfresco has an [active forum](http://community.alfresco.com/community/ecm) to support
community users of our products. If you have any questions then this is the fastest method
of getting an answer.
## Raising issues
If you want to raise an issue then please use the [issue tracker on GitHub](https://github.com/Alfresco/SearchServices/issues).
We may convert these to Jira tickets before working on them, as this is the system we use
internally for tracking development. The Jira project for this codebase is [SEARCH](https://issues.alfresco.com/jira/projects/SEARCH/issues)
and you may want to look here first to see if your issue has previously been encountered.[^jiraaccess]
There are some guidelines for raising a good issue [here](https://hub.alfresco.com/t5/alfresco-content-services-hub/reporting-an-issue/ba-p/289727).
[^jiraaccess]: Note that while we try to keep our Jira issues visible to everyone, some
are restricted as they relate to specific customers or security issues. Some older issues
are also restricted simply because we have not been through to check if they contain
sensitive information or not.
## Submitting changes
We have a [coding standards guidelines page](https://hub.alfresco.com/t5/alfresco-content-services-hub/coding-standards-for-alfresco-content-services/ba-p/290457)
although you will find numerous examples where we have not adhered to them. Please try to
maintain consistency with the guidelines for new code, but avoid reformatting large
blocks of code if these are not related to your change.
## Branches
Our codebase consists of long-lived release branches and short-lived feature branches. The
code that we expect to include in the next minor version is stored on `master`. All other
release branches have the prefix `release/`. Feature branches may have any other prefix,
but we usually use `feature/` or `fix/`. We expect code on release branches to be ready
to release, and in the rare occasion when a release branch is broken then we try to revert
changes to fix the branch as soon as possible.
As bug fixes often also need a change to ACS then we use a cherry-pick strategy to get the
fix to all necessary release branches. The fix should initially be merged to `master` and
it can then be cherry-picked back by using:
```git cherry-pick -x -m 1 [mergeCommitId]```
## Community Mirror
Pull requests to our community mirror will be accepted in our enterprise codebase and then
mirrored back to the community. You will always be credited with your commits, although if
you [sign your commits](https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work) then the
signature will be stripped by the mirroring process.[^dependabot]
[^dependabot]: This is the reason that pull requests submitted by Dependabot appear closed
rather than merged.
## Builds
Our builds are currently in our internal Bamboo instance. If you have access to the
internal code then you can find some more links in the [insight engine module](insight-engine/CONTRIBUTING.md).
Although the build results are not visible externally, it should be possible to run most of
the tests locally. We have divided our tests into unit tests, integration tests and
end-to-end tests. The unit and integration tests can be run using the maven `test` and
`verify` goals respectively. The end-to-end tests cannot currently be run externally as
they require some dependencies stored in our internal Nexus.
README.md
+4
View File
@@ -41,3 +41,7 @@ More details are available at [search-services](/search-services) folder.
**Following resources will not be available for Community users**
More details are available at [insight-engine](/insight-engine) folder.
## Contributing guide
Please use [this guide](CONTRIBUTING.md) to make a contribution to the project.
e2e-test/generator-alfresco-docker-compose/README.md
+2 -1
View File
@@ -111,7 +111,8 @@ When using Community, some different options can be combined:
? Would you like to use HTTP or mTLS for Alfresco-SOLR communication? http
? Would you like to use HTTP or HTTPs for Web Proxy? http
? Would you like to protect the access to SOLR REST API? Yes
? Would you like to use a SOLR Replication (2 nodes in master-slave)? No
? Would you like to use a SOLR Replication? No
? Would you like to compress Get Content responses? No
```
**Note** that when choosing *mTLS* or *HTTPs*, default certificates, truststores and keystores are provided for testing purposes. If you are planning to use this Docker Compose template for real environments, replace these cryptographic stores with another generated by yourself to increase the security of your system.
e2e-test/generator-alfresco-docker-compose/generators/app/index.js
+10 -2
View File
@@ -56,7 +56,7 @@ module.exports = class extends Generator {
type: 'confirm',
name: 'protectSolr',
message: 'Would you like to protect the access to SOLR REST API?',
default: 'true'
default: true
},
{
whenFunction: response => response.httpMode == 'http',
@@ -69,6 +69,13 @@ module.exports = class extends Generator {
{ name: "Yes - two nodes in a master-master configuration", value: "master-master" }
]
},
{
whenFunction: response => response.acsVersion == '6.2',
type: 'confirm',
name: 'gzip',
message: 'Would you like to compress Get Content responses?',
default: false
},
// Enterprise only options
{
whenFunction: response => response.alfrescoVersion == 'enterprise' && !response.replication,
@@ -208,7 +215,8 @@ module.exports = class extends Generator {
searchPath: searchBasePath,
zeppelin: (this.props.zeppelin ? "true" : "false"),
sharding: (this.props.sharding ? "true" : "false"),
shardingMethod: (this.props.shardingMethod)
shardingMethod: (this.props.shardingMethod),
gzip: (this.props.gzip ? "true" : "false")
}
);
e2e-test/generator-alfresco-docker-compose/generators/app/templates/6.2/.env
+9 -9
View File
@@ -1,15 +1,15 @@
ALFRESCO_TAG=latest
SHARE_TAG=6.1.0-RC3
POSTGRES_TAG=10.9
TRANSFORM_ROUTER_TAG=1.1.0-EA2
PDF_RENDERER_TAG=2.1.0-EA4
IMAGE_MAGICK_TAG=2.1.0-EA4
LIBREOFFICE_TAG=2.1.0-EA4
TIKA_TAG=2.1.0-EA4
TRANSFORM_MISC_TAG=2.1.0-EA4
SHARE_TAG=6.2.0
POSTGRES_TAG=11.4
TRANSFORM_ROUTER_TAG=1.1.0
PDF_RENDERER_TAG=2.1.0
IMAGE_MAGICK_TAG=2.1.0
LIBREOFFICE_TAG=2.1.0
TIKA_TAG=2.1.0
TRANSFORM_MISC_TAG=2.1.0
SHARED_FILE_STORE_TAG=0.5.3
ACTIVE_MQ_TAG=5.15.8
DIGITAL_WORKSPACE_TAG=1.1.0
DIGITAL_WORKSPACE_TAG=1.3.0
ACS_NGINX_TAG=3.0.1
ACS_COMMUNITY_NGINX_TAG=1.0.0
SEARCH_TAG=latest
e2e-test/generator-alfresco-docker-compose/generators/app/templates/6.2/docker-compose-ce.yml
+3
View File
@@ -13,6 +13,7 @@ services:
TRUSTSTORE_PASS: kT9X6oe68t
KEYSTORE_TYPE: JCEKS
KEYSTORE_PASS: kT9X6oe68t <% } %>
COMPRESS_CONTENT: "<%=gzip%>"
mem_limit: 1800m
environment:
JAVA_OPTS : "
@@ -66,6 +67,7 @@ services:
KEYSTORE_TYPE: JCEKS <% } %> <% if (replication) { %>
ENABLE_MASTER: "true"
ENABLE_SLAVE: "false" <% } %>
COMPRESS_CONTENT: "<%=gzip%>"
mem_limit: 1200m
environment:
#Solr needs to know how to register itself with Alfresco
@@ -108,6 +110,7 @@ services:
ENABLE_MASTER: <% if (replication == 'master-master') { %>"true"<% } else { %>"false"<% } %>
ENABLE_SLAVE: <% if (replication == 'master-master') { %>"false"<% } else { %>"true"<% } %>
MASTER_HOST: solr6 <% } %>
COMPRESS_CONTENT: "<%=gzip%>"
mem_limit: 1200m
environment:
#Solr needs to know how to register itself with Alfresco
e2e-test/generator-alfresco-docker-compose/generators/app/templates/images/alfresco/Dockerfile
+15 -1
View File
@@ -40,13 +40,27 @@ RUN if [ "$SOLR_COMMS" == "https" ] ; then \
[[:space:]]\+<\/Engine>/\n\
<\/Engine>\n\
<Connector port=\"8443\" protocol=\"org.apache.coyote.http11.Http11Protocol\"\n\
connectionTimeout=\"20000\"\n\
SSLEnabled=\"true\" maxThreads=\"150\" scheme=\"https\"\n\
keystoreFile=\"\/usr\/local\/tomcat\/alf_data\/keystore\/ssl.keystore\"\n\
keystorePass=\"${KEYSTORE_PASS}\" keystoreType=\"${KEYSTORE_TYPE}\" secure=\"true\" connectionTimeout=\"240000\"\n\
keystorePass=\"${KEYSTORE_PASS}\" keystoreType=\"${KEYSTORE_TYPE}\" secure=\"true\"\n\
truststoreFile=\"\/usr\/local\/tomcat\/alf_data\/keystore\/ssl.truststore\"\n\
truststorePass=\"${TRUSTSTORE_PASS}\" truststoreType=\"${TRUSTSTORE_TYPE}\" clientAuth=\"want\" sslProtocol=\"TLS\">\n\
<\/Connector>/g" ${TOMCAT_DIR}/conf/server.xml; \
fi
# GZIP COMPRESSION
ARG COMPRESS_CONTENT
ENV COMPRESS_CONTENT $COMPRESS_CONTENT
RUN if [ "$COMPRESS_CONTENT" == "true" ] ; then \
sed -i "s/\
[[:space:]]\+connectionTimeout=\"20000\"/\n\
connectionTimeout=\"20000\"\n\
compression=\"on\"\n\
compressionMinSize=\"1\"\n\
/g" ${TOMCAT_DIR}/conf/server.xml; \
fi
# Copy custom content model to deployment folder
COPY model/* $TOMCAT_DIR/shared/classes/alfresco/extension/
e2e-test/generator-alfresco-docker-compose/generators/app/templates/images/search/Dockerfile
+8
View File
@@ -125,6 +125,14 @@ RUN if [ "$ENABLE_SHARDING" == "true" ] ; then \
fi; \
fi
# GZIP COMPRESSION
ARG COMPRESS_CONTENT
ENV COMPRESS_CONTENT $COMPRESS_CONTENT
RUN if [ "$COMPRESS_CONTENT" == "true" ] ; then \
sed -i '/^bash.*/i sed -i "'"s/solr.request.content.compress=false/solr.request.content.compress=true/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
# Useless for 'none'/'http' communications with Alfresco
RUN mkdir ${DIST_DIR}/keystore \
&& chown -R solr:solr ${DIST_DIR}/keystore
e2e-test/pom.xml
+17 -6
View File
@@ -11,10 +11,11 @@
<name>Search Analytics E2E Tests</name>
<description>Test Project to test Search Service and Analytics Features on a complete setup of Alfresco, Share</description>
<properties>
<tas.rest.api.version>6.0.1.2</tas.rest.api.version>
<tas.cmis.api.version>6.0.0.4</tas.cmis.api.version>
<tas.utility.version>3.0.14</tas.utility.version>
<rm.version>3.2.0</rm.version>
<tas.rest.api.version>1.28</tas.rest.api.version>
<tas.rest.api.version>1.26</tas.rest.api.version>
<tas.cmis.api.version>1.13</tas.cmis.api.version>
<tas.utility.version>3.0.19</tas.utility.version>
<rm.version>3.3.0</rm.version>
<suiteXmlFile>src/test/resources/SearchSuite.xml</suiteXmlFile>
<test.exclude></test.exclude>
<test.include></test.include>
@@ -44,7 +45,7 @@
</dependency>
<dependency>
<groupId>org.alfresco.tas</groupId>
<artifactId>restapi-test</artifactId>
<artifactId>restapi</artifactId>
<version>${tas.rest.api.version}</version>
<scope>test</scope>
<exclusions>
@@ -56,7 +57,7 @@
</dependency>
<dependency>
<groupId>org.alfresco.tas</groupId>
<artifactId>cmis-test</artifactId>
<artifactId>cmis</artifactId>
<version>${tas.cmis.api.version}</version>
<scope>test</scope>
<exclusions>
@@ -71,6 +72,12 @@
<artifactId>alfresco-governance-services-automation-enterprise-rest-api</artifactId>
<version>${rm.version}</version>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.alfresco.tas</groupId>
<artifactId>restapi-test</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
@@ -88,6 +95,10 @@
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
</exclusion>
<exclusion>
<groupId>org.alfresco.tas</groupId>
<artifactId>restapi-test</artifactId>
</exclusion>
</exclusions>
</dependency>
<!-- alfresco tester settings -->
e2e-test/src/main/java/org/alfresco/search/TestGroup.java
+1
View File
@@ -38,6 +38,7 @@ public class TestGroup
public static final String ACS_61n = "ACS_61n"; // Alfresco Content Services 6.1 or above
public static final String ACS_611n = "ACS_611n"; // Alfresco Content Services 6.1.1 or above
public static final String ACS_62n = "ACS_62n"; // Alfresco Content Services 6.2 or above
public static final String ACS_63n = "ACS_63n"; // Alfresco Content Services 6.3 or above
public static final String AGS_302 = "AGS_302"; // Alfresco governance Services 3.0.2 or above
e2e-test/src/test/java/org/alfresco/test/search/functional/AbstractE2EFunctionalTest.java
+116 -65
View File
@@ -6,6 +6,16 @@
*/
package org.alfresco.test.search.functional;
import static java.util.Optional.ofNullable;
import static lombok.AccessLevel.PROTECTED;
import static org.testng.Assert.assertEquals;
import java.util.List;
import java.util.Set;
import java.util.stream.Collectors;
import lombok.Getter;
import org.alfresco.cmis.CmisWrapper;
import org.alfresco.dataprep.ContentService;
import org.alfresco.dataprep.SiteService.Visibility;
@@ -38,19 +48,14 @@ import org.springframework.test.context.testng.AbstractTestNGSpringContextTests;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.BeforeSuite;
import lombok.Getter;
import static java.util.Optional.ofNullable;
import static lombok.AccessLevel.PROTECTED;
/**
* @author meenal bhave
*/
@ContextConfiguration("classpath:alfresco-search-e2e-context.xml")
@ContextConfiguration ("classpath:alfresco-search-e2e-context.xml")
public abstract class AbstractE2EFunctionalTest extends AbstractTestNGSpringContextTests
{
/** The number of retries that a query will be tried before giving up. */
private static final int SEARCH_MAX_ATTEMPTS = 6;
protected static final int SEARCH_MAX_ATTEMPTS = 6;
private static final Logger LOGGER = LogFactory.getLogger();
@@ -76,26 +81,26 @@ public abstract class AbstractE2EFunctionalTest extends AbstractTestNGSpringCont
protected CmisWrapper cmisApi;
@Autowired
// @Getter(value = PROTECTED)
// @Getter(value = PROTECTED)
protected DataUser dataUser;
@Autowired
@Getter(value = PROTECTED)
@Getter (value = PROTECTED)
private ContentService contentService;
protected UserModel testUser, adminUserModel, testUser2;
protected SiteModel testSite, testSite2;
protected static String unique_searchString;
protected static final String SEARCH_LANGUAGE_CMIS = "cmis";
protected enum SearchLanguage {
CMIS,
AFTS
}
}
@BeforeSuite(alwaysRun = true)
@BeforeSuite (alwaysRun = true)
public void beforeSuite() throws Exception
{
super.springTestContextPrepareTestInstance();
@@ -105,7 +110,7 @@ public abstract class AbstractE2EFunctionalTest extends AbstractTestNGSpringCont
deployCustomModel("model/sharding-content-model.xml");
}
@BeforeClass(alwaysRun = true)
@BeforeClass (alwaysRun = true)
public void setup()
{
serverHealth.assertServerIsOnline();
@@ -210,7 +215,7 @@ public abstract class AbstractE2EFunctionalTest extends AbstractTestNGSpringCont
customModel.setNodeRef(modelInRepo.getId());
customModel.setNodeRef(customModel.getNodeRefWithoutVersion());
customModel.setCmisLocation(String.format("/Data Dictionary/Models/%s", fileName));
LOGGER.info("Custom Model file: " + customModel.getCmisLocation());
LOGGER.info("Custom Model file: " + customModel.getCmisLocation());
}
else
{
@@ -240,9 +245,9 @@ public abstract class AbstractE2EFunctionalTest extends AbstractTestNGSpringCont
/**
* Wait for Solr to finish indexing and search to return appropriate results
*
* @param userQuery Search Query
* @param contentToFind that's expected to be included / excluded from the results
*
* @param userQuery Search Query
* @param contentToFind that's expected to be included / excluded from the results
* @param expectedInResults Whether we expect the content in the results or not.
* @return true if search returns expected results, i.e. is given content is found or excluded from the results
*/
@@ -278,26 +283,25 @@ public abstract class AbstractE2EFunctionalTest extends AbstractTestNGSpringCont
return false;
}
/**
* Method to check if the contentName is returned in the SearchResponse
* @param response
* @param contentName
* @return
*/
public boolean isContentInSearchResponse(SearchResponse response, String contentName)
{
boolean found = response.getEntries().stream()
.map(entry -> entry.getModel().getName())
.filter(name -> name.equalsIgnoreCase(contentName) || contentName.isBlank()).count() > 0;
return found;
/**
* Method to check if the contentName is returned in the SearchResponse.
*
* @param response the search response
* @param contentName the text we are using as matching/verifying criteria.
* @return true if if the item with the contentName text is returned in the SearchResponse.
*/
public boolean isContentInSearchResponse(SearchResponse response, String contentName)
{
return response.getEntries().stream()
.map(entry -> entry.getModel().getName())
.anyMatch(name -> name.equalsIgnoreCase(contentName) || contentName.isBlank());
}
/**
* Wait for Solr to finish indexing: Indexing has caught up = true if search returns appropriate results
*
* @param userQuery: search query, this can include the fieldname, unique search string will guarantee accurate results
*
* @param userQuery: search query, this can include the fieldname, unique search string will guarantee accurate results
* @param expectedInResults, true if entry is expected in the results set
* @return true (indexing is finished) if search returns appropriate results
*/
@@ -309,7 +313,7 @@ public abstract class AbstractE2EFunctionalTest extends AbstractTestNGSpringCont
/**
* waitForIndexing method that matches / waits for filename, metadata to be indexed.
*
*
* @param userQuery
* @param expectedInResults
* @return
@@ -322,7 +326,7 @@ public abstract class AbstractE2EFunctionalTest extends AbstractTestNGSpringCont
/**
* waitForIndexing method that matches / waits for content to be indexed, this can take longer than metadata indexing.
* Since Metadata is indexed first, use this method where tests, queries need content to be indexed too.
*
*
* @param userQuery
* @param expectedInResults
* @return
@@ -334,9 +338,9 @@ public abstract class AbstractE2EFunctionalTest extends AbstractTestNGSpringCont
/**
* Wait for Solr to finish indexing: Indexing has caught up = true if search returns appropriate results
*
* @param fieldName: specific field to search for, e.g. name. When specified, the query will become: name:'userQuery'
* @param userQuery: search string, unique search string will guarantee accurate results
*
* @param fieldName: specific field to search for, e.g. name. When specified, the query will become: name:'userQuery'
* @param userQuery: search string, unique search string will guarantee accurate results
* @param expectedInResults, true if entry is expected in the results set
* @return true (indexing is finished) if search returns appropriate results
*/
@@ -349,7 +353,7 @@ public abstract class AbstractE2EFunctionalTest extends AbstractTestNGSpringCont
/**
* Run a search as admin user and return the response
*
*
* @param queryString: string to search for, unique search string will guarantee accurate results
* @return the search response from the API
*/
@@ -360,8 +364,8 @@ public abstract class AbstractE2EFunctionalTest extends AbstractTestNGSpringCont
/**
* Run a search as given user and return the response
*
* @param user: UserModel for the user you wish to run the query as
*
* @param user: UserModel for the user you wish to run the query as
* @param queryString: string to search for, unique search string will guarantee accurate results
* @return the search response from the API
*/
@@ -373,11 +377,11 @@ public abstract class AbstractE2EFunctionalTest extends AbstractTestNGSpringCont
searchRequest.setQuery(queryModel);
return restClient.authenticateUser(user).withSearchAPI().search(searchRequest);
}
/**
* Run a search with Spellcheck as given user and return the response
* @param user UserModel for the user you wish to run the query as
* @param queryModel The queryModel to search for, containing the query
* @param user UserModel for the user you wish to run the query as
* @param queryModel The queryModel to search for, containing the query
* @param spellcheckQuery The Spellcheck Model containing the query
* @return the search response from the API
*/
@@ -418,32 +422,22 @@ public abstract class AbstractE2EFunctionalTest extends AbstractTestNGSpringCont
query.setQuery(queryReq);
return query;
}
protected DataUser getDataUser()
{
return dataUser;
}
/**
* Helper method to test if the search query works and count matches where provided
* @param query: AFTS or cmis query string
* @param expectedCount: Only successful response is checked, when expectedCount is null (can not be exactly specified),
* @param setCmis: Query language is set to cmis when setCmis is true, AFTS when false
*
* @param query: AFTS or cmis query string
* @param expectedCount: Only successful response is checked, when expectedCount is null (can not be exactly specified),
* @return SearchResponse
*/
protected SearchResponse testSearchQuery(String query, Integer expectedCount, SearchLanguage queryLanguage)
{
RestRequestQueryModel queryModel = new RestRequestQueryModel();
queryModel.setQuery(query);
if (ofNullable(queryLanguage).isPresent())
{
queryModel.setLanguage(queryLanguage.toString());
}
SearchResponse response = queryAsUser(testUser, queryModel);
restClient.assertStatusCodeIs(HttpStatus.OK);
SearchResponse response = performSearch(testUser, query, queryLanguage);
if (ofNullable(expectedCount).isPresent())
{
@@ -452,7 +446,64 @@ public abstract class AbstractE2EFunctionalTest extends AbstractTestNGSpringCont
return response;
}
/**
* Helper method to test if the search query returns the expected results in the order given.
*
* @param query: AFTS or cmis query string
* @param expectedNames: The ordered list of names expected to be returned,
* @return SearchResponse
*/
protected SearchResponse testSearchQueryOrdered(String query, List<String> expectedNames, SearchLanguage queryLanguage)
{
SearchResponse response = performSearch(testUser, query, queryLanguage);
List<String> names = response.getEntries().stream().map(s -> s.getModel().getName()).collect(Collectors.toList());
// Include lists in failure message as TestNG won't do this for lists.
assertEquals(names, expectedNames, "Unexpected results for query: " + query + " Expected: " + expectedNames + " but got " + names);
return response;
}
/**
* Helper method to test if the search query returns the expected set of results.
*
* @param query: AFTS or cmis query string
* @param expectedNames: The ordered list of names expected to be returned,
* @return SearchResponse
*/
protected SearchResponse testSearchQueryUnordered(String query, Set<String> expectedNames, SearchLanguage queryLanguage)
{
SearchResponse response = performSearch(testUser, query, queryLanguage);
Set<String> names = response.getEntries().stream().map(s -> s.getModel().getName()).collect(Collectors.toSet());
assertEquals(names, expectedNames, "Unexpected results for query: " + query);
return response;
}
private SearchResponse performSearch(UserModel asUser, String query, SearchLanguage queryLanguage)
{
RestRequestQueryModel queryModel = new RestRequestQueryModel();
queryModel.setQuery(query);
if (!ofNullable(asUser).isPresent())
{
asUser = testUser;
}
if (ofNullable(queryLanguage).isPresent())
{
queryModel.setLanguage(queryLanguage.toString());
}
SearchResponse response = queryAsUser(asUser, queryModel);
return response;
}
/**
* Method to create and run a simple spellcheck query
* When a spellcheck query is run a user, the query inputed and the user query is inputted
@@ -464,8 +515,8 @@ public abstract class AbstractE2EFunctionalTest extends AbstractTestNGSpringCont
{
RestRequestSpellcheckModel spellCheck = new RestRequestSpellcheckModel();
spellCheck.setQuery(userQuery);
UserModel searchUser = ofNullable(user).isPresent()? user: testUser;
UserModel searchUser = ofNullable(user).isPresent() ? user : testUser;
SearchRequest searchReq = new SearchRequest();
RestRequestQueryModel queryReq = new RestRequestQueryModel();
queryReq.setQuery(query);
@@ -478,8 +529,8 @@ public abstract class AbstractE2EFunctionalTest extends AbstractTestNGSpringCont
/**
* Method to check the spellcheck object returned in the Search Response
* @param response SearchResponse
* @param spellCheckType String Values: searchInsteadFor, didYouMean or null
* @param response SearchResponse
* @param spellCheckType String Values: searchInsteadFor, didYouMean or null
* @param spellCheckSuggestion String Values: suggestion string or null
*/
public void testSearchSpellcheckResponse(SearchResponse response, String spellCheckType, String spellCheckSuggestion)
@@ -498,6 +549,6 @@ public abstract class AbstractE2EFunctionalTest extends AbstractTestNGSpringCont
{
response.getContext().assertThat().field("spellCheck").isNotEmpty();
response.getContext().getSpellCheck().assertThat().field("suggestions").contains(spellCheckSuggestion);
}
}
}
}
e2e-test/src/test/java/org/alfresco/test/search/functional/searchServices/cmis/AbstractCmisE2ETest.java
+2 -2
View File
@@ -61,7 +61,7 @@ public abstract class AbstractCmisE2ETest extends AbstractE2EFunctionalTest
protected boolean waitForIndexing(String query, long expectedCountResults)
{
for (int searchCount = 1; searchCount <= 3; searchCount++)
for (int searchCount = 1; searchCount <= SEARCH_MAX_ATTEMPTS; searchCount++)
{
try
@@ -71,7 +71,7 @@ public abstract class AbstractCmisE2ETest extends AbstractE2EFunctionalTest
}
catch (AssertionError ae)
{
LOGGER.debug(ae.toString());
LOGGER.info(String.format("WaitForIndexing in Progress: %s", ae.toString()));
}
e2e-test/src/test/java/org/alfresco/test/search/functional/searchServices/cmis/SolrSearchByAspectTests.java
+3 -1
View File
@@ -7,6 +7,7 @@ import org.alfresco.utility.data.provider.XMLTestDataProvider;
import org.alfresco.utility.model.FileModel;
import org.alfresco.utility.model.FolderModel;
import org.alfresco.utility.model.QueryModel;
import org.testng.Assert;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;
@@ -135,6 +136,7 @@ public class SolrSearchByAspectTests extends AbstractCmisE2ETest
.replace("NODE_REF[f1]", tasFolder1.getNodeRef())
.replace("NODE_REF[s1]", siteDoclibNodeRef);
cmisApi.authenticateUser(testUser).withQuery(currentQuery).assertResultsCount().equals(query.getResults());
cmisApi.authenticateUser(testUser);
Assert.assertTrue(waitForIndexing(currentQuery, query.getResults()), String.format("Result count not as expected for query: %s", currentQuery));
}
}
e2e-test/src/test/java/org/alfresco/test/search/functional/searchServices/cmis/SolrSearchByIdTests.java
+4 -2
View File
@@ -7,6 +7,7 @@ import org.alfresco.utility.data.provider.XMLTestDataProvider;
import org.alfresco.utility.model.FileModel;
import org.alfresco.utility.model.FolderModel;
import org.alfresco.utility.model.QueryModel;
import org.testng.Assert;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;
@@ -87,7 +88,7 @@ public class SolrSearchByIdTests extends AbstractCmisE2ETest
@Test(dataProviderClass = XMLTestDataProvider.class, dataProvider = "getQueriesData")
@XMLDataConfig(file = "src/test/resources/testdata/search-by-id.xml")
public void executeSearchByAspect(QueryModel query) throws Exception
public void executeSearchById(QueryModel query) throws Exception
{
String currentQuery = query.getValue()
.replace("NODE_REF[siteId]", siteDoclibNodeRef)
@@ -96,6 +97,7 @@ public class SolrSearchByIdTests extends AbstractCmisE2ETest
.replace("NODE_REF[f1]", tasFolder1.getNodeRef())
.replace("NODE_REF[f1-1]", tasSubFolder1.getNodeRef());
cmisApi.authenticateUser(testUser).withQuery(currentQuery).assertResultsCount().equals(query.getResults());
cmisApi.authenticateUser(testUser);
Assert.assertTrue(waitForIndexing(currentQuery, query.getResults()), String.format("Result count not as expected for query: %s", currentQuery));
}
}
e2e-test/src/test/java/org/alfresco/test/search/functional/searchServices/cmis/SolrSearchByPathTests.java
+4 -1
View File
@@ -7,6 +7,7 @@ import org.alfresco.utility.data.provider.XMLTestDataProvider;
import org.alfresco.utility.model.QueryModel;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.testng.Assert;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;
@@ -51,6 +52,8 @@ public class SolrSearchByPathTests extends AbstractCmisE2ETest
@XMLDataConfig(file = "src/test/resources/testdata/search-by-path.xml")
public void executeSearchByPathQueries(QueryModel query)
{
cmisApi.withQuery(query.getValue()).assertResultsCount().equals(query.getResults());
cmisApi.authenticateUser(testUser);
Assert.assertTrue(waitForIndexing(query.getValue(), query.getResults()), String.format("Result count not as expected for query: %s", query.getValue()));
}
}
e2e-test/src/test/java/org/alfresco/test/search/functional/searchServices/cmis/SolrSearchByPropertyTests.java
+424 -23
View File
@@ -1,5 +1,8 @@
package org.alfresco.test.search.functional.searchServices.cmis;
import java.util.List;
import java.util.Set;
import org.alfresco.utility.Utility;
import org.alfresco.utility.data.CustomObjectTypeProperties;
import org.alfresco.utility.data.provider.XMLDataConfig;
@@ -7,6 +10,7 @@ import org.alfresco.utility.data.provider.XMLTestDataProvider;
import org.alfresco.utility.model.FileModel;
import org.alfresco.utility.model.FolderModel;
import org.alfresco.utility.model.QueryModel;
import org.testng.Assert;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;
@@ -15,7 +19,7 @@ public class SolrSearchByPropertyTests extends AbstractCmisE2ETest
private FolderModel guestf, tesf, restf, testtttf, testf, testf1, testf2, testf3, testf4;
private FileModel guestc, restc, tesc, testtttc, testc, testc1, testc2, testc3;
@BeforeClass(alwaysRun = true)
@BeforeClass (alwaysRun = true)
public void dataPreparation() throws Exception
{
dataContent.usingAdmin().deployContentModel("model/tas-model.xml");
@@ -46,68 +50,465 @@ public class SolrSearchByPropertyTests extends AbstractCmisE2ETest
// Sites
dataContent.usingUser(testUser).usingSite(testSite).createCustomContent(guestf, "F:tas:folder",
new CustomObjectTypeProperties().addProperty("tas:TextPropertyF", "guestf text")
.addProperty("tas:IntPropertyF", 222));
.addProperty("tas:IntPropertyF", 222));
dataContent.usingUser(testUser).usingSite(testSite).createCustomContent(tesf, "F:tas:folder",
new CustomObjectTypeProperties().addProperty("tas:TextPropertyF", "tesf text")
.addProperty("tas:IntPropertyF", 224));
.addProperty("tas:IntPropertyF", 224));
// Sites >> Folders
dataContent.usingUser(testUser).usingResource(guestf).createCustomContent(restf, "F:tas:folder",
new CustomObjectTypeProperties().addProperty("tas:TextPropertyF", "restf text")
.addProperty("tas:IntPropertyF", 223));
.addProperty("tas:IntPropertyF", 223));
dataContent.usingUser(testUser).usingResource(guestf).createCustomContent(testtttf, "F:tas:folder",
new CustomObjectTypeProperties().addProperty("tas:TextPropertyF", "testtttf text")
.addProperty("tas:IntPropertyF", 225));
.addProperty("tas:IntPropertyF", 225));
dataContent.usingUser(testUser).usingResource(guestf).createCustomContent(testf, "F:tas:folder",
new CustomObjectTypeProperties().addProperty("tas:TextPropertyF", "testf text")
.addProperty("tas:IntPropertyF", 226));
.addProperty("tas:IntPropertyF", 226));
dataContent.usingUser(testUser).usingResource(guestf).createCustomContent(testf1, "F:tas:folder",
new CustomObjectTypeProperties().addProperty("tas:TextPropertyF", "testf1 text")
.addProperty("tas:IntPropertyF", 2221));
.addProperty("tas:IntPropertyF", 2221));
dataContent.usingUser(testUser).usingResource(guestf).createCustomContent(testf2, "F:tas:folder",
new CustomObjectTypeProperties().addProperty("tas:TextPropertyF", "testf2 text")
.addProperty("tas:IntPropertyF", 2222));
.addProperty("tas:IntPropertyF", 2222));
dataContent.usingUser(testUser).usingResource(guestf).createCustomContent(testf3, "F:tas:folder",
new CustomObjectTypeProperties().addProperty("tas:TextPropertyF", "testf3 text")
.addProperty("tas:IntPropertyF", 2223));
.addProperty("tas:IntPropertyF", 2223));
dataContent.usingUser(testUser).usingResource(guestf).createCustomContent(testf4, "F:tas:folder",
new CustomObjectTypeProperties().addProperty("tas:TextPropertyF", "testf4 text")
.addProperty("tas:IntPropertyF", 2224));
.addProperty("tas:IntPropertyF", 2224));
// Sites >> Files
dataContent.usingUser(testUser).usingResource(tesf).createCustomContent(guestc, "D:tas:document",
new CustomObjectTypeProperties().addProperty("tas:TextPropertyC", "guestc text")
.addProperty("tas:IntPropertyC", 222));
.addProperty("tas:IntPropertyC", 222));
dataContent.usingUser(testUser).usingResource(tesf).createCustomContent(restc, "D:tas:document",
new CustomObjectTypeProperties().addProperty("tas:TextPropertyC", "restc text")
.addProperty("tas:IntPropertyC", 223));
.addProperty("tas:IntPropertyC", 223));
dataContent.usingUser(testUser).usingResource(tesf).createCustomContent(tesc, "D:tas:document",
new CustomObjectTypeProperties().addProperty("tas:TextPropertyC", "tesc text")
.addProperty("tas:IntPropertyC", 224));
.addProperty("tas:IntPropertyC", 224));
dataContent.usingUser(testUser).usingResource(tesf).createCustomContent(testtttc, "D:tas:document",
new CustomObjectTypeProperties().addProperty("tas:TextPropertyC", "testtttc text")
.addProperty("tas:IntPropertyC", 225));
.addProperty("tas:IntPropertyC", 225));
dataContent.usingUser(testUser).usingResource(tesf).createCustomContent(testc, "D:tas:document",
new CustomObjectTypeProperties().addProperty("tas:TextPropertyC", "testc text")
.addProperty("tas:IntPropertyC", 226));
.addProperty("tas:IntPropertyC", 226));
dataContent.usingUser(testUser).usingResource(tesf).createCustomContent(testc1, "D:tas:document",
new CustomObjectTypeProperties().addProperty("tas:TextPropertyC", "testc1 text")
.addProperty("tas:IntPropertyC", 2221));
.addProperty("tas:IntPropertyC", 2221));
dataContent.usingUser(testUser).usingResource(tesf).createCustomContent(testc2, "D:tas:document",
new CustomObjectTypeProperties().addProperty("tas:TextPropertyC", "testc2 text")
.addProperty("tas:IntPropertyC", 2222));
.addProperty("tas:IntPropertyC", 2222));
dataContent.usingUser(testUser).usingResource(tesf).createCustomContent(testc3, "D:tas:document",
new CustomObjectTypeProperties().addProperty("tas:TextPropertyC", "testc3 text")
.addProperty("tas:IntPropertyC", 2223));
.addProperty("tas:IntPropertyC", 2223));
// wait for solr index
Utility.waitToLoopTime(getSolrWaitTimeInSeconds());
cmisApi.authenticateUser(testUser);
waitForIndexing("SELECT * FROM tas:document where cmis:name = 'testc3.txt'", 1);
}
@Test(dataProviderClass = XMLTestDataProvider.class, dataProvider = "getQueriesData")
@XMLDataConfig(file = "src/test/resources/testdata/search-by-property.xml")
public void executeSearchByProperty(QueryModel query) throws Exception
@Test
public void testFileNameEquality()
{
cmisApi.authenticateUser(testUser).withQuery(query.getValue()).assertResultsCount().equals(query.getResults());
String query = "SELECT * FROM tas:document where cmis:name = 'testc.txt'";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("testc.txt"));
}
@Test
public void testFileNameInequality()
{
String query = "SELECT * FROM tas:document where cmis:name <> 'testc.txt'";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("guestc.txt", "restc.txt", "tesc.txt", "testtttc.txt", "testc1.txt", "testc2.txt", "testc3.txt"));
}
@Test
public void testFileNameIn()
{
String query = "SELECT * FROM tas:document where cmis:name IN('testc.txt', 'guestc.txt', 'restc.txt')";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("testc.txt", "guestc.txt", "restc.txt"));
}
@Test
public void testFileNameNotIn()
{
// Nb. "gustc" is missing an "e".
String query = "SELECT * FROM tas:document where cmis:name NOT IN('testc.txt', 'gustc.txt', 'restc.txt')";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("guestc.txt", "tesc.txt", "testtttc.txt", "testc1.txt", "testc2.txt", "testc3.txt"));
}
@Test
public void testFileNameLike()
{
String query = "SELECT * FROM tas:document where cmis:name LIKE '%testc%'";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("testc.txt", "testc1.txt", "testc2.txt", "testc3.txt"));
}
@Test
public void testFileNameLikeExact()
{
String query = "SELECT * FROM tas:document where cmis:name LIKE 'testc.txt'";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("testc.txt"));
}
@Test
public void testFileNamePrefixSuffix()
{
String query = "SELECT * FROM tas:document where cmis:name LIKE 't%tc.txt'";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("testc.txt", "testtttc.txt"));
}
@Test
public void testFileNameUnderscore()
{
String query = "SELECT * FROM tas:document where cmis:name LIKE 't__tc.txt'";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("testc.txt"));
}
@Test
public void testFolderNameEquality()
{
String query = "SELECT * FROM tas:folder where cmis:name = 'testf'";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("testf"));
}
@Test
public void testFolderNameInequality()
{
String query = "SELECT * FROM tas:folder where cmis:name <> 'testf'";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("guestf", "tesf", "restf", "testtttf", "testf1", "testf2", "testf3", "testf4"));
}
@Test
public void testFolderNameIn()
{
String query = "SELECT * FROM tas:folder where cmis:name IN('testf', 'guestf', 'restf')";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("testf", "guestf", "restf"));
}
@Test
public void testFolderNameNotIn()
{
// Nb. "gustc" is missing an "e".
String query = "SELECT * FROM tas:folder where cmis:name NOT IN('testf', 'gustf', 'restf')";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("guestf", "tesf", "testtttf", "testf1", "testf2", "testf3", "testf4"));
}
@Test
public void testFolderNameLike()
{
String query = "SELECT * FROM tas:folder where cmis:name LIKE '%testf%'";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("testf", "testf1", "testf2", "testf3", "testf4"));
}
@Test
public void testFolderNameLikeExact()
{
String query = "SELECT * FROM tas:folder where cmis:name LIKE 'testf'";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("testf"));
}
@Test
public void testFolderNamePrefixSuffix()
{
String query = "SELECT * FROM tas:folder where cmis:name LIKE 't%tf'";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("testtttf", "testf"));
}
@Test
public void testFolderNameUnderscore()
{
String query = "SELECT * FROM tas:folder where cmis:name LIKE 't__tf'";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("testf"));
}
@Test
public void testFileNameOrderAsc()
{
String query = "SELECT * FROM tas:document where cmis:name LIKE '%testc%' ORDER BY cmis:name ASC";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningOrderedValues("cmis:name",
List.of("testc1.txt", "testc2.txt", "testc3.txt", "testc.txt"));
}
@Test
public void testFileNameOrderDesc()
{
String query = "SELECT * FROM tas:document where cmis:name LIKE '%testc%' ORDER BY cmis:name DESC";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningOrderedValues("cmis:name",
List.of("testc.txt", "testc3.txt", "testc2.txt", "testc1.txt"));
}
@Test
public void testFileOrderNameDescDateAsc()
{
String query = "SELECT * FROM tas:document where cmis:name LIKE '%testc%' ORDER BY cmis:name DESC, cmis:creationDate ASC";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningOrderedValues("cmis:name",
List.of("testc.txt", "testc3.txt", "testc2.txt", "testc1.txt"));
}
@Test
public void testFolderNameOrderAsc()
{
String query = "SELECT * FROM tas:folder where cmis:name LIKE '%testf%' ORDER BY cmis:name ASC";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningOrderedValues("cmis:name",
List.of("testf", "testf1", "testf2", "testf3", "testf4"));
}
@Test
public void testFolderNameOrderDesc()
{
String query = "SELECT * FROM tas:folder where cmis:name LIKE '%testf%' ORDER BY cmis:name DESC";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningOrderedValues("cmis:name",
List.of("testf4", "testf3", "testf2", "testf1", "testf"));
}
@Test
public void testFolderOrderNameDescDateAsc()
{
String query = "SELECT * FROM tas:folder where cmis:name LIKE '%testf%' ORDER BY cmis:name DESC, cmis:creationDate ASC";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningOrderedValues("cmis:name",
List.of("testf4", "testf3", "testf2", "testf1", "testf"));
}
@Test
public void testFileCustomPropertyEquality()
{
String query = "SELECT * FROM tas:document where tas:TextPropertyC = 'restc text'";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("restc.txt"));
}
@Test
public void testFileCustomPropertyInequality()
{
String query = "SELECT * FROM tas:document where tas:TextPropertyC <> 'testc1 text'";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("guestc.txt", "restc.txt", "tesc.txt", "testtttc.txt", "testc.txt", "testc2.txt", "testc3.txt"));
}
@Test
public void testFileCustomPropertyIn()
{
String query = "SELECT * FROM tas:document where tas:TextPropertyC IN('restc text', 'testc2 text', 'testc3 text')";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("restc.txt", "testc2.txt", "testc3.txt"));
}
@Test
public void testFileCustomPropertyNotIn()
{
String query = "SELECT * FROM tas:document where tas:TextPropertyC NOT IN('restc text', 'testc2 text', 'testc3 text')";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("guestc.txt", "tesc.txt", "testtttc.txt", "testc.txt", "testc1.txt"));
}
@Test
public void testFileCustomPropertyLike()
{
String query = "SELECT * FROM tas:document where tas:TextPropertyC LIKE '%restc%'";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("restc.txt"));
}
@Test
public void testFileCustomPropertyUnderscore()
{
String query = "SELECT * FROM tas:document where tas:TextPropertyC LIKE 't__tc text'";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("testc.txt"));
}
@Test
public void testFolderCustomPropertyEquality()
{
String query = "SELECT * FROM tas:folder where tas:TextPropertyF = 'restf text'";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("restf"));
}
@Test
public void testFolderCustomPropertyInequality()
{
String query = "SELECT * FROM tas:folder where tas:TextPropertyF <> 'testf1 text'";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("guestf", "tesf", "restf", "testtttf", "testf", "testf2", "testf3", "testf4"));
}
@Test
public void testFolderCustomPropertyIn()
{
String query = "SELECT * FROM tas:folder where tas:TextPropertyF IN('restf text', 'testf2 text', 'testf3 text')";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("restf", "testf2", "testf3"));
}
@Test
public void testFolderCustomPropertyNotIn()
{
String query = "SELECT * FROM tas:folder where tas:TextPropertyF NOT IN('restf text', 'testf2 text', 'testf3 text')";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("guestf", "tesf", "testtttf", "testf", "testf1", "testf4"));
}
@Test
public void testFolderCustomPropertyLike()
{
String query = "SELECT * FROM tas:folder where tas:TextPropertyF LIKE '%restf%'";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("restf"));
}
@Test
public void testFolderCustomPropertyUnderscore()
{
String query = "SELECT * FROM tas:folder where tas:TextPropertyF LIKE 't__tf text'";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("testf"));
}
@Test
public void testFileIntEquality()
{
String query = "SELECT * FROM tas:document where tas:IntPropertyC = '222'";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("guestc.txt"));
}
@Test
public void testFileIntInequality()
{
String query = "SELECT * FROM tas:document where tas:IntPropertyC <> '223'";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("guestc.txt", "tesc.txt", "testtttc.txt", "testc.txt", "testc1.txt", "testc2.txt", "testc3.txt"));
}
@Test
public void testFileIntLessThen()
{
String query = "SELECT * FROM tas:document where tas:IntPropertyC < '223'";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("guestc.txt"));
}
@Test
public void testFileIntLessThanOrEqual()
{
String query = "SELECT * FROM tas:document where tas:IntPropertyC <= '224'";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("guestc.txt", "restc.txt", "tesc.txt"));
}
@Test
public void testFileIntGreaterThanOrEqual()
{
String query = "SELECT * FROM tas:document where tas:IntPropertyC >= '224'";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("tesc.txt", "testtttc.txt", "testc.txt", "testc1.txt", "testc2.txt", "testc3.txt"));
}
@Test
public void testFileIntGreaterThan()
{
String query = "SELECT * FROM tas:document where tas:IntPropertyC > '224'";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("testtttc.txt", "testc.txt", "testc1.txt", "testc2.txt", "testc3.txt"));
}
@Test
public void testFileIntIn()
{
String query = "SELECT * FROM tas:document where tas:IntPropertyC IN('222', '223', '224', '225')";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("guestc.txt", "restc.txt", "tesc.txt", "testtttc.txt"));
}
@Test
public void testFileIntNotIn()
{
String query = "SELECT * FROM tas:document where tas:IntPropertyC NOT IN('222', '223', '224')";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("testtttc.txt", "testc.txt", "testc1.txt", "testc2.txt", "testc3.txt"));
}
@Test
public void testFolderIntEquality()
{
String query = "SELECT * FROM tas:folder where tas:IntPropertyF = '222'";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("guestf"));
}
@Test
public void testFolderIntInequality()
{
String query = "SELECT * FROM tas:folder where tas:IntPropertyF <> '223'";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("guestf", "tesf", "testtttf", "testf", "testf1", "testf2", "testf3", "testf4"));
}
@Test
public void testFolderIntLessThan()
{
String query = "SELECT * FROM tas:folder where tas:IntPropertyF < '223'";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("guestf"));
}
@Test
public void testFolderIntLessThanOrEqual()
{
String query = "SELECT * FROM tas:folder where tas:IntPropertyF <= '224'";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("guestf", "tesf", "restf"));
}
@Test
public void testFolderIntGreaterThanOrEqual()
{
String query = "SELECT * FROM tas:folder where tas:IntPropertyF >= '224'";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("tesf", "testtttf", "testf", "testf1", "testf2", "testf3", "testf4"));
}
@Test
public void testFolderIntGreaterThan()
{
String query = "SELECT * FROM tas:folder where tas:IntPropertyF > '224'";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("testtttf", "testf", "testf1", "testf2", "testf3", "testf4"));
}
@Test
public void testFolderIntIn()
{
String query = "SELECT * FROM tas:folder where tas:IntPropertyF IN('222', '223', '224', '225')";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("guestf", "tesf", "restf", "testtttf"));
}
@Test
public void testFolderIntNotIn()
{
String query = "SELECT * FROM tas:folder where tas:IntPropertyF NOT IN('222', '223', '224')";
cmisApi.authenticateUser(testUser).withQuery(query).assertValues().isReturningValues("cmis:name",
Set.of("testtttf", "testf", "testf1", "testf2", "testf3", "testf4"));
}
}
e2e-test/src/test/java/org/alfresco/test/search/functional/searchServices/cmis/SolrSearchInFolderTests.java
+3 -1
View File
@@ -7,6 +7,7 @@ import org.alfresco.utility.model.FileModel;
import org.alfresco.utility.model.FileType;
import org.alfresco.utility.model.FolderModel;
import org.alfresco.utility.model.QueryModel;
import org.testng.Assert;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;
@@ -55,6 +56,7 @@ public class SolrSearchInFolderTests extends AbstractCmisE2ETest
public void executeCMISQuery(QueryModel query) throws Exception
{
String currentQuery = String.format(query.getValue(), parentFolder.getNodeRef());
cmisApi.withQuery(currentQuery).assertResultsCount().equals(query.getResults());
cmisApi.authenticateUser(testUser);
Assert.assertTrue(waitForIndexing(currentQuery, query.getResults()), String.format("Result count not as expected for query: %s", currentQuery));
}
}
e2e-test/src/test/java/org/alfresco/test/search/functional/searchServices/cmis/SolrSearchInTreeTests.java
+3 -2
View File
@@ -7,6 +7,7 @@ import org.alfresco.utility.model.FileModel;
import org.alfresco.utility.model.FileType;
import org.alfresco.utility.model.FolderModel;
import org.alfresco.utility.model.QueryModel;
import org.testng.Assert;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;
@@ -59,7 +60,7 @@ public class SolrSearchInTreeTests extends AbstractCmisE2ETest
public void executeCMISQuery(QueryModel query) throws Exception
{
String currentQuery = String.format(query.getValue(), parentFolder.getNodeRef());
cmisApi.withQuery(currentQuery)
.assertResultsCount().equals(query.getResults());
cmisApi.authenticateUser(testUser);
Assert.assertTrue(waitForIndexing(currentQuery, query.getResults()), String.format("Result count not as expected for query: %s", currentQuery));
}
}
e2e-test/src/test/java/org/alfresco/test/search/functional/searchServices/cmis/SolrSearchScoreQueryTests.java
+59 -88
View File
@@ -9,6 +9,7 @@ import org.alfresco.utility.data.provider.XMLTestDataProvider;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.testng.Assert;
import org.testng.annotations.AfterClass;
import org.testng.annotations.Test;
@@ -54,8 +55,8 @@ public class SolrSearchScoreQueryTests extends AbstractCmisE2ETest
this.testData = testData;
this.testData.createUsers(dataUser);
this.testData.createSitesStructure(dataSite, dataContent, dataUser);
cmisApi.authenticateUser(dataUser.getCurrentUser());
testUser = dataUser.getCurrentUser();
cmisApi.authenticateUser(testUser);
}
/**
@@ -70,68 +71,50 @@ public class SolrSearchScoreQueryTests extends AbstractCmisE2ETest
+ "FROM cmis:document "
+ "WHERE CONTAINS('Quidditch') "
+ "ORDER BY orderCriteria";
if (waitForIndexing(query, 3))
{
cmisApi
.withQuery(query)
.assertColumnIsOrdered().isOrderedAsc("orderCriteria");
}
else
{
throw new AssertionError("Wait for indexing has failed!");
}
Assert.assertTrue(waitForIndexing(query, 3), String.format("Result count not as expected for query: %s", query));
cmisApi.withQuery(query).assertColumnIsOrdered().isOrderedAsc("orderCriteria");
}
/**
* Verify that results are inverse ordered
* @throws Exception
*/
@Test(dependsOnMethods = "prepareDataForScoreSearch")
/**
* Verify that results are inverse ordered
*
* @throws Exception
*/
@Test(dependsOnMethods = "prepareDataForScoreSearch")
public void scoreQueryOrderedDesc() throws Exception
{
String query = "SELECT cmis:objectId, SCORE() AS orderCriteria "
String query = "SELECT cmis:objectId, SCORE() AS orderCriteria "
+ "FROM cmis:document "
+ "WHERE CONTAINS('Quidditch') "
+ "ORDER BY orderCriteria DESC";
if (waitForIndexing(query, 3))
{
cmisApi
.withQuery(query).assertColumnIsOrdered().isOrderedDesc("orderCriteria");
}
else
{
throw new AssertionError("Wait for indexing has failed!");
}
Assert.assertTrue(waitForIndexing(query, 3), String.format("Result count not as expected for query: %s", query));
cmisApi.withQuery(query).assertColumnIsOrdered().isOrderedDesc("orderCriteria");
}
/**
* Verify that all SCORE results are between 0 and 1
* @throws Exception
*/
@Test(groups = { TestGroup.ACS_62n }, dependsOnMethods = "prepareDataForScoreSearch")
/**
* Verify that all SCORE results are between 0 and 1
*
* @throws Exception
*/
@Test(groups = { TestGroup.ACS_62n }, dependsOnMethods = "prepareDataForScoreSearch")
public void scoreQueryInRange() throws Exception
{
String query = "SELECT cmis:objectId, SCORE() "
+ "FROM cmis:document "
+ "WHERE CONTAINS('Quidditch')";
if (waitForIndexing(query, 3))
{
cmisApi
.withQuery(query)
.assertColumnValuesRange().isReturningValuesInRange("SEARCH_SCORE", BigDecimal.ZERO, BigDecimal.ONE);
}
else
{
throw new AssertionError("Wait for indexing has failed!");
}
String query = "SELECT cmis:objectId, SCORE() "
+ "FROM cmis:document "
+ "WHERE CONTAINS('Quidditch')";
Assert.assertTrue(waitForIndexing(query, 3), String.format("Result count not as expected for query: %s", query));
cmisApi.withQuery(query).assertColumnValuesRange().isReturningValuesInRange("SEARCH_SCORE", BigDecimal.ZERO, BigDecimal.ONE);
}
/**
@@ -143,45 +126,33 @@ public class SolrSearchScoreQueryTests extends AbstractCmisE2ETest
{
String query = "SELECT cmis:objectId, SCORE() AS orderCriteria "
+ "FROM cmis:document "
+ "WHERE CONTAINS('Quidditch')";
if (waitForIndexing(query, 3))
{
cmisApi
.withQuery(query)
.assertColumnValuesRange().isReturningValuesInRange("orderCriteria", BigDecimal.ZERO, BigDecimal.ONE);
}
else
{
throw new AssertionError("Wait for indexing has failed!");
}
+ "FROM cmis:document "
+ "WHERE CONTAINS('Quidditch')";
Assert.assertTrue(waitForIndexing(query, 3), String.format("Result count not as expected for query: %s", query));
cmisApi.withQuery(query).assertColumnValuesRange().isReturningValuesInRange("orderCriteria", BigDecimal.ZERO, BigDecimal.ONE);
}
/**
* Verify that SCORE is valid name for an alias
* Currently only supported with double quotes
* @throws Exception
*/
@Test(dependsOnMethods = "prepareDataForScoreSearch")
/**
* Verify that SCORE is valid name for an alias
* Currently only supported with double quotes
*
* @throws Exception
*/
@Test(dependsOnMethods = "prepareDataForScoreSearch")
public void scoreQueryScoreAsAlias() throws Exception
{
String query = "SELECT cmis:objectId, SCORE() AS \"score\" "
+ "FROM cmis:document "
+ "WHERE CONTAINS('Quidditch')";
if (waitForIndexing(query, 3))
{
cmisApi
.withQuery(query).assertResultsCount().equals(3);
}
else
{
throw new AssertionError("Wait for indexing has failed!");
}
String query = "SELECT cmis:objectId, SCORE() AS \"score\" "
+ "FROM cmis:document "
+ "WHERE CONTAINS('Quidditch')";
Assert.assertTrue(waitForIndexing(query, 3), String.format("Result count not as expected for query: %s", query));
cmisApi.withQuery(query).assertResultsCount().equals(3);
}
}
e2e-test/src/test/java/org/alfresco/test/search/functional/searchServices/search/FacetRangeSearchTest.java
+92 -118
View File
@@ -22,9 +22,7 @@ import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.core.Is.is;
import static org.testng.Assert.assertEquals;
import static org.testng.Assert.assertNull;
import static org.testng.Assert.assertTrue;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
@@ -66,8 +64,8 @@ import org.testng.annotations.Test;
* "gap": "+100DAY"
* }
* }
* @author Michael Suzuki
*
* @author Michael Suzuki
*/
public class FacetRangeSearchTest extends AbstractSearchServicesE2ETest
{
@@ -78,49 +76,37 @@ public class FacetRangeSearchTest extends AbstractSearchServicesE2ETest
waitForContentIndexing(file4.getContent(), true);
}
/** Check the error messages mention the mandatory fields when they are omitted. */
@Test
@TestRail(section = {TestGroup.REST_API, TestGroup.SEARCH }, executionType = ExecutionType.REGRESSION,
description = "Check facet intervals mandatory fields")
public void checkingFacetsMandatoryErrorMessages()
{
SearchRequest query = createQuery("cars");
List<RestRequestRangesModel> ranges = new ArrayList<>();
RestRequestRangesModel facetRangeModel = new RestRequestRangesModel();
ranges.add(facetRangeModel);
query.setRanges(ranges);
// Omit the field.
query.setRanges(List.of(createRangesModel(null, "0", "400", "20")));
query(query);
restClient.assertStatusCodeIs(HttpStatus.BAD_REQUEST).assertLastError()
.containsSummary(String.format(RestErrorModel.MANDATORY_PARAM, "field"));
ranges.clear();
facetRangeModel.setField("content.size");
ranges.add(facetRangeModel);
query.setRanges(ranges);
.containsSummary(String.format(RestErrorModel.MANDATORY_PARAM, "field"));
// Omit the start.
query.setRanges(List.of(createRangesModel("content.size", null, "400", "20")));
query(query);
restClient.assertStatusCodeIs(HttpStatus.BAD_REQUEST).assertLastError()
.containsSummary(String.format(RestErrorModel.MANDATORY_PARAM, "start"));
facetRangeModel.setStart("0");
ranges.clear();
ranges.add(facetRangeModel);
query.setRanges(ranges);
.containsSummary(String.format(RestErrorModel.MANDATORY_PARAM, "start"));
// Omit the end.
query.setRanges(List.of(createRangesModel("content.size", "0", null, "20")));
query(query);
restClient.assertStatusCodeIs(HttpStatus.BAD_REQUEST).assertLastError()
.containsSummary(String.format(RestErrorModel.MANDATORY_PARAM, "end"));
facetRangeModel.setEnd("400");
query.setRanges(ranges);
ranges.clear();
ranges.add(facetRangeModel);
.containsSummary(String.format(RestErrorModel.MANDATORY_PARAM, "end"));
// Omit the gap.
query.setRanges(List.of(createRangesModel("content.size", "0", "400", null)));
query(query);
restClient.assertStatusCodeIs(HttpStatus.BAD_REQUEST).assertLastError()
.containsSummary(String.format(RestErrorModel.MANDATORY_PARAM, "gap"));
facetRangeModel.setGap("100");
.containsSummary(String.format(RestErrorModel.MANDATORY_PARAM, "gap"));
}
@Test
@@ -131,13 +117,8 @@ public class FacetRangeSearchTest extends AbstractSearchServicesE2ETest
{
SearchRequest query = createQuery("* AND SITE:'" + testSite.getId() + "'");
RestRequestRangesModel facetRangeModel = new RestRequestRangesModel();
facetRangeModel.setField("content.size");
facetRangeModel.setStart("0");
facetRangeModel.setEnd("200");
facetRangeModel.setGap("20");
List<RestRequestRangesModel> ranges = new ArrayList<>();
ranges.add(facetRangeModel);
RestRequestRangesModel facetRangeModel = createRangesModel("content.size", "0", "200", "20");
List<RestRequestRangesModel> ranges = List.of(facetRangeModel);
query.setRanges(ranges);
SearchResponse response = query(query);
response.assertThat().entriesListIsNotEmpty();
@@ -147,14 +128,14 @@ public class FacetRangeSearchTest extends AbstractSearchServicesE2ETest
RestGenericBucketModel bucket = facetResponseModel.getBuckets().get(0);
bucket.assertThat().field("label").is("[20 - 40)");
bucket.assertThat().field("filterQuery").is("content.size:[\"20\" TO \"40\">");
Map<String,String> metric = (Map<String, String>) bucket.getMetrics().get(0).getValue();
Map<String, String> metric = (Map<String, String>) bucket.getMetrics().get(0).getValue();
assertEquals(Integer.valueOf(metric.get("count")).intValue(), 2, "Unexpected count for first bucket.");
Map<String, String> info = (Map<String, String>) bucket.getBucketInfo();
assertEquals(info.get("start"),"20");
assertEquals(info.get("end"),"40");
assertEquals(info.get("start"), "20");
assertEquals(info.get("end"), "40");
assertNull(info.get("count"));
assertEquals(info.get("startInclusive"),"true");
assertEquals(info.get("endInclusive"),"false");
assertEquals(info.get("startInclusive"), "true");
assertEquals(info.get("endInclusive"), "false");
bucket = facetResponseModel.getBuckets().get(1);
bucket.assertThat().field("label").is("[40 - 120)");
@@ -162,20 +143,20 @@ public class FacetRangeSearchTest extends AbstractSearchServicesE2ETest
metric = (Map<String, String>) bucket.getMetrics().get(0).getValue();
assertEquals(Integer.valueOf(metric.get("count")).intValue(), 1, "Unexpected count for second bucket.");
info = (Map<String, String>) bucket.getBucketInfo();
assertEquals(info.get("start"),"40");
assertEquals(info.get("end"),"120");
assertEquals(info.get("startInclusive"),"true");
assertEquals(info.get("endInclusive"),"false");
assertEquals(info.get("start"), "40");
assertEquals(info.get("end"), "120");
assertEquals(info.get("startInclusive"), "true");
assertEquals(info.get("endInclusive"), "false");
bucket = facetResponseModel.getBuckets().get(2);
bucket.assertThat().field("label").is("[120 - 200]");
bucket.assertThat().field("filterQuery").is("content.size:[\"120\" TO \"200\"]");
assertEquals(Integer.valueOf(metric.get("count")).intValue(), 1, "Unexpected count for third bucket.");
info = (Map<String, String>) bucket.getBucketInfo();
assertEquals(info.get("start"),"120");
assertEquals(info.get("end"),"200");
assertEquals(info.get("startInclusive"),"true");
assertEquals(info.get("endInclusive"),"true");
assertEquals(info.get("start"), "120");
assertEquals(info.get("end"), "200");
assertEquals(info.get("startInclusive"), "true");
assertEquals(info.get("endInclusive"), "true");
}
@Test
@@ -186,14 +167,9 @@ public class FacetRangeSearchTest extends AbstractSearchServicesE2ETest
{
SearchRequest query = createQuery("* AND SITE:'" + testSite.getId() + "'");
RestRequestRangesModel facetRangeModel = new RestRequestRangesModel();
facetRangeModel.setField("content.size");
facetRangeModel.setStart("0");
facetRangeModel.setEnd("200");
facetRangeModel.setGap("20");
RestRequestRangesModel facetRangeModel = createRangesModel("content.size", "0", "200", "20");
facetRangeModel.setHardend(true);
List<RestRequestRangesModel> ranges = new ArrayList<>();
ranges.add(facetRangeModel);
List<RestRequestRangesModel> ranges = List.of(facetRangeModel);
query.setRanges(ranges);
SearchResponse response = query(query);
response.assertThat().entriesListIsNotEmpty();
@@ -203,26 +179,26 @@ public class FacetRangeSearchTest extends AbstractSearchServicesE2ETest
RestGenericBucketModel bucket = facetResponseModel.getBuckets().get(0);
bucket.assertThat().field("label").is("[20 - 40)");
bucket.assertThat().field("filterQuery").is("content.size:[\"20\" TO \"40\">");
Map<String,String> metric = (Map<String, String>) bucket.getMetrics().get(0).getValue();
Map<String, String> metric = (Map<String, String>) bucket.getMetrics().get(0).getValue();
assertEquals(Integer.valueOf(metric.get("count")).intValue(), 2, "Unexpected count for first bucket.");
Map<String, String> info = (Map<String, String>) bucket.getBucketInfo();
assertEquals(info.get("start"),"20");
assertEquals(info.get("end"),"40");
assertEquals(info.get("startInclusive"),"true");
assertEquals(info.get("endInclusive"),"false");
assertEquals(info.get("start"), "20");
assertEquals(info.get("end"), "40");
assertEquals(info.get("startInclusive"), "true");
assertEquals(info.get("endInclusive"), "false");
assertNull(info.get("count"));
bucket = facetResponseModel.getBuckets().get(1);
bucket.assertThat().field("label").is("[40 - 120)");
bucket.assertThat().field("filterQuery").is("content.size:[\"40\" TO \"120\">");
info = (Map<String, String>) bucket.getBucketInfo();
assertEquals(info.get("start"),"40");
assertEquals(info.get("end"),"120");
assertEquals(info.get("start"), "40");
assertEquals(info.get("end"), "120");
metric = (Map<String, String>) bucket.getMetrics().get(0).getValue();
assertEquals(Integer.valueOf(metric.get("count")).intValue(), 1, "Unexpected count for second bucket.");
assertNull(info.get("count"));
assertEquals(info.get("startInclusive"),"true");
assertEquals(info.get("endInclusive"),"false");
assertEquals(info.get("startInclusive"), "true");
assertEquals(info.get("endInclusive"), "false");
bucket = facetResponseModel.getBuckets().get(2);
bucket.assertThat().field("label").is("[120 - 200]");
@@ -230,11 +206,11 @@ public class FacetRangeSearchTest extends AbstractSearchServicesE2ETest
metric = (Map<String, String>) bucket.getMetrics().get(0).getValue();
assertEquals(Integer.valueOf(metric.get("count")).intValue(), 1, "Unexpected count for third bucket.");
info = (Map<String, String>) bucket.getBucketInfo();
assertEquals(info.get("start"),"120");
assertEquals(info.get("end"),"200");
assertEquals(info.get("start"), "120");
assertEquals(info.get("end"), "200");
assertNull(info.get("count"));
assertEquals(info.get("startInclusive"),"true");
assertEquals(info.get("endInclusive"),"true");
assertEquals(info.get("startInclusive"), "true");
assertEquals(info.get("endInclusive"), "true");
}
/** This test relies on a document created in 2015 existing, probably part of the sample site. */
@@ -246,13 +222,8 @@ public class FacetRangeSearchTest extends AbstractSearchServicesE2ETest
{
SearchRequest query = createQuery("name:A*");
RestRequestRangesModel facetRangeModel = new RestRequestRangesModel();
facetRangeModel.setField("created");
facetRangeModel.setStart("2015-09-29T10:45:15.729Z");
facetRangeModel.setEnd("2016-09-29T10:45:15.729Z");
facetRangeModel.setGap("+280DAY");
List<RestRequestRangesModel> ranges = new ArrayList<>();
ranges.add(facetRangeModel);
RestRequestRangesModel facetRangeModel = createRangesModel("created", "2015-09-29T10:45:15.729Z", "2016-09-29T10:45:15.729Z", "+280DAY");
List<RestRequestRangesModel> ranges = List.of(facetRangeModel);
query.setRanges(ranges);
SearchResponse response = query(query);
response.assertThat().entriesListIsNotEmpty();
@@ -260,18 +231,18 @@ public class FacetRangeSearchTest extends AbstractSearchServicesE2ETest
RestGenericFacetResponseModel facetResponseModel = response.getContext().getFacets().get(0);
List<RestGenericBucketModel> buckets = facetResponseModel.getBuckets();
assertThat(buckets.size(),is(1));
assertThat(buckets.size(), is(1));
RestGenericBucketModel bucket = buckets.get(0);
bucket.assertThat().field("label").is("[2015-09-29T10:45:15.729Z - 2017-04-11T10:45:15.729Z]");
bucket.assertThat().field("filterQuery").is("created:[\"2015-09-29T10:45:15.729Z\" TO \"2017-04-11T10:45:15.729Z\"]");
bucket.getMetrics().get(0).assertThat().field("value").is("{count=1}");
Map<String, String> info = (Map<String, String>) bucket.getBucketInfo();
assertEquals(info.get("start"),"2015-09-29T10:45:15.729Z");
assertEquals(info.get("end"),"2017-04-11T10:45:15.729Z");
assertNull(info.get("count"),"1");
assertEquals(info.get("startInclusive"),"true");
assertEquals(info.get("endInclusive"),"true");
assertEquals(info.get("start"), "2015-09-29T10:45:15.729Z");
assertEquals(info.get("end"), "2017-04-11T10:45:15.729Z");
assertNull(info.get("count"), "1");
assertEquals(info.get("startInclusive"), "true");
assertEquals(info.get("endInclusive"), "true");
}
@Test
@@ -280,19 +251,9 @@ public class FacetRangeSearchTest extends AbstractSearchServicesE2ETest
public void searchDateAndSizeRanges()
{
SearchRequest query = createQuery("* AND SITE:'" + testSite.getId() + "'");
List<RestRequestRangesModel> ranges = new ArrayList<>();
RestRequestRangesModel facetRangeModel = new RestRequestRangesModel();
facetRangeModel.setField("created");
facetRangeModel.setStart("2015-09-29T10:45:15.729Z");
facetRangeModel.setEnd("2016-09-29T10:45:15.729Z");
facetRangeModel.setGap("+280DAY");
ranges.add(facetRangeModel);
RestRequestRangesModel facetCountRangeModel = new RestRequestRangesModel();
facetCountRangeModel.setField("content.size");
facetCountRangeModel.setStart("0");
facetCountRangeModel.setEnd("500");
facetCountRangeModel.setGap("200");
ranges.add(facetCountRangeModel);
RestRequestRangesModel facetRangeModel = createRangesModel("created", "2015-09-29T10:45:15.729Z", "2016-09-29T10:45:15.729Z", "+280DAY");
RestRequestRangesModel facetCountRangeModel = createRangesModel("content.size", "0", "500", "200");
List<RestRequestRangesModel> ranges = List.of(facetRangeModel, facetCountRangeModel);
query.setRanges(ranges);
}
@@ -304,16 +265,10 @@ public class FacetRangeSearchTest extends AbstractSearchServicesE2ETest
{
SearchRequest query = createQuery("* AND SITE:'" + testSite.getId() + "'");
RestRequestRangesModel facetRangeModel = new RestRequestRangesModel();
facetRangeModel.setField("content.size");
facetRangeModel.setStart("0");
facetRangeModel.setEnd("200");
facetRangeModel.setGap("20");
List<String> include = new ArrayList<>();
include.add("upper");
RestRequestRangesModel facetRangeModel = createRangesModel("content.size", "0", "200", "20");
List<String> include = List.of("upper");
facetRangeModel.setInclude(include);
List<RestRequestRangesModel> ranges = new ArrayList<>();
ranges.add(facetRangeModel);
List<RestRequestRangesModel> ranges = List.of(facetRangeModel);
query.setRanges(ranges);
SearchResponse response = query(query);
response.assertThat().entriesListIsNotEmpty();
@@ -323,14 +278,14 @@ public class FacetRangeSearchTest extends AbstractSearchServicesE2ETest
RestGenericBucketModel bucket = facetResponseModel.getBuckets().get(0);
bucket.assertThat().field("label").is("(20 - 40]");
bucket.assertThat().field("filterQuery").is("content.size:<\"20\" TO \"40\"]");
Map<String,String> metric = (Map<String, String>) bucket.getMetrics().get(0).getValue();
Map<String, String> metric = (Map<String, String>) bucket.getMetrics().get(0).getValue();
assertEquals(Integer.valueOf(metric.get("count")).intValue(), 2, "Unexpected count for first bucket.");
Map<String, String> info = (Map<String, String>) bucket.getBucketInfo();
assertEquals(info.get("start"),"20");
assertEquals(info.get("end"),"40");
assertEquals(info.get("start"), "20");
assertEquals(info.get("end"), "40");
assertNull(info.get("count"));
assertEquals(info.get("startInclusive"),"false");
assertEquals(info.get("endInclusive"),"true");
assertEquals(info.get("startInclusive"), "false");
assertEquals(info.get("endInclusive"), "true");
bucket = facetResponseModel.getBuckets().get(1);
bucket.assertThat().field("label").is("(40 - 120]");
@@ -338,10 +293,10 @@ public class FacetRangeSearchTest extends AbstractSearchServicesE2ETest
metric = (Map<String, String>) bucket.getMetrics().get(0).getValue();
assertEquals(Integer.valueOf(metric.get("count")).intValue(), 1, "Unexpected count for second bucket.");
info = (Map<String, String>) bucket.getBucketInfo();
assertEquals(info.get("start"),"40");
assertEquals(info.get("end"),"120");
assertEquals(info.get("startInclusive"),"false");
assertEquals(info.get("endInclusive"),"true");
assertEquals(info.get("start"), "40");
assertEquals(info.get("end"), "120");
assertEquals(info.get("startInclusive"), "false");
assertEquals(info.get("endInclusive"), "true");
bucket = facetResponseModel.getBuckets().get(2);
bucket.assertThat().field("label").is("(120 - 200]");
@@ -349,9 +304,28 @@ public class FacetRangeSearchTest extends AbstractSearchServicesE2ETest
metric = (Map<String, String>) bucket.getMetrics().get(0).getValue();
assertEquals(Integer.valueOf(metric.get("count")).intValue(), 1, "Unexpected count for third bucket.");
info = (Map<String, String>) bucket.getBucketInfo();
assertEquals(info.get("start"),"120");
assertEquals(info.get("end"),"200");
assertEquals(info.get("startInclusive"),"false");
assertEquals(info.get("endInclusive"),"true");
}
assertEquals(info.get("start"), "120");
assertEquals(info.get("end"), "200");
assertEquals(info.get("startInclusive"), "false");
assertEquals(info.get("endInclusive"), "true");
}
/**
* Create a ranges model with the values given.
*
* @param field The field to facet on.
* @param start The lowest facet value.
* @param end The highest facet value.
* @param gap The size of the buckets.
* @return The facet ranges model.
*/
private RestRequestRangesModel createRangesModel(String field, String start, String end, String gap)
{
RestRequestRangesModel facetRangeModel = new RestRequestRangesModel();
facetRangeModel.setField(field);
facetRangeModel.setStart(start);
facetRangeModel.setEnd(end);
facetRangeModel.setGap(gap);
return facetRangeModel;
}
}
e2e-test/src/test/java/org/alfresco/test/search/functional/searchServices/search/SearchAFTSInFieldTest.java
+326
View File
@@ -0,0 +1,326 @@
/*
* Copyright 2019 Alfresco Software, Ltd. All rights reserved.
* License rights for this program may be obtained from Alfresco Software, Ltd.
* pursuant to a written agreement and any use of this program without such an
* agreement is prohibited.
*/
package org.alfresco.test.search.functional.searchServices.search;
import static java.util.List.of;
import static jersey.repackaged.com.google.common.collect.Sets.newHashSet;
import java.util.ArrayList;
import java.util.List;
import java.util.Set;
import java.util.stream.Stream;
import org.alfresco.rest.model.RestNodeAssociationModelCollection;
import org.alfresco.rest.model.RestNodeChildAssociationModel;
import org.alfresco.rest.search.SearchResponse;
import org.alfresco.search.TestGroup;
import org.alfresco.test.search.functional.AbstractE2EFunctionalTest;
import org.alfresco.utility.model.FileModel;
import org.alfresco.utility.model.FileType;
import org.alfresco.utility.model.FolderModel;
import org.testng.Assert;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;
/**
* Test class tests AFTS Search In Field works
* Created for Search-840
*
* @author Meenal Bhave
*/
public class SearchAFTSInFieldTest extends AbstractE2EFunctionalTest
{
private FolderModel folder1, folder2;
private FileModel file1, file2, file3, file4, file5;
@BeforeClass(alwaysRun = true)
public void dataPreparation()
{
// Create Folders:
// Folder1: Expected to be found with file.txt
folder1 = new FolderModel("file txt folder");
dataContent.usingUser(testUser).usingSite(testSite).createFolder(folder1);
// Folder2: Not expected to be found with file.txt
folder2 = new FolderModel("txt files folder");
dataContent.usingUser(testUser).usingSite(testSite).createFolder(folder2);
// Create File(s): Expected to be found with file.txt
file1 = new FileModel("file.txt", "file.txt", "", FileType.TEXT_PLAIN, "file.txt");
file2 = new FileModel("1-file.txt", "1-file.txt", "", FileType.TEXT_PLAIN, "1-file.txt");
file3 = new FileModel("file1.txt", "file1.txt", "", FileType.TEXT_PLAIN, "file1.txt");
file4 = new FileModel("txt file", "txt file", "", FileType.TEXT_PLAIN, "txt file");
// Not Expected to be found with file.txt
file5 = new FileModel("txt files", "txt files", "", FileType.TEXT_PLAIN, "txt files");
of(file1, file2, file3, file4, file5).forEach(
f -> dataContent.usingUser(testUser).usingSite(testSite).usingResource(folder1).createContent(f));
waitForContentIndexing(file5.getContent(), true);
}
@Test(priority = 1, groups = { TestGroup.ACS_63n })
public void testSearchInFieldName()
{
// Field names in various formats
Stream<String> fieldNames = Stream.of("{http://www.alfresco.org/model/content/1.0}name",
"@{http://www.alfresco.org/model/content/1.0}name",
"cm_name",
"cm:name",
"@cm:name",
"name");
// For each field name, check that queries return consistent results with / out ''
fieldNames.forEach(fieldName ->
{
// Query string without quotes
String query = fieldName + ":file.txt";
Set<String> expectedNames = newHashSet();
expectedNames.add("file.txt"); // file1
expectedNames.add("1-file.txt"); // file2
expectedNames.add("file1.txt"); // file3
expectedNames.add("txt file"); // file4
expectedNames.add("file txt folder"); // folder1
testSearchQueryUnordered(query, expectedNames, SearchLanguage.AFTS);
// Query string in single quotes
query = fieldName + ":'file.txt'";
testSearchQueryUnordered(query, expectedNames, SearchLanguage.AFTS);
});
}
@Test(priority = 2, groups = { TestGroup.ACS_63n })
public void testSearchInFieldTitle()
{
// Field names in various formats
Stream<String> fieldNames = Stream.of("{http://www.alfresco.org/model/content/1.0}title",
"@{http://www.alfresco.org/model/content/1.0}title",
"cm_title",
"cm:title",
"@cm:title");
// For each field name, check that queries return consistent results with / out ''
fieldNames.forEach(fieldName ->
{
String query = fieldName + ":" + file2.getName();
boolean fileFound = isContentInSearchResults(query, file2.getName(), true);
Assert.assertTrue(fileFound, "File Not found for query: " + query);
testSearchQuery(query, 1, SearchLanguage.AFTS);
query = fieldName + ":'" + file2.getName() + "\'";
fileFound = isContentInSearchResults(query, file2.getName(), true);
Assert.assertTrue(fileFound, "File Not found for query: " + query);
testSearchQuery(query, 1, SearchLanguage.AFTS);
});
}
@Test(priority = 3, groups = { TestGroup.ACS_63n })
public void testSearchInFieldContent()
{
// Field names in various formats
List<String> fieldNames = new ArrayList<>();
fieldNames.add("TEXT");
fieldNames.add("{http://www.alfresco.org/model/dictionary/1.0}content");
fieldNames.add("cm:content");
fieldNames.add("d:content");
// For each field name, check that queries return consistent results with / out ''
fieldNames.forEach(fieldName ->
{
String query = fieldName + ":" + file3.getContent();
boolean fileFound = isContentInSearchResults(query, file3.getName(), true);
Assert.assertTrue(fileFound, "File Not found for query: " + query);
Integer resultCount1 = testSearchQuery(query, null, SearchLanguage.AFTS).getPagination().getTotalItems();
query = fieldName + ":'" + file3.getContent() + "\'";
fileFound = isContentInSearchResults(query, file3.getName(), true);
Assert.assertTrue(fileFound, "File Not found for query: " + query);
testSearchQuery(query, resultCount1, SearchLanguage.AFTS).getPagination().getTotalItems();
});
}
@Test(priority = 4)
public void testSearchInFieldSITE()
{
String query = "SITE:" + testSite.getId();
boolean fileFound = isContentInSearchResults(query, folder1.getName(), true);
Assert.assertTrue(fileFound, "Site Not found for query: " + query);
Integer resultCount1 = testSearchQuery(query, 8, SearchLanguage.AFTS).getPagination().getTotalItems();
query = "SITE:'" + testSite.getId() + "\'";
fileFound = isContentInSearchResults(query, folder1.getName(), true);
Assert.assertTrue(fileFound, "Site Not found for query: " + query);
testSearchQuery(query, resultCount1, SearchLanguage.AFTS).getPagination().getTotalItems();
}
@Test(priority = 5)
public void testSearchInFieldTYPE()
{
// Field names in various formats
List<String> fieldNames = new ArrayList<>();
fieldNames.add("TYPE");
fieldNames.add("EXACTTYPE");
// For each field name, check that queries return consistent results with / out ''
fieldNames.forEach(fieldName -> {
String query = fieldName + ":cm\\:content" + " and =cm:name:" + file1.getName();
boolean fileFound = isContentInSearchResults(query, file1.getName(), true);
Assert.assertTrue(fileFound, "Content Not found for query: " + query);
Integer resultCount1 = testSearchQuery(query, 1, SearchLanguage.AFTS).getPagination().getTotalItems();
query = fieldName + ":'cm:content'" + " and =cm:name:" + file1.getName();
fileFound = isContentInSearchResults(query, file1.getName(), true);
Assert.assertTrue(fileFound, "Content Not found for query: " + query);
testSearchQuery(query, resultCount1, SearchLanguage.AFTS).getPagination().getTotalItems();
});
}
@Test(priority = 6)
public void testSearchInFieldID() throws Exception
{
String query = "ID:'workspace://SpacesStore/" + file1.getNodeRefWithoutVersion() + "'";
boolean fileFound = isContentInSearchResults(query, file1.getName(), true);
Assert.assertTrue(fileFound, "Content Not found for query: " + query);
testSearchQuery(query, 1, SearchLanguage.AFTS).getPagination().getTotalItems();
}
@Test(priority = 7)
public void testSearchInFieldPARENT()
{
String query = "PARENT:" + folder1.getNodeRefWithoutVersion();
Set<String> expectedNames = newHashSet();
expectedNames.add(file1.getName());
expectedNames.add(file2.getName());
expectedNames.add(file3.getName());
expectedNames.add(file4.getName());
expectedNames.add(file5.getName());
testSearchQueryUnordered(query, expectedNames, SearchLanguage.AFTS);
query = "PARENT:'" + folder1.getNodeRefWithoutVersion() + "\'";
testSearchQueryUnordered(query, expectedNames, SearchLanguage.AFTS);
}
@Test(priority = 8)
public void testSearchInFieldPRIMARYPARENT() throws Exception
{
// Create Secondary association in testFolder2
RestNodeChildAssociationModel childAssoc1 = new RestNodeChildAssociationModel(file1.getNodeRefWithoutVersion(), "cm:contains");
String secondaryChildrenBody = "[" + childAssoc1.toJson() + "]";
restClient.authenticateUser(testUser).withCoreAPI().usingResource(folder2).createSecondaryChildren(secondaryChildrenBody);
RestNodeAssociationModelCollection secondaryChildren = restClient.authenticateUser(testUser).withCoreAPI().usingResource(folder2).getSecondaryChildren();
secondaryChildren.getEntryByIndex(0).assertThat().field("id").is(file1.getNodeRefWithoutVersion());
String query = "PRIMARYPARENT:'workspace://SpacesStore/" + folder1.getNodeRef() + "'";
Set<String> expectedNames = newHashSet();
expectedNames.add(file1.getName());
expectedNames.add(file2.getName());
expectedNames.add(file3.getName());
expectedNames.add(file4.getName());
expectedNames.add(file5.getName());
testSearchQueryUnordered(query, expectedNames, SearchLanguage.AFTS);
query = "PARENT:'workspace://SpacesStore/" + folder2.getNodeRef() + "'";
boolean fileFound = isContentInSearchResults(query, file1.getName(), true);
Assert.assertTrue(fileFound, "Expected Content Not found for query: " + query);
testSearchQuery(query, 1, SearchLanguage.AFTS);
}
@Test(priority = 9, groups = { TestGroup.ACS_63n })
public void testSearchInFieldNameExactMatch()
{
// Check that queries return consistent results with / out ''
String query = "=name:" + file1.getName();
boolean fileFound = isContentInSearchResults(query, file1.getName(), true);
Assert.assertTrue(fileFound, "File Not found for query: " + query);
Integer resultCount1 = testSearchQuery(query, 1, SearchLanguage.AFTS).getPagination().getTotalItems();
Assert.assertSame(resultCount1, 1, "File count does not match for query: " + query);
query = "=name:'" + file1.getName() + "\'";
fileFound = isContentInSearchResults(query, file1.getName(), true);
Assert.assertTrue(fileFound, "File Not found for query: " + query);
testSearchQuery(query, resultCount1, SearchLanguage.AFTS).getPagination().getTotalItems();
}
@Test(priority = 10, groups = { TestGroup.ACS_63n })
public void testSearchInFieldNameQueryExpansion()
{
// Check that queries return consistent results with / out ''
String query = "~name:" + file1.getName();
Set<String> expectedNames = newHashSet();
expectedNames.add(file1.getName());
expectedNames.add(file2.getName());
expectedNames.add(file3.getName());
expectedNames.add(file4.getName());
expectedNames.add(folder1.getName());
testSearchQueryUnordered(query, expectedNames, SearchLanguage.AFTS);
query = "~name:'" + file1.getName() + "\'";
testSearchQueryUnordered(query, expectedNames, SearchLanguage.AFTS);
}
@Test(priority = 11, groups = { TestGroup.ACS_63n })
public void testWithConjunctionDisjunctionAndNegation()
{
// Query string to include Conjunction, Disjunction and Negation
String query1 = "~name:" + file1.getName(); // Query expected to return 5 results
String query2 = "=name:" + file2.getName(); // Query expected to return 1 result
String query3 = "=name:" + file3.getName(); // Query expected to return 1 result
// Check Query with Conjunction, Negation, Disjunction: returns right results
// "~name:file.txt and ! (=name:1-file.txt or =name:file1.txt)"
String query = query1 + " and ! (" + query2 + " or " + query3 + ")";
// Check that expected files are included in the results
Set<String> expectedNames = newHashSet();
expectedNames.add("file.txt"); // file1
expectedNames.add("txt file"); // file4
expectedNames.add("file txt folder"); // folder1
SearchResponse response = testSearchQueryUnordered(query, expectedNames, SearchLanguage.AFTS);
// Check result count is 5-(1+1)=3
int resultCount = response.getPagination().getTotalItems();
Assert.assertEquals(resultCount, 3, "File count does not match for query: " + query);
// Check that file2 and file3 are excluded from the results
boolean fileFound = isContentInSearchResponse(response, file2.getName());
Assert.assertFalse(fileFound, "File2 found for query: " + query);
fileFound = isContentInSearchResponse(response, file3.getName());
Assert.assertFalse(fileFound, "File3 found for query: " + query);
}
}
e2e-test/src/test/java/org/alfresco/test/search/functional/searchServices/search/SearchSecondaryAssociationTest.java
+91
View File
@@ -0,0 +1,91 @@
/*
* Copyright 2019 Alfresco Software, Ltd. All rights reserved.
* License rights for this program may be obtained from Alfresco Software, Ltd.
* pursuant to a written agreement and any use of this program without such an
* agreement is prohibited.
*/
package org.alfresco.test.search.functional.searchServices.search;
import org.alfresco.rest.model.RestNodeAssociationModelCollection;
import org.alfresco.rest.model.RestNodeChildAssociationModel;
import org.alfresco.test.search.functional.AbstractE2EFunctionalTest;
import org.alfresco.utility.data.CustomObjectTypeProperties;
import org.alfresco.utility.model.FileModel;
import org.alfresco.utility.model.FolderModel;
import org.testng.Assert;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;
/**
* Test class tests content in the secondary parent is found too
* Created for Search-1313
*
* @author Meenal Bhave
*/
public class SearchSecondaryAssociationTest extends AbstractE2EFunctionalTest
{
private FolderModel testFolder1, testFolder2;
private FileModel file1;
@BeforeClass(alwaysRun = true)
public void dataPreparation()
{
// Folders
testFolder1 = new FolderModel("folder1");
testFolder2 = new FolderModel("folder2");
// File(s)
file1 = new FileModel("file1.txt");
file1.setContent("content file 1");
// Create folder1
dataContent.usingUser(testUser).usingSite(testSite).createCustomContent(testFolder1, "cmis:folder", new CustomObjectTypeProperties());
// Create file1
dataContent.usingUser(testUser).usingResource(testFolder1).createCustomContent(file1, "cmis:document", new CustomObjectTypeProperties());
// Create folder2
dataContent.usingUser(testUser).usingSite(testSite).createCustomContent(testFolder2, "cmis:folder", new CustomObjectTypeProperties());
// wait for solr index
waitForMetadataIndexing(file1.getName(), true);
}
@Test(priority = 1)
public void testSearchPathForSecondaryAssociation() throws Exception
{
String queryPathFolder1 = "PATH:\"/app:company_home/st:sites/cm:" + testSite.getTitle() +
"/cm:documentLibrary/cm:" + testFolder1.getName() + "/cm:" + file1.getName() + "\"";
// Test if file can be found in folder1: Primary Parent
boolean found = isContentInSearchResults(queryPathFolder1, file1.getName(), true);
Assert.assertTrue(found, "File Not found using Primary Parent Path");
String queryPathFolder2 = "PATH:\"/app:company_home/st:sites/cm:" + testSite.getTitle() +
"/cm:documentLibrary/cm:" + testFolder2.getName() + "/cm:" + file1.getName() + "\"";
// Test if file can not be found in folder2
found = isContentInSearchResults(queryPathFolder2, file1.getName(), false);
Assert.assertTrue(found, "File found using Secondary Parent Path");
// Create Secondary association in folder2
RestNodeChildAssociationModel childAssoc1 = new RestNodeChildAssociationModel(file1.getNodeRefWithoutVersion(), "cm:contains");
String secondaryChildrenBody = "[" + childAssoc1.toJson() + "]";
restClient.authenticateUser(testUser).withCoreAPI().usingResource(testFolder2).createSecondaryChildren(secondaryChildrenBody);
RestNodeAssociationModelCollection secondaryChildren = restClient.authenticateUser(testUser).withCoreAPI().usingResource(testFolder2).getSecondaryChildren();
secondaryChildren.getEntryByIndex(0).assertThat().field("id").is(file1.getNodeRefWithoutVersion());
// Test if file can be found in folder2: Secondary Parent
found = isContentInSearchResults(queryPathFolder2, file1.getName(), true);
Assert.assertTrue(found, "File Not found using Secondary Parent Path");
// Remove Secondary association
restClient.authenticateUser(testUser).withCoreAPI().usingResource(testFolder2).deleteSecondaryChild(secondaryChildren.getEntryByIndex(0));
// Test if file can not be found in folder2
found = isContentInSearchResults(queryPathFolder2, file1.getName(), false);
Assert.assertTrue(found, "File found using Secondary Parent Path");
}
}
e2e-test/src/test/java/org/alfresco/test/search/functional/searchServices/search/ShardInfoTest.java
+1
View File
@@ -101,6 +101,7 @@ public class ShardInfoTest extends AbstractE2EFunctionalTest
RestShardInfoModel model = shardInfoModel.getModel();
assertEquals(model.getTemplate(), "rerank");
assertEquals(model.getShardMethod(), "DB_ID");
assertEquals(model.getMode(), "MIXED");
assertTrue(model.getHasContent());
assertTrue(stores.contains(model.getStores()));
e2e-test/src/test/java/org/alfresco/test/search/functional/searchServices/search/tracker/CascadingTrackerIntegrationTest.java → e2e-test/src/test/java/org/alfresco/test/search/functional/searchServices/search/tracker/CascadingIntegrationTest.java
+1 -1
View File
@@ -31,7 +31,7 @@ import static org.testng.Assert.assertTrue;
* @author Alessandro Benedetti
* @author Meenal Bhave
*/
public class CascadingTrackerIntegrationTest extends AbstractE2EFunctionalTest
public class CascadingIntegrationTest extends AbstractE2EFunctionalTest
{
@Autowired
protected DataContent dataContent;
e2e-test/src/test/resources/testdata/search-by-property.xml
Vendored
-188
View File
@@ -1,188 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<testData xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xsi:noNamespaceSchemaLocation="https://gitlab.alfresco.com/tas/tas-namespaces/raw/master/input-test-data.xsd">
<queries>
<query
value="SELECT * FROM tas:document where cmis:name = 'testc.txt'"
expectedResults="1" />
<query
value="SELECT * FROM tas:document where cmis:name &lt;&gt; 'testc.txt'"
expectedResults="7" />
<query
value="SELECT * FROM tas:document where cmis:name &lt; 'testc.txt'"
expectedResults="3" />
<query
value="SELECT * FROM tas:document where cmis:name &lt;= 'testc.txt'"
expectedResults="4" />
<query
value="SELECT * FROM tas:document where cmis:name &gt; 'testc.txt'"
expectedResults="4" />
<query
value="SELECT * FROM tas:document where cmis:name &gt;= 'testc.txt'"
expectedResults="5" />
<query
value="SELECT * FROM tas:document where cmis:name IN('testc.txt', 'guestc.txt', 'restc.txt')"
expectedResults="3" />
<query
value="SELECT * FROM tas:document where cmis:name NOT IN('testc.txt', 'gustc.txt', 'restc.txt')"
expectedResults="6" />
<query
value="SELECT * FROM tas:document where cmis:name LIKE '%testc%'"
expectedResults="4" />
<query
value="SELECT * FROM tas:document where cmis:name LIKE 'testc.txt'"
expectedResults="1" />
<query
value="SELECT * FROM tas:document where cmis:name LIKE 't%tc.txt'"
expectedResults="2" />
<query
value="SELECT * FROM tas:document where cmis:name LIKE 't__tc.txt'"
expectedResults="1" />
<query
value="SELECT * FROM tas:folder where cmis:name = 'testf'"
expectedResults="1" />
<query
value="SELECT * FROM tas:folder where cmis:name &lt;&gt; 'testf'"
expectedResults="8" />
<query
value="SELECT * FROM tas:folder where cmis:name &lt; 'testf'"
expectedResults="3" />
<query
value="SELECT * FROM tas:folder where cmis:name &lt;= 'testf'"
expectedResults="4" />
<query
value="SELECT * FROM tas:folder where cmis:name &gt; 'testf'"
expectedResults="5" />
<query
value="SELECT * FROM tas:folder where cmis:name &gt;= 'testf'"
expectedResults="6" />
<query
value="SELECT * FROM tas:folder where cmis:name IN('testf', 'guestf', 'restf')"
expectedResults="3" />
<query
value="SELECT * FROM tas:folder where cmis:name NOT IN('testf', 'gustf', 'restf')"
expectedResults="7" />
<query
value="SELECT * FROM tas:folder where cmis:name LIKE '%testf%'"
expectedResults="5" />
<query
value="SELECT * FROM tas:folder where cmis:name LIKE 'testf'"
expectedResults="1" />
<query
value="SELECT * FROM tas:folder where cmis:name LIKE 't%tf'"
expectedResults="2" />
<query
value="SELECT * FROM tas:folder where cmis:name LIKE 't__tf'"
expectedResults="1" />
<query
value="SELECT * FROM tas:document where cmis:name LIKE '%testc%' ORDER BY cmis:name ASC"
expectedResults="4" />
<query
value="SELECT * FROM tas:document where cmis:name LIKE '%testc%' ORDER BY cmis:name DESC"
expectedResults="4" />
<query
value="SELECT * FROM tas:document where cmis:name LIKE '%testc%' ORDER BY cmis:name DESC, cmis:creationDate ASC"
expectedResults="4" />
<query
value="SELECT * FROM tas:folder where cmis:name LIKE '%testf%' ORDER BY cmis:name ASC"
expectedResults="5" />
<query
value="SELECT * FROM tas:folder where cmis:name LIKE '%testf%' ORDER BY cmis:name DESC"
expectedResults="5" />
<query
value="SELECT * FROM tas:folder where cmis:name LIKE '%testf%' ORDER BY cmis:name DESC, cmis:creationDate ASC"
expectedResults="5" />
<query
value="SELECT * FROM tas:document where tas:TextPropertyC = 'restc text'"
expectedResults="1" />
<query
value="SELECT * FROM tas:document where tas:TextPropertyC &lt;&gt; 'testc1 text'"
expectedResults="7" />
<query
value="SELECT * FROM tas:document where tas:TextPropertyC IN('restc text', 'testc2 text', 'testc3 text')"
expectedResults="3" />
<query
value="SELECT * FROM tas:document where tas:TextPropertyC NOT IN('restc text', 'testc2 text', 'testc3 text')"
expectedResults="5" />
<query
value="SELECT * FROM tas:document where tas:TextPropertyC LIKE '%restc%'"
expectedResults="1" />
<query
value="SELECT * FROM tas:document where tas:TextPropertyC &gt;= 'testc'"
expectedResults="5" />
<query
value="SELECT * FROM tas:document where tas:TextPropertyC LIKE 't__tc text'"
expectedResults="1" />
<query
value="SELECT * FROM tas:folder where tas:TextPropertyF = 'restf text'"
expectedResults="1" />
<query
value="SELECT * FROM tas:folder where tas:TextPropertyF &lt;&gt; 'testf1 text'"
expectedResults="8" />
<query
value="SELECT * FROM tas:folder where tas:TextPropertyF IN('restf text', 'testf2 text', 'testf3 text')"
expectedResults="3" />
<query
value="SELECT * FROM tas:folder where tas:TextPropertyF NOT IN('restf text', 'testf2 text', 'testf3 text')"
expectedResults="6" />
<query
value="SELECT * FROM tas:folder where tas:TextPropertyF LIKE '%restf%'"
expectedResults="1" />
<query
value="SELECT * FROM tas:folder where tas:TextPropertyF &gt;= 'testf text'"
expectedResults="6" />
<query
value="SELECT * FROM tas:folder where tas:TextPropertyF LIKE 't__tf text'"
expectedResults="1" />
<query
value="SELECT * FROM tas:document where tas:IntPropertyC = '222'"
expectedResults="1" />
<query
value="SELECT * FROM tas:document where tas:IntPropertyC &lt;&gt; '223'"
expectedResults="7" />
<query
value="SELECT * FROM tas:document where tas:IntPropertyC &lt; '223'"
expectedResults="1" />
<query
value="SELECT * FROM tas:document where tas:IntPropertyC &lt;= '224'"
expectedResults="3" />
<query
value="SELECT * FROM tas:document where tas:IntPropertyC &gt;= '224'"
expectedResults="6" />
<query
value="SELECT * FROM tas:document where tas:IntPropertyC &gt; '224'"
expectedResults="5" />
<query
value="SELECT * FROM tas:document where tas:IntPropertyC IN('222', '223', '224', '225')"
expectedResults="4" />
<query
value="SELECT * FROM tas:document where tas:IntPropertyC NOT IN('222', '223', '224')"
expectedResults="5" />
<query
value="SELECT * FROM tas:folder where tas:IntPropertyF = '222'"
expectedResults="1" />
<query
value="SELECT * FROM tas:folder where tas:IntPropertyF &lt;&gt; '223'"
expectedResults="8" />
<query
value="SELECT * FROM tas:folder where tas:IntPropertyF &lt; '223'"
expectedResults="1" />
<query
value="SELECT * FROM tas:folder where tas:IntPropertyF &lt;= '224'"
expectedResults="3" />
<query
value="SELECT * FROM tas:folder where tas:IntPropertyF &gt;= '224'"
expectedResults="7" />
<query
value="SELECT * FROM tas:folder where tas:IntPropertyF &gt; '224'"
expectedResults="6" />
<query
value="SELECT * FROM tas:folder where tas:IntPropertyF IN('222', '223', '224', '225')"
expectedResults="4" />
<query
value="SELECT * FROM tas:folder where tas:IntPropertyF NOT IN('222', '223', '224')"
expectedResults="6" />
</queries>
</testData>
pom.xml
+4 -1
View File
@@ -29,7 +29,7 @@
<properties>
<java.version>11</java.version>
<solr.base.version>6.6.5</solr.base.version>
<solr.version>${solr.base.version}-patched.1</solr.version>
<solr.version>${solr.base.version}-patched.2</solr.version>
<!-- The location to download the solr zip file from. -->
<!-- <solr.zip>https://archive.apache.org/dist/lucene/solr/${solr.version}/solr-${solr.version}.zip</solr.zip> -->
<!-- Solr startup scripts do not work with any Java version higher than 9 so the scripts have been patched -->
@@ -66,6 +66,9 @@
</goals>
</execution>
</executions>
<configuration>
<runOrder>alphabetical</runOrder>
</configuration>
</plugin>
</plugins>
</build>
search-services/README.md
+16
View File
@@ -338,6 +338,22 @@ This Docker Image is available at Alfresco Docker Hub:
To use the public image instead of the local one (`searchservices:develop`) just use `alfresco/alfresco-search-services:1.3.x.x` labels.
## Docker Master-Slave setup
### Enable Search Slave Replica config
To enable slave node specify environment value `REPLICATION_TYPE=slave`, by default Master config is enabled and slave is disabled.
During deployment time whenever Search Services or Insight Engine image starts, it will execute the script [search_config_setup.sh](/packaging/src/docker) which will configure the slave config setup based on the value specified in the script.
To run the docker image:
```bash
$ docker run -p 8984:8983 -e REPLICATION_TYPE=slave -e ALFRESCO_SECURE_COMMS=none -e SOLR_CREATE_ALFRESCO_DEFAULTS=alfresco,archive searchservices:develop
```
Solr-slave End point: [http://localhost:8984/solr](http://localhost:8984/solr)
To generate your own Docker-compose file please follow [generator-alfresco-docker-compose](../e2e-test/generator-alfresco-docker-compose/README.md)
### Use Alfresco Search Services Docker Image with Docker Compose
Sample configuration in a Docker Compose file using **Plain HTTP** protocol to communicate with Alfresco Repository.
search-services/alfresco-search/doc/architecture/trackers/00001-content-tracker.md
-1
View File
@@ -155,7 +155,6 @@ The following table illustrates the configuration properties used by the Tracker
|shard.method|"DB_ID"|Data (Documents, ACLs) Routing criteria among shards| | |Y|Y| | |
|alfresco.fingerprint|true|true if we want to compute the content Fingerprint| | |Y|| | |
|alfresco.index.transformContent|true| | | |Y|| | |
|alfresco.version|5.0.0|The target Alfresco version| | | | | | |
|alfresco.corePoolSize|4|The number of threads to keep in the pool, even if they are idle|Y|Y|Y|Y|Y|Y|
|alfresco.maximumPoolSize|-1|The maximum number of threads allowed in the pool|Y|Y|Y|Y|Y|Y|
|alfresco.keepAliveTime|120|When the number of threads is greater than the core pool size, this is the maximum time that excess idle threads will wait for new tasks before terminating|Y|Y|Y|Y|Y|Y|
search-services/alfresco-search/pom.xml
+101 -3
View File
@@ -96,18 +96,79 @@
<version>2.3.2</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.9</version>
</dependency>
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-core</artifactId>
<version>${cxf.version}</version>
</dependency>
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-bindings-soap</artifactId>
<version>${cxf.version}</version>
</dependency>
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-bindings-xml</artifactId>
<version>${cxf.version}</version>
</dependency>
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-databinding-jaxb</artifactId>
<version>${cxf.version}</version>
</dependency>
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-frontend-jaxws</artifactId>
<version>${cxf.version}</version>
</dependency>
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-frontend-simple</artifactId>
<version>${cxf.version}</version>
</dependency>
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-transports-http</artifactId>
<version>${cxf.version}</version>
</dependency>
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-ws-addr</artifactId>
<version>${cxf.version}</version>
</dependency>
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-ws-policy</artifactId>
<version>${cxf.version}</version>
</dependency>
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-wsdl</artifactId>
<version>${cxf.version}</version>
</dependency>
<dependency>
<groupId>xpp3</groupId>
<artifactId>xpp3</artifactId>
<version>1.1.4c</version>
</dependency>
<!-- Test dependencies -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<version>4.13</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<version>3.1.0</version>
<version>3.2.4</version>
<scope>test</scope>
</dependency>
@@ -120,7 +181,7 @@
<dependency>
<groupId>com.carrotsearch.randomizedtesting</groupId>
<artifactId>randomizedtesting-runner</artifactId>
<version>2.7.4</version>
<version>2.7.6</version>
<scope>test</scope>
</dependency>
<dependency>
@@ -131,6 +192,43 @@
</dependency>
</dependencies>
<repositories>
<repository>
<id>alfresco-public</id>
<url>https://artifacts.alfresco.com/nexus/content/groups/public</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
<repository>
<id>alfresco-public-snapshots</id>
<url>https://artifacts.alfresco.com/nexus/content/groups/public-snapshots</url>
<releases>
<enabled>false</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
<repository>
<id>central</id>
<name>Central Repository</name>
<url>https://repo.maven.apache.org/maven2</url>
<layout>default</layout>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
<repository>
<id>maven-restlet</id>
<name>Public online Restlet repository</name>
<url>http://maven.restlet.talend.com</url>
</repository>
</repositories>
<build>
<finalName>alfresco-solr</finalName>
<plugins>
search-services/alfresco-search/src/main/java/org/alfresco/solr/AlfrescoCollatableMLTextFieldType.java
+26 -23
View File
@@ -39,12 +39,9 @@ import org.springframework.extensions.surf.util.I18NUtil;
/**
* @author Andy
*
*/
public class AlfrescoCollatableMLTextFieldType extends StrField
{
/* (non-Javadoc)
* @see org.apache.solr.schema.StrField#getSortField(org.apache.solr.schema.SchemaField, boolean)
*/
@@ -75,7 +72,6 @@ public class AlfrescoCollatableMLTextFieldType extends StrField
}
public static class MLTextSortFieldComparatorSource extends FieldComparatorSource
{
@@ -101,16 +97,20 @@ public class AlfrescoCollatableMLTextFieldType extends StrField
private final String[] values;
private BinaryDocValues docTerms;
private Bits docsWithField;
/**
* An array of flags - one for each document in the segment. Each bit is set to true if the document has the
* field or false otherwise. If this is set to null then all docs in the segment have the field.
*/
Bits docsWithField;
private final String field;
final Collator collator;
Collator collator;
private String bottom;
private String top;
String bottom;
String top;
Locale collatorLocale;
@@ -138,7 +138,7 @@ public class AlfrescoCollatableMLTextFieldType extends StrField
{
final String comparableString = findBestValue(doc, docTerms.get(doc));
return compareValues(bottom, comparableString);
}
public void copy(int slot, int doc)
@@ -153,13 +153,14 @@ public class AlfrescoCollatableMLTextFieldType extends StrField
private String findBestValue(int doc, BytesRef term)
{
if (term.length == 0 && docsWithField.get(doc) == false) {
if (term.length == 0 && docsWithField != null && docsWithField.get(doc) == false)
{
return null;
}
String withLocale = term.utf8ToString();
// split strin into MLText object
// split string into MLText object
if (withLocale == null)
{
return withLocale;
@@ -231,14 +232,15 @@ public class AlfrescoCollatableMLTextFieldType extends StrField
{
docTerms = DocValues.getBinary(context.reader(), field);
docsWithField = DocValues.getDocsWithField(context.reader(), field);
if (docsWithField instanceof Bits.MatchAllBits) {
docsWithField = null;
if (docsWithField instanceof Bits.MatchAllBits)
{
docsWithField = null;
}
return this;
}
@Override
public int compareValues(String val1, String val2)
public int compareValues(String val1, String val2)
{
if (val1 == null)
{
@@ -254,9 +256,10 @@ public class AlfrescoCollatableMLTextFieldType extends StrField
}
return collator.compare(val1, val2);
}
@Override
public void setScorer(Scorer scorer) {}
}
@Override
public void setScorer(Scorer scorer)
{
}
}
}
search-services/alfresco-search/src/main/java/org/alfresco/solr/AlfrescoCollatableTextFieldType.java
+11 -7
View File
@@ -104,16 +104,20 @@ public class AlfrescoCollatableTextFieldType extends StrField
private final String[] values;
private BinaryDocValues docTerms;
private Bits docsWithField;
/**
* An array of flags - one for each document in the segment. Each bit is set to true if the document has the
* field or false otherwise. If this is set to null then all docs in the segment have the field.
*/
Bits docsWithField;
private final String field;
final Collator collator;
Collator collator;
private String bottom;
String bottom;
private String top;
String top;
Locale collatorLocale;
@@ -141,7 +145,6 @@ public class AlfrescoCollatableTextFieldType extends StrField
{
final String comparableString = findBestValue(doc, docTerms.get(doc));
return compareValues(bottom, comparableString);
}
public void copy(int slot, int doc)
@@ -156,7 +159,8 @@ public class AlfrescoCollatableTextFieldType extends StrField
private String findBestValue(int doc, BytesRef term)
{
if (term.length == 0 && docsWithField.get(doc) == false) {
if (term.length == 0 && docsWithField != null && docsWithField.get(doc) == false)
{
return null;
}
search-services/alfresco-search/src/main/java/org/alfresco/solr/AlfrescoSolrDataModel.java
+1 -1
View File
@@ -523,7 +523,7 @@ public class AlfrescoSolrDataModel implements QueryConstants
}
catch (IOException e)
{
log.info("Failed to read shared properties fat " + propertiesFile.getAbsolutePath());
log.info("Failed to read shared properties at " + propertiesFile.getAbsolutePath());
}
return props;
search-services/alfresco-search/src/main/java/org/alfresco/solr/HandlerReportHelper.java
-10
View File
@@ -145,9 +145,6 @@ class HandlerReportHelper
return payload;
}
/**
* Builds Tracker report
*/
static NamedList<Object> buildTrackerReport(TrackerRegistry trackerRegistry, InformationServer srv, String coreName, Long fromTx, Long toTx, Long fromAclTx, Long toAclTx,
Long fromTime, Long toTime) throws JSONException
{
@@ -157,7 +154,6 @@ class HandlerReportHelper
AclTracker aclTracker = trackerRegistry.getTrackerForCore(coreName, AclTracker.class);
IndexHealthReport aclReport = aclTracker.checkIndex(toTx, toAclTx, fromTime, toTime);
NamedList<Object> ihr = new SimpleOrderedMap<>();
ihr.add("Alfresco version", aclTracker.getAlfrescoVersion());
ihr.add("DB acl transaction count", aclReport.getDbAclTransactionCount());
ihr.add("Count of duplicated acl transactions in the index", aclReport.getDuplicatedAclTxInIndex()
.cardinality());
@@ -277,12 +273,6 @@ class HandlerReportHelper
Date end = new Date(now.getTime() + remainingTxTimeMillis);
Duration remainingTx = new Duration(now, end);
long remainingChangeSetTimeMillis = 0;
now = new Date();
end = new Date(now.getTime() + remainingChangeSetTimeMillis);
Duration remainingChangeSet = new Duration(now, end);
NamedList<Object> ftsSummary = new SimpleOrderedMap<>();
long remainingContentTimeMillis = 0;
srv.addFTSStatusCounts(ftsSummary);
search-services/alfresco-search/src/main/java/org/alfresco/solr/InformationServer.java
+8 -1
View File
@@ -88,7 +88,7 @@ public interface InformationServer extends InformationServerCollectionProvider
void indexNode(Node node, boolean overwrite) throws IOException, AuthenticationException, JSONException;
void indexNodes(List<Node> nodes, boolean overwrite, boolean cascade) throws IOException, AuthenticationException, JSONException;
void indexNodes(List<Node> nodes, boolean overwrite) throws IOException, AuthenticationException, JSONException;
void cascadeNodes(List<NodeMetaData> nodes, boolean overwrite) throws IOException, AuthenticationException, JSONException;
@@ -181,4 +181,11 @@ public interface InformationServer extends InformationServerCollectionProvider
String getBaseUrl();
void flushContentStore() throws IOException;
/**
* Check if cascade tracking is enabled.
*
* @return true if cascade tracking is enabled (note that this is the default behaviour if not specified in the properties file).
*/
boolean cascadeTrackingEnabled();
}
search-services/alfresco-search/src/main/java/org/alfresco/solr/SolrInformationServer.java
+188 -110
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2015 Alfresco Software Limited.
* Copyright (C) 2019 Alfresco Software Limited.
*
* This file is part of Alfresco
*
@@ -19,6 +19,7 @@
package org.alfresco.solr;
import static java.util.Optional.ofNullable;
import static org.alfresco.repo.search.adaptor.lucene.QueryConstants.FIELD_ACLID;
import static org.alfresco.repo.search.adaptor.lucene.QueryConstants.FIELD_ACLTXCOMMITTIME;
import static org.alfresco.repo.search.adaptor.lucene.QueryConstants.FIELD_ACLTXID;
@@ -76,16 +77,32 @@ import java.io.PrintWriter;
import java.io.StringWriter;
import java.nio.charset.StandardCharsets;
import java.text.DecimalFormat;
import java.util.*;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections;
import java.util.EnumMap;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Iterator;
import java.util.LinkedHashMap;
import java.util.LinkedHashSet;
import java.util.LinkedList;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Map.Entry;
import java.util.Objects;
import java.util.Properties;
import java.util.Set;
import java.util.concurrent.locks.ReentrantReadWriteLock;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.util.zip.GZIPInputStream;
import com.carrotsearch.hppc.IntArrayList;
import com.carrotsearch.hppc.LongHashSet;
import com.carrotsearch.hppc.cursors.LongCursor;
import org.alfresco.httpclient.AuthenticationException;
import org.alfresco.model.ContentModel;
import org.alfresco.opencmis.dictionary.CMISStrictDictionaryService;
@@ -134,13 +151,26 @@ import org.apache.lucene.analysis.Analyzer;
import org.apache.lucene.analysis.TokenStream;
import org.apache.lucene.analysis.tokenattributes.CharTermAttribute;
import org.apache.lucene.document.Document;
import org.apache.lucene.document.LongPoint;
import org.apache.lucene.index.IndexCommit;
import org.apache.lucene.index.IndexableField;
import org.apache.lucene.index.LeafReaderContext;
import org.apache.lucene.index.NumericDocValues;
import org.apache.lucene.index.ReaderUtil;
import org.apache.lucene.index.Term;
import org.apache.lucene.search.*;
import org.apache.lucene.search.BooleanClause;
import org.apache.lucene.search.BooleanQuery;
import org.apache.lucene.search.Collector;
import org.apache.lucene.search.LeafCollector;
import org.apache.lucene.search.LegacyNumericRangeQuery;
import org.apache.lucene.search.Query;
import org.apache.lucene.search.ScoreDoc;
import org.apache.lucene.search.Scorer;
import org.apache.lucene.search.Sort;
import org.apache.lucene.search.SortField;
import org.apache.lucene.search.TermQuery;
import org.apache.lucene.search.TopDocs;
import org.apache.lucene.search.TopFieldCollector;
import org.apache.lucene.util.BytesRefBuilder;
import org.apache.solr.common.SolrDocument;
import org.apache.solr.common.SolrDocumentList;
@@ -164,6 +194,7 @@ import org.apache.solr.schema.SchemaField;
import org.apache.solr.search.DelegatingCollector;
import org.apache.solr.search.DocIterator;
import org.apache.solr.search.DocList;
import org.apache.solr.search.DocSet;
import org.apache.solr.search.QueryWrapperFilter;
import org.apache.solr.search.SolrIndexSearcher;
import org.apache.solr.update.AddUpdateCommand;
@@ -178,14 +209,14 @@ import org.springframework.extensions.surf.util.I18NUtil;
import org.springframework.util.FileCopyUtils;
/**
* This is the Solr4 implementation of the information server (index).
* This is the Apache Solr implementation of the information server (index).
*
* @author Ahmed Owian
* @since 5.0
*/
public class SolrInformationServer implements InformationServer
{
private final static Log LOGGER = new Log(SolrInformationServer.class);
private final static long TWO_MINUTES = 120000;
private static final String NO_SITE = "_REPOSITORY_";
private static final String SHARED_FILES = "_SHARED_FILES_";
@@ -200,23 +231,20 @@ public class SolrInformationServer implements InformationServer
public static final String AND = " AND ";
public static final String OR = " OR ";
public static final String REQUEST_HANDLER_ALFRESCO_FULL_TEXT_SEARCH = "/afts";
public static final String REQUEST_HANDLER_NATIVE = "/native";
public static final String REQUEST_HANDLER_ALFRESCO = "/alfresco";
public static final String REQUEST_HANDLER_SELECT = "/select";
public static final String REQUEST_HANDLER_GET = "/get";
public static final String RESPONSE_DEFAULT_IDS = "response";
public static final String RESPONSE_DEFAULT_ID = "doc";
private static final String REQUEST_HANDLER_NATIVE = "/native";
static final String REQUEST_HANDLER_GET = "/get";
private static final String RESPONSE_DEFAULT_IDS = "response";
static final String RESPONSE_DEFAULT_ID = "doc";
public static final String PREFIX_ERROR = "ERROR-";
static final String PREFIX_ERROR = "ERROR-";
public static final String DOC_TYPE_NODE = "Node";
public static final String DOC_TYPE_UNINDEXED_NODE = "UnindexedNode";
public static final String DOC_TYPE_ERROR_NODE = "ErrorNode";
private static final String DOC_TYPE_UNINDEXED_NODE = "UnindexedNode";
private static final String DOC_TYPE_ERROR_NODE = "ErrorNode";
public static final String DOC_TYPE_ACL = "Acl";
public static final String DOC_TYPE_TX = "Tx";
public static final String DOC_TYPE_ACL_TX = "AclTx";
public static final String DOC_TYPE_STATE = "State";
private static final String DOC_TYPE_STATE = "State";
public static final String SOLR_HOST = "solr.host";
public static final String SOLR_PORT = "solr.port";
@@ -227,6 +255,8 @@ public class SolrInformationServer implements InformationServer
*/
private static final int BATCH_FACET_TXS = 4096;
private static final String FINGERPRINT_FIELD = "MINHASH";
/** Shared property to determine if the cascade tracking is enabled. */
public static final String CASCADE_TRACKER_ENABLED = "alfresco.cascade.tracker.enabled";
private final AlfrescoCoreAdminHandler adminHandler;
private final SolrCore core;
@@ -235,7 +265,6 @@ public class SolrInformationServer implements InformationServer
private final TrackerStats trackerStats = new TrackerStats(this);
private final AlfrescoSolrDataModel dataModel;
private final SolrContentStore solrContentStore;
private final String alfrescoVersion;
private final boolean transformContent;
private final boolean recordUnindexedNodes;
private final long lag;
@@ -271,7 +300,7 @@ public class SolrInformationServer implements InformationServer
protected enum FTSStatus {New, Dirty, Clean}
class DocListCollector implements Collector, LeafCollector
static class DocListCollector implements Collector, LeafCollector
{
private IntArrayList docs = new IntArrayList();
private int docBase;
@@ -303,7 +332,7 @@ public class SolrInformationServer implements InformationServer
}
}
class TxnCacheFilter extends DelegatingCollector
static class TxnCacheFilter extends DelegatingCollector
{
private NumericDocValues currentLongs;
private Map txnLRU;
@@ -330,7 +359,7 @@ public class SolrInformationServer implements InformationServer
}
}
class TxnCollector extends DelegatingCollector
static class TxnCollector extends DelegatingCollector
{
private NumericDocValues currentLongs;
private long txnFloor;
@@ -372,7 +401,7 @@ public class SolrInformationServer implements InformationServer
}
}
class LRU extends LinkedHashMap<Long,Long>
static class LRU extends LinkedHashMap<Long,Long>
{
private int maxSize;
@@ -395,7 +424,7 @@ public class SolrInformationServer implements InformationServer
boolean isDefinitionExists(QName qName);
}
abstract class TransactionInfoReporter
static abstract class TransactionInfoReporter
{
protected final IndexHealthReport report;
@@ -428,7 +457,6 @@ public class SolrInformationServer implements InformationServer
this.solrContentStore = solrContentStore;
Properties p = core.getResourceLoader().getCoreProperties();
alfrescoVersion = p.getProperty("alfresco.version", "Unknown");
transformContent = Boolean.parseBoolean(p.getProperty("alfresco.index.transformContent", "true"));
recordUnindexedNodes = Boolean.parseBoolean(p.getProperty("alfresco.recordUnindexedNodes", "true"));
lag = Integer.parseInt(p.getProperty("alfresco.lag", "1000"));
@@ -438,7 +466,7 @@ public class SolrInformationServer implements InformationServer
dataModel = AlfrescoSolrDataModel.getInstance();
contentStreamLimit = Integer.parseInt(p.getProperty("alfresco.contentStreamLimit", "10000000"));
// build base URL - host and port have to come from configuration.
props = AlfrescoSolrDataModel.getCommonConfig();
hostName = ConfigUtil.locateProperty(SOLR_HOST, props.getProperty(SOLR_HOST));
@@ -453,6 +481,14 @@ public class SolrInformationServer implements InformationServer
return this.adminHandler;
}
/** {@inheritDoc} */
@Override
public boolean cascadeTrackingEnabled()
{
String cascadeTrackerEnabledProp = ofNullable((String) props.get(CASCADE_TRACKER_ENABLED)).orElse("true");
return Boolean.valueOf(cascadeTrackerEnabledProp);
}
@Override
public synchronized void initSkippingDescendantDocs()
{
@@ -504,12 +540,7 @@ public class SolrInformationServer implements InformationServer
report.add("Node count with FTSStatus New", newCount);
}
}
public String getAlfrescoVersion()
{
return this.alfrescoVersion;
}
@Override
public void afterInitModels()
{
@@ -522,7 +553,7 @@ public class SolrInformationServer implements InformationServer
String query = FIELD_ACLID + ":" + aclid + AND + FIELD_DOC_TYPE + ":" + DOC_TYPE_ACL;
long count = this.getDocListSize(query);
aclReport.setIndexedAclDocCount(count);
// TODO Could add INACLTXID later, but would need acl change set id.
return aclReport;
}
@@ -780,7 +811,7 @@ public class SolrInformationServer implements InformationServer
long count = this.getDocListSize(query);
nodeReport.setIndexedNodeDocCount(count);
}
@Override
public void commit() throws IOException
{
@@ -893,26 +924,26 @@ public class SolrInformationServer implements InformationServer
return searcherOpened;
}
@Override
public void deleteByAclChangeSetId(Long aclChangeSetId) throws IOException
{
deleteById(FIELD_INACLTXID, aclChangeSetId);
}
@Override
public void deleteByAclId(Long aclId) throws IOException
{
isIdIndexCache.clear();
deleteById(FIELD_ACLID, aclId);
}
@Override
public void deleteByNodeId(Long nodeId) throws IOException
{
deleteById(FIELD_DBID, nodeId);
}
@Override
public void deleteByTransactionId(Long transactionId) throws IOException
{
@@ -925,7 +956,7 @@ public class SolrInformationServer implements InformationServer
{
return this.dataModel.getAlfrescoModels();
}
@SuppressWarnings({ "unchecked", "rawtypes" })
@Override
public Iterable<Entry<String, Object>> getCoreStats() throws IOException
@@ -1049,7 +1080,7 @@ public class SolrInformationServer implements InformationServer
return coreSummary;
}
@Override
public DictionaryComponent getDictionaryService(String alternativeDictionary)
{
@@ -1373,7 +1404,10 @@ public class SolrInformationServer implements InformationServer
public void dirtyTransaction(long txnId)
{
this.cleanContentCache.remove(txnId);
this.cleanCascadeCache.remove(txnId);
if (cascadeTrackingEnabled())
{
this.cleanCascadeCache.remove(txnId);
}
}
@Override
@@ -1462,9 +1496,9 @@ public class SolrInformationServer implements InformationServer
long start = System.nanoTime();
if ((node.getStatus() == SolrApiNodeStatus.DELETED)
|| (node.getStatus() == SolrApiNodeStatus.NON_SHARD_DELETED)
|| (node.getStatus() == SolrApiNodeStatus.NON_SHARD_UPDATED)
|| (node.getStatus() == SolrApiNodeStatus.UNKNOWN))
|| (node.getStatus() == SolrApiNodeStatus.UNKNOWN)
|| cascadeTrackingEnabled() && ((node.getStatus() == SolrApiNodeStatus.NON_SHARD_DELETED)
|| (node.getStatus() == SolrApiNodeStatus.NON_SHARD_UPDATED)))
{
// fix up any secondary paths
NodeMetaDataParameters nmdp = new NodeMetaDataParameters();
@@ -1472,8 +1506,8 @@ public class SolrInformationServer implements InformationServer
nmdp.setToNodeId(node.getId());
List<NodeMetaData> nodeMetaDatas;
if ((node.getStatus() == SolrApiNodeStatus.DELETED)
|| (node.getStatus() == SolrApiNodeStatus.NON_SHARD_DELETED)
|| (node.getStatus() == SolrApiNodeStatus.NON_SHARD_UPDATED))
|| cascadeTrackingEnabled() && ((node.getStatus() == SolrApiNodeStatus.NON_SHARD_DELETED)
|| (node.getStatus() == SolrApiNodeStatus.NON_SHARD_UPDATED)))
{
// Fake the empty node metadata for this parent deleted node
NodeMetaData nodeMetaData = createDeletedNodeMetaData(node);
@@ -1483,7 +1517,7 @@ public class SolrInformationServer implements InformationServer
{
nodeMetaDatas = repositoryClient.getNodesMetaData(nmdp, Integer.MAX_VALUE);
}
NodeMetaData nodeMetaData;
if (!nodeMetaDatas.isEmpty())
{
@@ -1501,7 +1535,7 @@ public class SolrInformationServer implements InformationServer
finally
{
unlock(nodeMetaData.getId());
}
}
}
}
// else, the node has moved on to a later transaction, and it will be indexed later
@@ -1510,10 +1544,9 @@ public class SolrInformationServer implements InformationServer
deleteNode(processor, request, node);
}
if ((node.getStatus() == SolrApiNodeStatus.UPDATED)
|| (node.getStatus() == SolrApiNodeStatus.UNKNOWN)
|| (node.getStatus() == SolrApiNodeStatus.NON_SHARD_UPDATED))
if (node.getStatus() == SolrApiNodeStatus.UPDATED
|| node.getStatus() == SolrApiNodeStatus.UNKNOWN
|| (cascadeTrackingEnabled() && node.getStatus() == SolrApiNodeStatus.NON_SHARD_UPDATED))
{
long nodeId = node.getId();
@@ -1547,7 +1580,7 @@ public class SolrInformationServer implements InformationServer
StringPropertyValue pValue = (StringPropertyValue) properties.get(ContentModel.PROP_IS_INDEXED);
if (pValue != null)
{
boolean isIndexed = Boolean.valueOf(pValue.getValue());
boolean isIndexed = Boolean.parseBoolean(pValue.getValue());
if (!isIndexed)
{
LOGGER.debug("Clearing unindexed");
@@ -1787,7 +1820,7 @@ public class SolrInformationServer implements InformationServer
@Override
public void indexNodes(List<Node> nodes, boolean overwrite, boolean cascade) throws IOException, JSONException
public void indexNodes(List<Node> nodes, boolean overwrite) throws IOException, JSONException
{
UpdateRequestProcessor processor = null;
try (SolrQueryRequest request = newSolrQueryRequest())
@@ -1798,8 +1831,13 @@ public class SolrInformationServer implements InformationServer
EnumMap<SolrApiNodeStatus, List<Long>> nodeStatusToNodeIds = new EnumMap<>(SolrApiNodeStatus.class);
categorizeNodes(nodes, nodeIdsToNodes, nodeStatusToNodeIds);
List<Long> deletedNodeIds = mapNullToEmptyList(nodeStatusToNodeIds.get(SolrApiNodeStatus.DELETED));
List<Long> shardDeletedNodeIds = mapNullToEmptyList(nodeStatusToNodeIds.get(SolrApiNodeStatus.NON_SHARD_DELETED));
List<Long> shardUpdatedNodeIds = mapNullToEmptyList(nodeStatusToNodeIds.get(SolrApiNodeStatus.NON_SHARD_UPDATED));
List<Long> shardDeletedNodeIds = Collections.emptyList();
List<Long> shardUpdatedNodeIds = Collections.emptyList();
if (cascadeTrackingEnabled())
{
shardDeletedNodeIds = mapNullToEmptyList(nodeStatusToNodeIds.get(SolrApiNodeStatus.NON_SHARD_DELETED));
shardUpdatedNodeIds = mapNullToEmptyList(nodeStatusToNodeIds.get(SolrApiNodeStatus.NON_SHARD_UPDATED));
}
List<Long> unknownNodeIds = mapNullToEmptyList(nodeStatusToNodeIds.get(SolrApiNodeStatus.UNKNOWN));
List<Long> updatedNodeIds = mapNullToEmptyList(nodeStatusToNodeIds.get(SolrApiNodeStatus.UPDATED));
@@ -1820,6 +1858,12 @@ public class SolrInformationServer implements InformationServer
{
NodeMetaDataParameters nmdp = new NodeMetaDataParameters();
nmdp.setNodeIds(unknownNodeIds);
// When deleting nodes, no additional information is required
nmdp.setIncludeChildIds(false);
nmdp.setIncludeChildAssociations(false);
nmdp.setIncludeAspects(false);
nmdp.setIncludePaths(false);
nmdp.setIncludeParentAssociations(false);
nodeMetaDatas.addAll(repositoryClient.getNodesMetaData(nmdp, Integer.MAX_VALUE));
}
@@ -1861,10 +1905,12 @@ public class SolrInformationServer implements InformationServer
nodeIds.addAll(unknownNodeIds);
nodeIds.addAll(shardUpdatedNodeIds);
nmdp.setNodeIds(nodeIds);
nmdp.setIncludeChildIds(false);
nmdp.setIncludeChildAssociations(false);
// Fetches bulk metadata
List<NodeMetaData> nodeMetaDatas = repositoryClient.getNodesMetaData(nmdp, Integer.MAX_VALUE);
NEXT_NODE:
for (NodeMetaData nodeMetaData : nodeMetaDatas)
{
@@ -1883,7 +1929,7 @@ public class SolrInformationServer implements InformationServer
continue;
}
if (nodeIdsToNodes.get(nodeMetaData.getId()).getStatus() == SolrApiNodeStatus.NON_SHARD_UPDATED)
if (cascadeTrackingEnabled() && nodeIdsToNodes.get(nodeMetaData.getId()).getStatus() == SolrApiNodeStatus.NON_SHARD_UPDATED)
{
if (nodeMetaData.getProperties().get(ContentModel.PROP_CASCADE_TX) != null)
{
@@ -1902,7 +1948,7 @@ public class SolrInformationServer implements InformationServer
StringPropertyValue pValue = (StringPropertyValue) properties.get(ContentModel.PROP_IS_INDEXED);
if (pValue != null)
{
boolean isIndexed = Boolean.valueOf(pValue.getValue());
boolean isIndexed = Boolean.parseBoolean(pValue.getValue());
if (!isIndexed)
{
LOGGER.debug("Clearing unindexed");
@@ -1958,7 +2004,7 @@ public class SolrInformationServer implements InformationServer
}
}
private void addToNewDocAndCache(NodeMetaData nodeMetaData, SolrInputDocument newDoc) throws IOException
private void addToNewDocAndCache(NodeMetaData nodeMetaData, SolrInputDocument newDoc)
{
addFieldsToDoc(nodeMetaData, newDoc);
SolrInputDocument cachedDoc = null;
@@ -1984,7 +2030,7 @@ public class SolrInformationServer implements InformationServer
{
StringPropertyValue latProp = ((StringPropertyValue)nodeMetaData.getProperties().get(ContentModel.PROP_LATITUDE));
StringPropertyValue lonProp = ((StringPropertyValue)nodeMetaData.getProperties().get(ContentModel.PROP_LONGITUDE));
if((latProp != null) && (lonProp != null))
{
String lat = latProp.getValue();
@@ -2022,14 +2068,14 @@ public class SolrInformationServer implements InformationServer
}
doc.addField(FIELD_ISNODE, "T");
// FIELD_FTSSTATUS is set when adding content properties to indicate whether or not the cache is clean.
doc.addField(FIELD_TENANT, AlfrescoSolrDataModel.getTenantId(nodeMetaData.getTenantDomain()));
updatePathRelatedFields(nodeMetaData, doc);
updateNamePathRelatedFields(nodeMetaData, doc);
updateAncestorRelatedFields(nodeMetaData, doc);
doc.addField(FIELD_PARENT_ASSOC_CRC, nodeMetaData.getParentAssocsCrc());
if (nodeMetaData.getOwner() != null)
{
doc.addField(FIELD_OWNER, nodeMetaData.getOwner());
@@ -2115,7 +2161,7 @@ public class SolrInformationServer implements InformationServer
}
}
static void addPropertiesToDoc(Map<QName, PropertyValue> properties, boolean isContentIndexedForNode,
static void addPropertiesToDoc(Map<QName, PropertyValue> properties, boolean isContentIndexedForNode,
SolrInputDocument newDoc, SolrInputDocument cachedDoc, boolean transformContentFlag)
{
for (Entry<QName, PropertyValue> property : properties.entrySet())
@@ -2123,7 +2169,7 @@ public class SolrInformationServer implements InformationServer
QName propertyQName = property.getKey();
newDoc.addField(FIELD_PROPERTIES, propertyQName.toString());
newDoc.addField(FIELD_PROPERTIES, propertyQName.getPrefixString());
PropertyValue value = property.getValue();
if(value != null)
{
@@ -2200,12 +2246,20 @@ public class SolrInformationServer implements InformationServer
private void deleteErrorNode(UpdateRequestProcessor processor, SolrQueryRequest request, Node node) throws IOException
{
String errorDocId = PREFIX_ERROR + node.getId();
DeleteUpdateCommand delErrorDocCmd = new DeleteUpdateCommand(request);
delErrorDocCmd.setId(errorDocId);
processor.processDelete(delErrorDocCmd);
}
String errorDocId = PREFIX_ERROR + node.getId();
// Try finding the node before performing removal operation
DocSet docSet = request.getSearcher().getDocSet(new TermQuery(new Term(FIELD_SOLR4_ID, errorDocId)));
if (docSet.size() > 0)
{
DeleteUpdateCommand delErrorDocCmd = new DeleteUpdateCommand(request);
delErrorDocCmd.setId(errorDocId);
processor.processDelete(delErrorDocCmd);
}
}
private void deleteNode(UpdateRequestProcessor processor, SolrQueryRequest request, Node node) throws IOException
{
@@ -2218,9 +2272,17 @@ public class SolrInformationServer implements InformationServer
private void deleteNode(UpdateRequestProcessor processor, SolrQueryRequest request, long dbid) throws IOException
{
DeleteUpdateCommand delDocCmd = new DeleteUpdateCommand(request);
delDocCmd.setQuery(FIELD_DBID + ":" + dbid);
processor.processDelete(delDocCmd);
// Try finding the node before performing removal operation
DocSet docSet = request.getSearcher().getDocSet(LongPoint.newExactQuery(FIELD_DBID, dbid));
if (docSet.size() > 0)
{
DeleteUpdateCommand delDocCmd = new DeleteUpdateCommand(request);
delDocCmd.setQuery(FIELD_DBID + ":" + dbid);
processor.processDelete(delDocCmd);
}
}
private boolean isContentIndexedForNode(Map<QName, PropertyValue> properties)
@@ -2232,7 +2294,7 @@ public class SolrInformationServer implements InformationServer
.get(ContentModel.PROP_IS_CONTENT_INDEXED);
if (pValue != null)
{
boolean isIndexed = Boolean.valueOf(pValue.getValue());
boolean isIndexed = Boolean.parseBoolean(pValue.getValue());
if (!isIndexed)
{
isContentIndexed = false;
@@ -2282,7 +2344,7 @@ public class SolrInformationServer implements InformationServer
return fieldName;
}
private void addContentPropertyMetadata(SolrInputDocument doc, QName propertyQName,
private void addContentPropertyMetadata(SolrInputDocument doc, QName propertyQName,
AlfrescoSolrDataModel.ContentFieldType type, GetTextContentResponse textContentResponse)
{
IndexedField indexedField = AlfrescoSolrDataModel.getInstance().getIndexedFieldForContentPropertyMetadata(
@@ -2308,7 +2370,7 @@ public class SolrInformationServer implements InformationServer
}
}
private static void addContentPropertyMetadata(SolrInputDocument doc, QName propertyQName,
private static void addContentPropertyMetadata(SolrInputDocument doc, QName propertyQName,
ContentPropertyValue contentPropertyValue, AlfrescoSolrDataModel.ContentFieldType type)
{
IndexedField indexedField = AlfrescoSolrDataModel.getInstance().getIndexedFieldForContentPropertyMetadata(
@@ -2338,8 +2400,8 @@ public class SolrInformationServer implements InformationServer
}
}
}
private static void addContentPropertyToDocUsingCache(SolrInputDocument newDoc, SolrInputDocument cachedDoc,
private static void addContentPropertyToDocUsingCache(SolrInputDocument newDoc, SolrInputDocument cachedDoc,
QName propertyQName, ContentPropertyValue contentPropertyValue, boolean transformContentFlag)
{
addContentPropertyMetadata(newDoc, propertyQName, contentPropertyValue, AlfrescoSolrDataModel.ContentFieldType.DOCID);
@@ -2347,14 +2409,14 @@ public class SolrInformationServer implements InformationServer
addContentPropertyMetadata(newDoc, propertyQName, contentPropertyValue, AlfrescoSolrDataModel.ContentFieldType.LOCALE);
addContentPropertyMetadata(newDoc, propertyQName, contentPropertyValue, AlfrescoSolrDataModel.ContentFieldType.MIMETYPE);
addContentPropertyMetadata(newDoc, propertyQName, contentPropertyValue, AlfrescoSolrDataModel.ContentFieldType.ENCODING);
if (!transformContentFlag)
{
// Marks it as Clean so we do not get the actual content
markFTSStatus(newDoc, FTSStatus.Clean);
return;
}
if (cachedDoc != null)
{
ofNullable(cachedDoc.getField("MINHASH"))
@@ -2372,32 +2434,38 @@ public class SolrInformationServer implements InformationServer
addFieldIfNotSet(newDoc, field);
}
String transformationStatusFieldName = getSolrFieldNameForContentPropertyMetadata(propertyQName,
String transformationStatusFieldName = getSolrFieldNameForContentPropertyMetadata(propertyQName,
AlfrescoSolrDataModel.ContentFieldType.TRANSFORMATION_STATUS);
newDoc.addField(transformationStatusFieldName, cachedDoc.getFieldValue(transformationStatusFieldName));
if (transformationStatusFieldName != null){
newDoc.addField(transformationStatusFieldName, cachedDoc.getFieldValue(transformationStatusFieldName));
}
String transformationExceptionFieldName = getSolrFieldNameForContentPropertyMetadata(propertyQName,
AlfrescoSolrDataModel.ContentFieldType.TRANSFORMATION_EXCEPTION);
newDoc.addField(transformationExceptionFieldName, cachedDoc.getFieldValue(transformationExceptionFieldName));
String transformationTimeFieldName = getSolrFieldNameForContentPropertyMetadata(propertyQName,
if (transformationExceptionFieldName != null){
newDoc.addField(transformationExceptionFieldName, cachedDoc.getFieldValue(transformationExceptionFieldName));
}
String transformationTimeFieldName = getSolrFieldNameForContentPropertyMetadata(propertyQName,
AlfrescoSolrDataModel.ContentFieldType.TRANSFORMATION_TIME);
newDoc.addField(transformationTimeFieldName, cachedDoc.getFieldValue(transformationTimeFieldName));
if (transformationTimeFieldName != null){
newDoc.addField(transformationTimeFieldName, cachedDoc.getFieldValue(transformationTimeFieldName));
}
// Gets the new content docid and compares to that of the cachedDoc to mark the content as clean/dirty
String fldName = getSolrFieldNameForContentPropertyMetadata(propertyQName,
AlfrescoSolrDataModel.ContentFieldType.DOCID);
if(newDoc.getFieldValue(FIELD_FTSSTATUS) == null)
{
newDoc.addField(FIELD_FTSSTATUS, cachedDoc.getFieldValue(FIELD_FTSSTATUS));
}
if(cachedDoc.getFieldValue(fldName) != null)
{
long cachedDocContentDocid = Long.valueOf(String.valueOf(cachedDoc.getFieldValue(fldName)));
long cachedDocContentDocid = Long.parseLong(String.valueOf(cachedDoc.getFieldValue(fldName)));
long currentContentDocid = contentPropertyValue.getId();
// If we have used out of date content we mark it as dirty
// Otherwise we leave it alone - it could already be marked as dirty/New and require an update
if (cachedDocContentDocid != currentContentDocid)
{
// The cached content is out of date
@@ -2409,7 +2477,7 @@ public class SolrInformationServer implements InformationServer
markFTSStatus(newDoc, FTSStatus.Dirty);
}
}
else
else
{
// There is not a SolrInputDocument in the solrContentStore, so no content is added now to the new solr doc
markFTSStatus(newDoc, FTSStatus.New);
@@ -2448,7 +2516,7 @@ public class SolrInformationServer implements InformationServer
doc.removeField(FIELD_FTSSTATUS);
doc.addField(FIELD_FTSSTATUS, status.toString());
}
private void addContentToDoc(SolrInputDocument doc, long dbId) throws AuthenticationException, IOException
{
Collection<String> fieldNames = doc.deepCopy().getFieldNames();
@@ -2464,15 +2532,15 @@ public class SolrInformationServer implements InformationServer
// Could update multi content but it is broken ....
}
}
private void addContentPropertyToDocUsingAlfrescoRepository(SolrInputDocument doc,
QName propertyQName, long dbId, String locale) throws AuthenticationException, IOException
{
long start = System.nanoTime();
// Expensive call to be done with ContentTracker
GetTextContentResponse response = repositoryClient.getTextContent(dbId, propertyQName, null);
addContentPropertyMetadata(doc, propertyQName, AlfrescoSolrDataModel.ContentFieldType.TRANSFORMATION_STATUS,
response);
addContentPropertyMetadata(doc, propertyQName, AlfrescoSolrDataModel.ContentFieldType.TRANSFORMATION_EXCEPTION,
@@ -2481,6 +2549,10 @@ public class SolrInformationServer implements InformationServer
response);
InputStream ris = response.getContent();
if (Objects.equals(response.getContentEncoding(), "gzip"))
{
ris = new GZIPInputStream(ris);
}
String textContent = "";
try
{
@@ -2521,7 +2593,7 @@ public class SolrInformationServer implements InformationServer
long end = System.nanoTime();
this.getTrackerStats().addDocTransformationTime(end - start);
StringBuilder builder = new StringBuilder(textContent.length() + 16);
builder.append("\u0000").append(locale).append("\u0000");
builder.append(textContent);
@@ -2543,7 +2615,7 @@ public class SolrInformationServer implements InformationServer
}
private static void addMLTextPropertyToDoc(SolrInputDocument doc, FieldInstance field, MLTextPropertyValue mlTextPropertyValue)
{
{
if(field.isLocalised())
{
StringBuilder sort = new StringBuilder(128);
@@ -2551,20 +2623,20 @@ public class SolrInformationServer implements InformationServer
{
final String propValue = mlTextPropertyValue.getValue(locale);
LOGGER.debug("ML {} in {} of {}", field.getField(), locale, propValue);
if((locale == null) || (propValue == null))
{
continue;
}
}
StringBuilder builder = new StringBuilder(propValue.length() + 16);
builder.append("\u0000").append(locale.toString()).append("\u0000").append(propValue);
if(!field.isSort())
{
doc.addField(field.getField(), builder.toString());
}
if (sort.length() > 0)
{
sort.append("\u0000");
@@ -2605,7 +2677,10 @@ public class SolrInformationServer implements InformationServer
input.addField(FIELD_INTXID, txn.getId());
input.addField(FIELD_TXCOMMITTIME, txn.getCommitTimeMs());
input.addField(FIELD_DOC_TYPE, DOC_TYPE_TX);
input.addField(FIELD_CASCADE_FLAG, 0);
if (cascadeTrackingEnabled())
{
input.addField(FIELD_CASCADE_FLAG, 0);
}
cmd.solrDoc = input;
processor.processAdd(cmd);
}
@@ -2646,8 +2721,11 @@ public class SolrInformationServer implements InformationServer
input.addField(FIELD_S_TXID, info.getId());
input.addField(FIELD_S_TXCOMMITTIME, info.getCommitTimeMs());
//Set the cascade flag to 1. This means cascading updates have not been done yet.
input.addField(FIELD_CASCADE_FLAG, 1);
if (cascadeTrackingEnabled())
{
//Set the cascade flag to 1. This means cascading updates have not been done yet.
input.addField(FIELD_CASCADE_FLAG, 1);
}
cmd.solrDoc = input;
processor.processAdd(cmd);
@@ -2874,7 +2952,7 @@ public class SolrInformationServer implements InformationServer
{
activeTrackerThreadsLock.writeLock().unlock();
}
}
@Override
@@ -2901,7 +2979,7 @@ public class SolrInformationServer implements InformationServer
SolrIndexSearcher solrIndexSearcher = refCounted.get();
NumericDocValues dbidDocValues = solrIndexSearcher.getSlowAtomicReader().getNumericDocValues(QueryConstants.FIELD_DBID);
List<Node> batch = new ArrayList<>(200);
DocList docList = cloud.getDocList(nativeRequestHandler, request, query.startsWith("{") ? query : "{!afts}"+query);
for (DocIterator it = docList.iterator(); it.hasNext(); /**/)
@@ -2916,16 +2994,16 @@ public class SolrInformationServer implements InformationServer
node.setTxnId(Long.MAX_VALUE);
batch.add(node);
if(batch.size() >= 200)
{
indexNodes(batch, true, true);
indexNodes(batch, true);
batch.clear();
}
}
if(batch.size() > 0)
{
indexNodes(batch, true, true);
indexNodes(batch, true);
batch.clear();
}
}
@@ -3414,7 +3492,7 @@ public class SolrInformationServer implements InformationServer
SolrQueryRequest request, UpdateRequestProcessor processor, LinkedHashSet<Long> stack)
throws AuthenticationException, IOException, JSONException
{
// skipDescendantDocsForSpecificAspects is initialised on a synchronised method, so access must be also synchronised
synchronized (this)
{
@@ -3584,7 +3662,7 @@ public class SolrInformationServer implements InformationServer
field, 1); // Min count of 1 ensures that the id returned is in the index
for (Map.Entry<String, Integer> idCount : idCounts)
{
long idInIndex = Long.valueOf(idCount.getKey());
long idInIndex = Long.parseLong(idCount.getKey());
// Only looks at facet values that fit the query
if (batchStartId <= idInIndex && idInIndex <= batchEndId)
search-services/alfresco-search/src/main/java/org/alfresco/solr/content/SolrContentStore.java
+17 -3
View File
@@ -43,12 +43,14 @@ import java.io.OutputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.StandardCopyOption;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.function.Predicate;
import java.util.stream.Stream;
import java.util.zip.GZIPInputStream;
import java.util.zip.GZIPOutputStream;
@@ -236,7 +238,8 @@ public final class SolrContentStore implements Closeable, AccessMode
{
try
{
return Files.lines(Paths.get(root, ".version"))
return Files.readAllLines(Paths.get(root, ".version"))
.stream()
.map(Long::parseLong)
.findFirst()
.orElse(NO_VERSION_AVAILABLE);
@@ -250,18 +253,29 @@ public final class SolrContentStore implements Closeable, AccessMode
@Override
public void setLastCommittedVersion(long version)
{
File tmpFile = new File(root, ".version-" + new SimpleDateFormat(SnapShooter.DATE_FMT, Locale.ROOT).format(new Date()));
try
{
File tmpFile = new File(root, ".version-" + new SimpleDateFormat(SnapShooter.DATE_FMT, Locale.ROOT).format(new Date()));
FileWriter wr = new FileWriter(tmpFile);
wr.write(Long.toString(version));
wr.close();
tmpFile.renameTo(new File(root, ".version"));
// file.renameTo(..) does not work on windows. Use Files.move instead.
Files.move(tmpFile.toPath(), new File(root, ".version").toPath(), StandardCopyOption.ATOMIC_MOVE);
}
catch (IOException exception)
{
logger.error("Unable to persist the last committed content store version {}. See the stacktrace below for furtger details.", version, exception);
try
{
Files.delete(tmpFile.toPath());
}
catch (IOException e)
{
logger.error("Unable to delete tmp contentstore version file {}.", version);
}
}
}
search-services/alfresco-search/src/main/java/org/alfresco/solr/handler/AlfrescoIndexFetcher.java
+93 -74
View File
@@ -34,40 +34,10 @@
*/
package org.alfresco.solr.handler;
import static java.util.List.of;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.ALIAS;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.CHECKSUM;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.CMD_CONTENT_STORE_FILES;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.CMD_DETAILS;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.CMD_GET_FILE;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.CMD_GET_FILE_LIST;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.CMD_INDEX_VERSION;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.COMMAND;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.COMPRESSION;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.CONF_FILES;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.CONF_FILE_SHORT;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.CONTENT_STORE_FILES;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.CONTENT_STORE_FILE_LIST;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.CONTENT_STORE_VERSION;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.EXTERNAL;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.FILE;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.FILE_STREAM;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.FileInfo;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.GENERATION;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.INTERNAL;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.MASTER_URL;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.NO_INDEX_REPLICATION_REQUIRED;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.OFFSET;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.SIZE;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.TLOG_FILE;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.TLOG_FILES;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.getCheckSum;
import static org.apache.solr.common.params.CommonParams.JAVABIN;
import static org.apache.solr.common.params.CommonParams.NAME;
import com.google.common.base.Strings;
import com.google.common.collect.Lists;
import org.alfresco.solr.content.SolrContentStore;
import org.apache.commons.io.FilenameUtils;
import org.apache.http.client.HttpClient;
import org.apache.lucene.codecs.CodecUtil;
import org.apache.lucene.index.IndexCommit;
@@ -152,6 +122,37 @@ import java.util.zip.Adler32;
import java.util.zip.Checksum;
import java.util.zip.InflaterInputStream;
import static java.util.List.of;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.ALIAS;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.CHECKSUM;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.CMD_CONTENT_STORE_FILES;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.CMD_DETAILS;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.CMD_GET_FILE;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.CMD_GET_FILE_LIST;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.CMD_INDEX_VERSION;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.COMMAND;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.COMPRESSION;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.CONF_FILES;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.CONF_FILE_SHORT;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.CONTENT_STORE_FILES;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.CONTENT_STORE_FILE_LIST;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.CONTENT_STORE_VERSION;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.EXTERNAL;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.FILE;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.FILE_STREAM;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.FileInfo;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.GENERATION;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.INTERNAL;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.MASTER_URL;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.NO_INDEX_REPLICATION_REQUIRED;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.OFFSET;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.SIZE;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.TLOG_FILE;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.TLOG_FILES;
import static org.alfresco.solr.handler.AlfrescoReplicationHandler.getCheckSum;
import static org.apache.solr.common.params.CommonParams.JAVABIN;
import static org.apache.solr.common.params.CommonParams.NAME;
/**
* <p> Provides functionality of downloading changed index files as well as config files and a timer for scheduling fetches from the
* master. </p>
@@ -188,6 +189,13 @@ class AlfrescoIndexFetcher
private final Map<String, FileInfo> confFileInfoCache = new HashMap<>();
private volatile Date replicationStartTimeStamp;
private RTimer replicationTimer;
/**
* The map<String, object> contains the following fields:
* NAME : String -> file name(with path for contentstore files)
* SIZE : long -> file size
* CHECKSUM : long -> checksum
*/
private volatile List<Map<String, Object>> filesToDownload;
private volatile List<Map<String, Object>> confFilesToDownload;
private volatile List<Map<String, Object>> tlogFilesToDownload;
@@ -197,6 +205,7 @@ class AlfrescoIndexFetcher
private volatile List<Map<String, Object>> confFilesDownloaded;
private volatile List<Map<String, Object>> tlogFilesDownloaded;
private volatile List<Map<String, Object>> contentStoreFilesDownloaded;
private volatile Map<String, Object> currentFile;
private volatile DirectoryFileFetcher dirFileFetcher;
private volatile LocalFsFileFetcher localFileFetcher;
@@ -467,6 +476,8 @@ class AlfrescoIndexFetcher
Map<String, List<Map<String, Object>>> contentStoreMap = (Map<String, List<Map<String, Object>>>) response
.get(CONTENT_STORE_FILES);
fullContentStoreReplication = false;
if (contentStoreMap != null)
{
contentStoreFilesToDownload = Collections.synchronizedList(contentStoreMap.get(SolrContentStore.ADDS));
@@ -776,30 +787,37 @@ class AlfrescoIndexFetcher
}
}
if (contentStoreReplicationNeeded)
{
try {
if (contentStoreFilesToDownload != null)
if (contentStoreReplicationNeeded)
{
bytesDownloaded += downloadContentStoreFiles(contentStore.getRootLocation());
if (contentStoreFilesToDownload != null)
{
bytesDownloaded += downloadContentStoreFiles(contentStore.getRootLocation());
}
if (contentStoreFilesToDelete != null)
{
deleteContentStoreFiles(contentStore.getRootLocation(), contentStoreFilesToDelete);
}
if (fullContentStoreReplication)
{
cleanUpContentStore(contentStore.getRootLocation());
}
contentStore.setLastCommittedVersion(masterContentStoreVersion);
LOG.info("content store has been updated to version: {}", masterContentStoreVersion);
}
if (contentStoreFilesToDelete != null)
{
deleteContentStoreFiles(contentStore.getRootLocation(), contentStoreFilesToDelete);
}
if (fullContentStoreReplication)
{
cleanUpContentStore(contentStore.getRootLocation());
}
contentStore.setLastCommittedVersion(masterContentStoreVersion);
} catch (Exception e) {
LOG.error("impossible to complete content store replication {}", e);
}
final long timeTakenSeconds = getReplicationTimeElapsed();
final Long bytesDownloadedPerSecond = (timeTakenSeconds != 0 ? bytesDownloaded / timeTakenSeconds :
null);
null);
LOG.info("Total time taken for download (fullCopy={},bytesDownloaded={}) : {} secs ({} bytes/sec)",
isFullCopyNeeded, bytesDownloaded, timeTakenSeconds, bytesDownloadedPerSecond);
@@ -1683,37 +1701,30 @@ class AlfrescoIndexFetcher
* @param contentStorePath
* @throws IOException
*/
private void copyTmpContentStoreToContentStore(File tmpContentStoreDir, String contentStorePath) throws IOException
private void copyTmpContentStoreToContentStore(File tmpContentStoreDir, String contentStorePath) throws Exception
{
String tmpContentStorePath = tmpContentStoreDir.getPath();
try
{
Files.walk(tmpContentStoreDir.toPath()).forEach(p -> {
File tmpFile = new File(p.toUri());
if (!tmpFile.isDirectory())
Files.walk(tmpContentStoreDir.toPath()).forEach(p -> {
File tmpFile = new File(p.toUri());
if (!tmpFile.isDirectory())
{
File csFile = new File(p.toString().replace(tmpContentStorePath, contentStorePath));
try
{
File csFile = new File(p.toString().replaceFirst(tmpContentStorePath, contentStorePath));
try
{
Files.createDirectories(Paths.get(csFile.getParent()));
tmpFile.renameTo(csFile);
}
catch (IOException e)
{
LOG.error("impossible to copy {}", csFile.toString());
}
Files.createDirectories(Paths.get(csFile.getParent()));
Files.move(tmpFile.toPath(), csFile.toPath(), StandardCopyOption.ATOMIC_MOVE);
}
});
}
catch (IOException e)
{
LOG.error("impossible tmp content store");
throw e;
}
catch (IOException e)
{
throw new RuntimeException(e);
}
}
});
}
/**
* Deletes the files in filesToDelete list from contentStore
* @param contentStorePath
@@ -1733,16 +1744,23 @@ class AlfrescoIndexFetcher
* Deletes from contentstore all the files that has not been updated.
* @param contentStorePath
*/
private void cleanUpContentStore(String contentStorePath)
private void cleanUpContentStore(String contentStorePath) throws Exception
{
AtomicInteger fileDeleted = new AtomicInteger();
Set<String> fileNames = contentStoreFilesToDownload.stream().map(e -> (String) e.get(NAME))
// This is the set of the ONLY files that should be in contentStore.
// This set is computed from the information got from master. After a full replication, only the files
// that have been downloaded from master (contentStoreFilesToDownload) should be in contentStore.
// The file paths are translated in the current OS path notation.
Set<String> contentStoreFiles = contentStoreFilesToDownload.stream()
.map(e -> (String) e.get(NAME))
.map(FilenameUtils::separatorsToSystem)
.collect(Collectors.toSet());
try
{
Files.walk(Paths.get(contentStorePath)).forEach(p -> {
File f = new File(p.toUri());
if (!f.isDirectory() && !fileNames.contains(p.toString().replaceFirst(contentStorePath, "")))
if (!f.isDirectory() && !contentStoreFiles.contains(p.toString().replace(contentStorePath, "")))
{
try
{
@@ -1756,9 +1774,10 @@ class AlfrescoIndexFetcher
}
});
}
catch (IOException e)
catch (Exception e)
{
LOG.error("Impossible to delete unnecessary files. Content store may contains unused contents");
throw(e);
}
LOG.info("deleted {} unnecessary files from content store", fileDeleted);
search-services/alfresco-search/src/main/java/org/alfresco/solr/handler/AlfrescoReplicationHandler.java
+3
View File
@@ -2122,6 +2122,7 @@ public class AlfrescoReplicationHandler extends RequestHandlerBase implements So
e.printStackTrace();
}
}
}
}
catch (Exception e)
@@ -2169,6 +2170,7 @@ public class AlfrescoReplicationHandler extends RequestHandlerBase implements So
if (bytesRead <= 0)
{
writeNothingAndFlush();
inputStream.close();
break;
}
@@ -2185,6 +2187,7 @@ public class AlfrescoReplicationHandler extends RequestHandlerBase implements So
fos.flush();
}
}
}
}
}
search-services/alfresco-search/src/main/java/org/alfresco/solr/lifecycle/SolrCoreLoadListener.java
+26 -16
View File
@@ -21,6 +21,15 @@ package org.alfresco.solr.lifecycle;
import static java.util.Arrays.asList;
import static java.util.Optional.ofNullable;
import static org.alfresco.solr.SolrInformationServer.CASCADE_TRACKER_ENABLED;
import java.util.ArrayList;
import java.util.Collection;
import java.util.List;
import java.util.Properties;
import java.util.function.Function;
import java.util.function.Predicate;
import org.alfresco.opencmis.dictionary.CMISStrictDictionaryService;
import org.alfresco.solr.AlfrescoCoreAdminHandler;
import org.alfresco.solr.AlfrescoSolrDataModel;
@@ -54,13 +63,6 @@ import org.apache.solr.search.SolrIndexSearcher;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.ArrayList;
import java.util.Collection;
import java.util.List;
import java.util.Properties;
import java.util.function.Function;
import java.util.function.Predicate;
/**
* Listeners for *FIRST SEARCHER* events in order to prepare and register the SolrContentStore and the Tracking Subsystem.
*
@@ -251,25 +253,33 @@ public class SolrCoreLoadListener extends AbstractSolrEventListener
MetadataTracker metadataTracker =
registerAndSchedule(
new MetadataTracker(true, props, repositoryClient, core.getName(), srv),
new MetadataTracker(true, props, repositoryClient, core.getName(), srv, true),
core,
props,
trackerRegistry,
scheduler);
CascadeTracker cascadeTracker =
registerAndSchedule(
new CascadeTracker(props, repositoryClient, core.getName(), srv),
core,
props,
trackerRegistry,
scheduler);
List<Tracker> trackers = new ArrayList<>();
String cascadeTrackerEnabledProp = ofNullable((String) props.get(CASCADE_TRACKER_ENABLED)).orElse("true");
if (Boolean.valueOf(cascadeTrackerEnabledProp))
{
CascadeTracker cascadeTracker =
registerAndSchedule(
new CascadeTracker(props, repositoryClient, core.getName(), srv),
core,
props,
trackerRegistry,
scheduler);
trackers.add(cascadeTracker);
}
//The CommitTracker will acquire these locks in order
//The ContentTracker will likely have the longest runs so put it first to ensure the MetadataTracker is not paused while
//waiting for the ContentTracker to release it's lock.
//The aclTracker will likely have the shortest runs so put it last.
return asList(cascadeTracker, contentTracker, metadataTracker, aclTracker);
trackers.addAll(asList(contentTracker, metadataTracker, aclTracker));
return trackers;
}
/**
search-services/alfresco-search/src/main/java/org/alfresco/solr/tracker/AbstractTracker.java
+18 -34
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2005-2014 Alfresco Software Limited.
* Copyright (C) 2005-2019 Alfresco Software Limited.
*
* This file is part of Alfresco
*
@@ -18,6 +18,8 @@
*/
package org.alfresco.solr.tracker;
import static java.util.Optional.ofNullable;
import java.lang.invoke.MethodHandles;
import java.net.ConnectException;
import java.net.SocketTimeoutException;
@@ -49,11 +51,10 @@ public abstract class AbstractTracker implements Tracker
protected SOLRAPIClient client;
InformationServer infoSrv;
protected String coreName;
protected StoreRef storeRef;
protected long batchCount;
protected String alfrescoVersion;
protected TrackerStats trackerStats;
protected boolean runPostModelLoadInit = true;
StoreRef storeRef;
long batchCount;
TrackerStats trackerStats;
boolean runPostModelLoadInit = true;
private int maxLiveSearchers;
private volatile boolean shutdown = false;
@@ -63,9 +64,9 @@ public abstract class AbstractTracker implements Tracker
protected volatile TrackerState state;
protected int shardCount;
protected int shardInstance;
protected String shardMethod;
String shardMethod;
protected boolean transformContent;
protected String shardTemplate;
String shardTemplate;
protected volatile boolean rollback;
protected final Type type;
@@ -102,12 +103,8 @@ public abstract class AbstractTracker implements Tracker
transformContent = Boolean.parseBoolean(p.getProperty("alfresco.index.transformContent", "true"));
this.trackerStats = this.infoSrv.getTrackerStats();
alfrescoVersion = p.getProperty("alfresco.version", "5.0.0");
this.type = type;
LOGGER.info("Solr built for Alfresco version: {}", alfrescoVersion);
}
@@ -183,12 +180,9 @@ public abstract class AbstractTracker implements Tracker
if(this.state == null)
{
/*
* Set the global state for the tracker here.
*/
this.state = getTrackerState();
LOGGER.debug("##### Setting tracker global state.");
LOGGER.debug("State set: {}", this.state.toString());
LOGGER.debug("Global Tracker State set to: {}", this.state.toString());
this.state.setRunning(true);
}
else
@@ -237,12 +231,11 @@ public abstract class AbstractTracker implements Tracker
finally
{
infoSrv.unregisterTrackerThread();
if(state != null)
{
//During a rollback state is set to null.
ofNullable(state).ifPresent(tstate -> {
// During a rollback state is set to null.
state.setRunning(false);
state.setCheck(false);
}
});
runLock.release();
}
}
@@ -284,7 +277,7 @@ public abstract class AbstractTracker implements Tracker
/**
* Allows time for the scheduled asynchronous tasks to complete
*/
protected synchronized void waitForAsynchronous()
synchronized void waitForAsynchronous()
{
AbstractWorkerRunnable currentRunnable = this.threadHandler.peekHeadReindexWorker();
while (currentRunnable != null)
@@ -305,12 +298,12 @@ public abstract class AbstractTracker implements Tracker
}
}
public int getMaxLiveSearchers()
int getMaxLiveSearchers()
{
return maxLiveSearchers;
}
protected void checkShutdown()
void checkShutdown()
{
if(shutdown)
{
@@ -345,20 +338,11 @@ public abstract class AbstractTracker implements Tracker
return this.writeLock;
}
public Semaphore getRunLock()
Semaphore getRunLock()
{
return this.runLock;
}
/**
* @return Alfresco version Solr was built for
*/
@Override
public String getAlfrescoVersion()
{
return alfrescoVersion;
}
public Properties getProps()
{
return props;
search-services/alfresco-search/src/main/java/org/alfresco/solr/tracker/CommitTracker.java
+15 -7
View File
@@ -19,7 +19,11 @@
package org.alfresco.solr.tracker;
import static java.util.Optional.empty;
import static java.util.Optional.ofNullable;
import java.util.List;
import java.util.Optional;
import java.util.Properties;
import java.util.concurrent.atomic.AtomicInteger;
@@ -41,7 +45,8 @@ public class CommitTracker extends AbstractTracker
private MetadataTracker metadataTracker;
private AclTracker aclTracker;
private ContentTracker contentTracker;
private CascadeTracker cascadeTracker;
/** The cascade tracker. Note that this may be empty if cascade tracking is disabled. */
private Optional<CascadeTracker> cascadeTracker = empty();
private AtomicInteger rollbackCount = new AtomicInteger(0);
protected final static Logger log = LoggerFactory.getLogger(CommitTracker.class);
@@ -71,7 +76,7 @@ public class CommitTracker extends AbstractTracker
} else if(tracker instanceof ContentTracker) {
this.contentTracker = (ContentTracker)tracker;
} else if(tracker instanceof CascadeTracker) {
this.cascadeTracker = (CascadeTracker)tracker;
this.cascadeTracker = ofNullable((CascadeTracker) tracker);
}
}
@@ -182,8 +187,11 @@ public class CommitTracker extends AbstractTracker
contentTracker.getWriteLock().acquire();
assert(contentTracker.getWriteLock().availablePermits() == 0);
cascadeTracker.getWriteLock().acquire();
assert(cascadeTracker.getWriteLock().availablePermits() == 0);
if (cascadeTracker.isPresent())
{
cascadeTracker.get().getWriteLock().acquire();
assert (cascadeTracker.get().getWriteLock().availablePermits() == 0);
}
infoSrv.rollback();
}
@@ -206,12 +214,12 @@ public class CommitTracker extends AbstractTracker
contentTracker.invalidateState();
//Reset cascadeTracker
cascadeTracker.setRollback(false);
cascadeTracker.invalidateState();
cascadeTracker.ifPresent(c -> c.setRollback(false));
cascadeTracker.ifPresent(c -> invalidateState());
//Release the locks
contentTracker.getWriteLock().release();
cascadeTracker.getWriteLock().release();
cascadeTracker.ifPresent(c -> c.getWriteLock().release());
rollbackCount.incrementAndGet();
}
search-services/alfresco-search/src/main/java/org/alfresco/solr/tracker/MetadataTracker.java
+131 -7
View File
@@ -32,6 +32,7 @@ import org.alfresco.repo.index.shard.ShardState;
import org.alfresco.solr.BoundedDeque;
import org.alfresco.solr.InformationServer;
import org.alfresco.solr.NodeReport;
import org.alfresco.solr.SolrInformationServer;
import org.alfresco.solr.TrackerState;
import org.alfresco.solr.adapters.IOpenBitSet;
import org.alfresco.solr.client.GetNodesParameters;
@@ -40,6 +41,7 @@ import org.alfresco.solr.client.Node.SolrApiNodeStatus;
import org.alfresco.solr.client.SOLRAPIClient;
import org.alfresco.solr.client.Transaction;
import org.alfresco.solr.client.Transactions;
import org.alfresco.util.Pair;
import org.apache.commons.codec.EncoderException;
import org.json.JSONException;
import org.slf4j.Logger;
@@ -63,14 +65,89 @@ public class MetadataTracker extends CoreStatePublisher implements Tracker
private ConcurrentLinkedQueue<Long> nodesToIndex = new ConcurrentLinkedQueue<>();
private ConcurrentLinkedQueue<Long> nodesToPurge = new ConcurrentLinkedQueue<>();
private ConcurrentLinkedQueue<String> queriesToReindex = new ConcurrentLinkedQueue<>();
/**
* Check if nextTxCommitTimeService is available in the repository.
* This service is used to find the next available transaction commit time from a given time,
* so periods of time where no document updating is happening can be skipped while getting
* pending transactions list.
*
* {@link org.alfresco.solr.client.SOLRAPIClient#GET_NEXT_TX_COMMIT_TIME}
*/
private boolean nextTxCommitTimeServiceAvailable = false;
/**
* Check if txInteravlCommitTimeService is available in the repository.
* This service returns the minimum and the maximum commit time for transactions in a node id range,
* so method sharding DB_ID_RANGE can skip transactions not relevant for the DB ID range.
*
* {@link org.alfresco.solr.client.SOLRAPIClient#GET_TX_INTERVAL_COMMIT_TIME}
*/
private boolean txIntervalCommitTimeServiceAvailable = false;
/** Whether the cascade tracking is enabled. */
private boolean cascadeTrackerEnabled = true;
public MetadataTracker(final boolean isMaster, Properties p, SOLRAPIClient client, String coreName,
InformationServer informationServer)
InformationServer informationServer)
{
this(isMaster, p, client, coreName, informationServer, false);
}
/**
* MetadataTracker constructor
*
* @param isMaster is true if SOLR instance is master, false otherwise
* @param p includes SOLR core properties (from environment variables and properties file)
* @param client Alfresco Repository http client
* @param coreName Name of the SOLR Core (alfresco, archive)
* @param informationServer SOLR Information Server
* @param checkRepoServicesAvailability is true if Repo Services availability needs to be checked
*/
public MetadataTracker(final boolean isMaster, Properties p, SOLRAPIClient client, String coreName,
InformationServer informationServer, boolean checkRepoServicesAvailability)
{
super(isMaster, p, client, coreName, informationServer, Tracker.Type.METADATA);
transactionDocsBatchSize = Integer.parseInt(p.getProperty("alfresco.transactionDocsBatchSize", "100"));
nodeBatchSize = Integer.parseInt(p.getProperty("alfresco.nodeBatchSize", "10"));
threadHandler = new ThreadHandler(p, coreName, "MetadataTracker");
cascadeTrackerEnabled = informationServer.cascadeTrackingEnabled();
// In order to apply performance optimizations, checking the availability of Repo Web Scripts is required.
// As these services are available from ACS 6.2
if (checkRepoServicesAvailability)
{
// Try invoking getNextTxCommitTime service
try
{
client.getNextTxCommitTime(coreName, 0l);
nextTxCommitTimeServiceAvailable = true;
}
catch (NoSuchMethodException e)
{
log.warn("nextTxCommitTimeService is not available. Upgrade your ACS Repository version in order to use this feature: {} ", e.getMessage());
}
catch (Exception e)
{
log.error("Checking nextTxCommitTimeService failed.", e);
}
// Try invoking txIntervalCommitTime service
try
{
client.getTxIntervalCommitTime(coreName, 0l, 0l);
txIntervalCommitTimeServiceAvailable = true;
}
catch (NoSuchMethodException e)
{
log.warn("txIntervalCommitTimeServiceAvailable is not available. If you are using DB_ID_RANGE shard method, "
+ "upgrade your ACS Repository version in order to use this feature: {} ", e.getMessage());
}
catch (Exception e)
{
log.error("Checking txIntervalCommitTimeServiceAvailable failed.", e);
}
}
}
MetadataTracker()
@@ -518,7 +595,7 @@ public class MetadataTracker extends CoreStatePublisher implements Tracker
}
protected Transactions getSomeTransactions(BoundedDeque<Transaction> txnsFound, Long fromCommitTime, long timeStep,
int maxResults, long endTime) throws AuthenticationException, IOException, JSONException, EncoderException
int maxResults, long endTime) throws AuthenticationException, IOException, JSONException, EncoderException, NoSuchMethodException
{
long actualTimeStep = timeStep;
@@ -546,6 +623,17 @@ public class MetadataTracker extends CoreStatePublisher implements Tracker
{
transactions = client.getTransactions(startTime, null, startTime + actualTimeStep, null, maxResults, shardstate);
startTime += actualTimeStep;
// If no transactions are found, advance the time window to the next available transaction commit time
if (nextTxCommitTimeServiceAvailable && transactions.getTransactions().size() == 0)
{
Long nextTxCommitTime = client.getNextTxCommitTime(coreName, startTime);
if (nextTxCommitTime != -1)
{
log.info("Advancing transactions from {} to {}", startTime, nextTxCommitTime);
transactions = client.getTransactions(nextTxCommitTime, null, nextTxCommitTime + actualTimeStep, null, maxResults, shardstate);
}
}
} while (((transactions.getTransactions().size() == 0) && (startTime < endTime))
|| ((transactions.getTransactions().size() > 0) && alreadyFoundTransactions(txnsFound, transactions)));
@@ -605,9 +693,46 @@ public class MetadataTracker extends CoreStatePublisher implements Tracker
*
*/
Long fromCommitTime = getTxFromCommitTime(txnsFound, state.getLastGoodTxCommitTimeInIndex());
Long fromCommitTime = getTxFromCommitTime(txnsFound,
state.getLastIndexedTxCommitTime() == 0 ? state.getLastGoodTxCommitTimeInIndex() : state.getLastIndexedTxCommitTime());
log.debug("#### Check txnsFound : " + txnsFound.size());
log.debug("======= fromCommitTime: " + fromCommitTime);
// When using DB_ID_RANGE, fromCommitTime cannot be before the commit time of the first transaction
// for the DB_ID_RANGE to be indexed and commit time of the last transaction cannot be lower than fromCommitTime.
// When there isn't nodes in that range, -1 is returned as commit times
if (docRouter instanceof DBIDRangeRouter && txIntervalCommitTimeServiceAvailable)
{
DBIDRangeRouter dbIdRangeRouter = (DBIDRangeRouter) docRouter;
Pair<Long, Long> commitTimes = client.getTxIntervalCommitTime(coreName,
dbIdRangeRouter.getStartRange(), dbIdRangeRouter.getEndRange());
Long shardMinCommitTime = commitTimes.getFirst();
Long shardMaxCommitTime = commitTimes.getSecond();
// Node Range it's not still available in repository
if (shardMinCommitTime == -1)
{
log.debug("#### [DB_ID_RANGE] No nodes in range [" + dbIdRangeRouter.getStartRange() + "-"
+ dbIdRangeRouter.getEndRange() + "] "
+ "exist in the repository. Skipping metadata tracking.");
return;
}
if (fromCommitTime > shardMaxCommitTime)
{
log.debug("#### [DB_ID_RANGE] Last commit time is greater that max commit time in in range ["
+ dbIdRangeRouter.getStartRange() + "-" + dbIdRangeRouter.getEndRange() + "]. "
+ "Skipping metadata tracking.");
return;
}
// Initial commit time for Node Range is greater than calculated from commit time
if (fromCommitTime < shardMinCommitTime)
{
log.debug("#### [DB_ID_RANGE] SKIPPING TRANSACTIONS FROM " + fromCommitTime + " TO "
+ shardMinCommitTime);
fromCommitTime = shardMinCommitTime;
}
}
log.debug("#### Get txn from commit time: " + fromCommitTime);
transactions = getSomeTransactions(txnsFound, fromCommitTime, TIME_STEP_1_HR_IN_MS, 2000,
@@ -836,7 +961,7 @@ public class MetadataTracker extends CoreStatePublisher implements Tracker
List<Node> filteredNodes = filterNodes(nodes);
if(filteredNodes.size() > 0)
{
this.infoServer.indexNodes(filteredNodes, true, false);
this.infoServer.indexNodes(filteredNodes, true);
}
}
@@ -856,9 +981,8 @@ public class MetadataTracker extends CoreStatePublisher implements Tracker
{
filteredList.add(node);
}
else
else if (cascadeTrackerEnabled)
{
if(node.getStatus() == SolrApiNodeStatus.UPDATED)
{
Node doCascade = new Node();
@@ -964,7 +1088,7 @@ public class MetadataTracker extends CoreStatePublisher implements Tracker
}
public IndexHealthReport checkIndex(Long toTx, Long toAclTx, Long fromTime, Long toTime)
throws IOException, AuthenticationException, JSONException, EncoderException
throws IOException, AuthenticationException, JSONException, EncoderException, NoSuchMethodException
{
// DB TX Count
long firstTransactionCommitTime = 0;
search-services/alfresco-search/src/main/java/org/alfresco/solr/tracker/SlaveCoreStatePublisher.java
+18
View File
@@ -1,3 +1,21 @@
/*
* Copyright (C) 2005-2019 Alfresco Software Limited.
*
* This file is part of Alfresco
*
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
*/
package org.alfresco.solr.tracker;
import static org.alfresco.solr.tracker.Tracker.Type.NODE_STATE_PUBLISHER;
search-services/alfresco-search/src/main/java/org/alfresco/solr/tracker/Tracker.java
-2
View File
@@ -32,8 +32,6 @@ public interface Tracker
boolean hasMaintenance() throws Exception;
Semaphore getWriteLock();
String getAlfrescoVersion();
void setShutdown(boolean shutdown);
search-services/alfresco-search/src/main/resources/solr/instance/conf/shared.properties
+4 -1
View File
@@ -30,4 +30,7 @@ alfresco.cross.locale.property.1={http://www.alfresco.org/model/content/1.0}lock
# alfresco.cross.locale.datatype.1={http://www.alfresco.org/model/dictionary/1.0}content
# alfresco.cross.locale.datatype.2={http://www.alfresco.org/model/dictionary/1.0}mltext
alfresco.model.tracker.cron=0/10 * * * * ? *
alfresco.model.tracker.cron=0/10 * * * * ? *
# Whether path queries are enabled.
alfresco.cascade.tracker.enabled=true
search-services/alfresco-search/src/main/resources/solr/instance/templates/noRerank/conf/solrconfig.xml
+1 -1
View File
@@ -1154,7 +1154,7 @@
https://wiki.apache.org/solr/SolrCloud/
-->
<requestHandler name="/replication" class="solr.ReplicationHandler" >
<requestHandler name="/replication" class="org.alfresco.solr.handler.AlfrescoReplicationHandler">
<!--
To enable simple master/slave replication, uncomment one of the
sections below, depending on whether this solr instance should be
search-services/alfresco-search/src/main/resources/solr/instance/templates/noRerank/conf/solrcore.properties
+8
View File
@@ -174,6 +174,14 @@ solr.suggester.enabled=true
# -1 to disable suggester build throttling
solr.suggester.minSecsBetweenBuilds=3600
#
# Request content text compression
# When enabling this option, Tomcat Connector or HTTP Web Proxy (NGINX, Apache) compression must be also enabled
# This setting can improve performance when having high network latency or large documents in the repository
#
solr.request.content.compress=false
#
# Limit the maximum text size of transformed content sent to the index - in bytes
#
search-services/alfresco-search/src/main/resources/solr/instance/templates/rerank/conf/solrcore.properties
+7
View File
@@ -174,6 +174,13 @@ solr.suggester.enabled=true
# -1 to disable suggester build throttling
solr.suggester.minSecsBetweenBuilds=3600
#
# Request content text compression
# When enabling this option, Tomcat Connector or HTTP Web Proxy (NGINX, Apache) compression must be also enabled
# This setting can improve performance when having high network latency or large documents in the repository
#
solr.request.content.compress=false
#
# Limit the maximum text size of transformed content sent to the index - in bytes
#
search-services/alfresco-search/src/test/java/org/alfresco/solr/AbstractAlfrescoDistributedIT.java
+50 -83
View File
@@ -33,7 +33,6 @@ import org.apache.solr.core.SolrCore;
import org.apache.solr.request.LocalSolrQueryRequest;
import org.apache.solr.request.SolrQueryRequest;
import org.apache.solr.response.SolrQueryResponse;
import org.junit.Before;
import org.junit.BeforeClass;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -83,8 +82,9 @@ import static org.alfresco.repo.search.adaptor.lucene.QueryConstants.FIELD_VERSI
* class hierarchy. Ideally this function should be retired in favour of better
* annotations..
*
* @since solr 1.5
* @author Michael Suzuki
* @since solr 1.4.1
* @author Michael Suzuki
* @author Andrea Gazzarini
*/
@ThreadLeakLingering(linger = 5000)
public abstract class AbstractAlfrescoDistributedIT extends SolrITInitializer
@@ -92,8 +92,8 @@ public abstract class AbstractAlfrescoDistributedIT extends SolrITInitializer
protected static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
protected String[] deadServers;
protected static SolrResponsesComparator solrComparator = new SolrResponsesComparator();
protected static RandomSupplier solrRandomSupplier;
protected static SolrResponsesComparator SOLR_RESPONSE_COMPARATOR = new SolrResponsesComparator();
protected static RandomSupplier SOLR_RANDOM_SUPPLIER;
// to stress with higher thread counts and requests, make sure the junit
// xml formatter is not being used (all output will be buffered before
@@ -112,34 +112,26 @@ public abstract class AbstractAlfrescoDistributedIT extends SolrITInitializer
* Test configs may use the <code>${hostContext}</code> variable to access
* this system property.
* </p>
*
*/
@BeforeClass
public static void setUpSolrTestProperties()
{
SOLR_RANDOM_SUPPLIER = new RandomSupplier();
System.setProperty("alfresco.test", "true");
System.setProperty("solr.tests.maxIndexingThreads", "10");
System.setProperty("solr.tests.ramBufferSizeMB", "1024");
}
@Before
public void setupPerTest()
{
this.solrRandomSupplier = new RandomSupplier();
}
public static String[] fieldNames = new String[]
{ "n_ti1", "n_f1", "n_tf1", "n_d1", "n_td1", "n_l1", "n_tl1", "n_dt1", "n_tdt1" };
public static final String[] FIELD_NAMES = new String[] { "n_ti1", "n_f1", "n_tf1", "n_d1", "n_td1", "n_l1", "n_tl1", "n_dt1", "n_tdt1" };
protected static String[] getFieldNames()
{
return fieldNames;
return FIELD_NAMES;
}
protected void putHandleDefaults() {
solrComparator.putHandleDefaults();
protected static void putHandleDefaults()
{
SOLR_RESPONSE_COMPARATOR.putHandleDefaults();
}
/**
@@ -147,8 +139,6 @@ public abstract class AbstractAlfrescoDistributedIT extends SolrITInitializer
* @param query - query to execute
* @param count - min number of results each shard must satisfy
* @param waitMillis - total ms to wait
* @return
* @throws Exception
*/
public static boolean checkMinCountPerShard(Query query, int count, long waitMillis) throws SolrServerException,IOException
{
@@ -160,7 +150,7 @@ public abstract class AbstractAlfrescoDistributedIT extends SolrITInitializer
for (SolrClient singleShard : shardedClients)
{
allShardCompliant = false;
int totalHits = 0;
int totalHits;
int cycles = 1;
while ((new Date()).getTime() < timeout && (!allShardCompliant))
{
@@ -172,11 +162,11 @@ public abstract class AbstractAlfrescoDistributedIT extends SolrITInitializer
}
try
{
Thread.sleep((long) (500 * cycles++));
Thread.sleep(500 * cycles++);
}
catch (InterruptedException e)
{
continue;
// Ignore
}
}
}
@@ -205,11 +195,6 @@ public abstract class AbstractAlfrescoDistributedIT extends SolrITInitializer
/**
* Waits until all cores (including shards) reach a count.
*
* @param query
* @param count
* @param waitMillis
* @throws Exception
*/
public static void waitForDocCountAllCores(Query query, int count, long waitMillis) throws Exception
{
@@ -230,12 +215,9 @@ public abstract class AbstractAlfrescoDistributedIT extends SolrITInitializer
/**
* Delele by query on all Clients
*
* @param q
* @throws Exception
*/
public static void deleteByQueryAllClients(String q) throws Exception {
public static void deleteByQueryAllClients(String q) throws Exception
{
List<SolrClient> clients = getStandaloneAndShardedClients();
for (SolrClient client : clients) {
@@ -245,14 +227,11 @@ public abstract class AbstractAlfrescoDistributedIT extends SolrITInitializer
/**
* Gets the Default test client.
*
* @return
*/
protected static SolrClient getDefaultTestClient()
{
return solrCollectionNameToStandaloneClient.get(DEFAULT_TEST_CORENAME);
}
protected static List<SolrClient> getShardedClients()
{
@@ -266,7 +245,7 @@ public abstract class AbstractAlfrescoDistributedIT extends SolrITInitializer
*/
public static List<SolrClient> getStandaloneAndShardedClients()
{
List<SolrClient> clients = new ArrayList();
List<SolrClient> clients = new ArrayList<>();
clients.addAll(solrCollectionNameToStandaloneClient.values());
clients.addAll(clientShards);
return clients;
@@ -275,17 +254,11 @@ public abstract class AbstractAlfrescoDistributedIT extends SolrITInitializer
public static List<SolrClient> getStandaloneClients()
{
List<SolrClient> clients = new ArrayList();
clients.addAll(solrCollectionNameToStandaloneClient.values());
return clients;
return new ArrayList<>(solrCollectionNameToStandaloneClient.values());
}
/**
* Waits for the doc count on the first core available, then checks all the Shards match.
* @param query
* @param count
* @param waitMillis
* @throws Exception
*/
public static void waitForDocCount(Query query, int count, long waitMillis) throws Exception
{
@@ -332,7 +305,8 @@ public abstract class AbstractAlfrescoDistributedIT extends SolrITInitializer
return escapedField + ":" + value + " ";
}
protected static String escapeQueryChars(String query) {
protected static String escapeQueryChars(String query)
{
return query.replaceAll("\\:", "\\\\:")
.replaceAll("\\{", "\\\\{")
.replaceAll("\\}", "\\\\}");
@@ -341,7 +315,7 @@ public abstract class AbstractAlfrescoDistributedIT extends SolrITInitializer
public static SolrQuery luceneToSolrQuery(Query query)
{
String[] terms = query.toString().split(" ");
String escapedQuery = new String();
String escapedQuery = "";
for (String t : terms)
{
escapedQuery += escapeQueryClause(t);
@@ -366,8 +340,10 @@ public abstract class AbstractAlfrescoDistributedIT extends SolrITInitializer
waitForShardsCount(solrQuery, count, waitMillis, start);
}
protected static void injectDocToShards(long txnId, long aclId, long dbId, String owner) throws Exception {
for(SolrClient clientShard : clientShards) {
protected static void injectDocToShards(long txnId, long aclId, long dbId, String owner) throws Exception
{
for(SolrClient clientShard : clientShards)
{
SolrInputDocument doc = new SolrInputDocument();
String id = AlfrescoSolrDataModel.getNodeDocumentId(AlfrescoSolrDataModel.DEFAULT_TENANT, aclId, dbId);
doc.addField(FIELD_SOLR4_ID, id);
@@ -383,21 +359,20 @@ public abstract class AbstractAlfrescoDistributedIT extends SolrITInitializer
/**
* Gets the cores for the jetty instances
* @return
*/
protected static List<SolrCore> getJettyCores(Collection<JettySolrRunner> runners)
{
List<SolrCore> cores = new ArrayList();
List<SolrCore> cores = new ArrayList<>();
for (JettySolrRunner jettySolrRunner : runners)
{
jettySolrRunner.getCoreContainer().getCores().forEach(aCore -> cores.add(aCore));
cores.addAll(jettySolrRunner.getCoreContainer().getCores());
}
return cores;
}
protected static List<AlfrescoCoreAdminHandler> getAdminHandlers(Collection<JettySolrRunner> runners)
{
List<AlfrescoCoreAdminHandler> coreAdminHandlers = new ArrayList();
List<AlfrescoCoreAdminHandler> coreAdminHandlers = new ArrayList<>();
for (JettySolrRunner jettySolrRunner : runners)
{
CoreContainer coreContainer = jettySolrRunner.getCoreContainer();
@@ -418,32 +393,27 @@ public abstract class AbstractAlfrescoDistributedIT extends SolrITInitializer
List<SolrClient> clients = getShardedClients();
SolrQuery query = luceneToSolrQuery(new TermQuery(new Term(FIELD_DOC_TYPE, SolrInformationServer.DOC_TYPE_NODE)));
StringBuilder error = new StringBuilder();
for (int i = 0; i < clients.size(); ++i)
for (SolrClient client : clients)
{
SolrClient client = clients.get(i);
QueryResponse response = client.query(query);
int totalHits = (int) response.getResults().getNumFound();
if(totalHits > 0)
if (totalHits > 0)
{
shardHit++;
}
if(totalHits < count)
if (totalHits < count)
{
if (ignoreZero && totalHits == 0)
{
log.info(client+": have zero hits ");
}
else
{
error.append(" "+client+": ");
error.append("Expected nodes per shard greater than "+count+" found "+totalHits+" : "+query.toString());
if (ignoreZero && totalHits == 0) {
log.info(client + ": have zero hits ");
} else {
error.append(" " + client + ": ");
error.append("Expected nodes per shard greater than " + count + " found " + totalHits + " : " + query.toString());
}
}
log.info(client+": Hits "+totalHits);
log.info(client + ": Hits " + totalHits);
}
@@ -560,7 +530,7 @@ public abstract class AbstractAlfrescoDistributedIT extends SolrITInitializer
protected String getShardsString()
{
Random r = solrRandomSupplier.getRandomGenerator();
Random r = SOLR_RANDOM_SUPPLIER.getRandomGenerator();
if (deadServers == null)
return shards;
@@ -610,7 +580,7 @@ public abstract class AbstractAlfrescoDistributedIT extends SolrITInitializer
protected static SolrInputDocument addRandFields(SolrInputDocument sdoc)
{
addFields(sdoc, solrRandomSupplier.getRandFields(getFieldNames(), solrRandomSupplier.getRandValues()));
addFields(sdoc, SOLR_RANDOM_SUPPLIER.getRandFields(getFieldNames(), SOLR_RANDOM_SUPPLIER.getRandValues()));
return sdoc;
}
@@ -662,7 +632,7 @@ public abstract class AbstractAlfrescoDistributedIT extends SolrITInitializer
{
UpdateResponse controlRsp = add(client1, params, sdocs);
UpdateResponse specificRsp = add(client2, params, sdocs);
solrComparator.compareSolrResponses(specificRsp, controlRsp);
SOLR_RESPONSE_COMPARATOR.compareSolrResponses(specificRsp, controlRsp);
return specificRsp;
}
@@ -724,9 +694,6 @@ public abstract class AbstractAlfrescoDistributedIT extends SolrITInitializer
/**
* * Commits to the specified client, and optionally all shards
* @param client
* @param andShards
* @throws Exception
*/
protected static void commit(SolrClient client, boolean andShards) throws Exception
{
@@ -743,7 +710,7 @@ public abstract class AbstractAlfrescoDistributedIT extends SolrITInitializer
protected static QueryResponse queryRandomShard(ModifiableSolrParams params) throws SolrServerException, IOException
{
Random r = solrRandomSupplier.getRandomGenerator();
Random r = SOLR_RANDOM_SUPPLIER.getRandomGenerator();
int which = r.nextInt(clientShards.size());
SolrClient client = clientShards.get(which);
QueryResponse rsp = client.query(params);
@@ -755,13 +722,13 @@ public abstract class AbstractAlfrescoDistributedIT extends SolrITInitializer
params.set("distrib", "false");
QueryRequest request = getAlfrescoRequest(json, params);
QueryResponse controlRsp = request.process(solrClient);
solrComparator.validateResponse(controlRsp);
SOLR_RESPONSE_COMPARATOR.validateResponse(controlRsp);
if (andShards)
{
params.remove("distrib");
setDistributedParams(params);
QueryResponse rsp = queryRandomShard(json, params);
solrComparator.compareResponses(rsp, controlRsp);
SOLR_RESPONSE_COMPARATOR.compareResponses(rsp, controlRsp);
return rsp;
}
else
@@ -772,7 +739,7 @@ public abstract class AbstractAlfrescoDistributedIT extends SolrITInitializer
protected static QueryResponse queryRandomShard(String json, SolrParams params) throws SolrServerException, IOException
{
Random r = solrRandomSupplier.getRandomGenerator();
Random r = SOLR_RANDOM_SUPPLIER.getRandomGenerator();
int which = r.nextInt(clientShards.size());
SolrClient client = clientShards.get(which);
QueryRequest request = getAlfrescoRequest(json, params);
@@ -791,13 +758,13 @@ public abstract class AbstractAlfrescoDistributedIT extends SolrITInitializer
*/
protected QueryResponse query(SolrClient solrClient, boolean setDistribParams, SolrParams p) throws Exception
{
Random r = solrRandomSupplier.getRandomGenerator();
Random r = SOLR_RANDOM_SUPPLIER.getRandomGenerator();
final ModifiableSolrParams params = new ModifiableSolrParams(p);
// TODO: look into why passing true causes fails
params.set("distrib", "false");
final QueryResponse controlRsp = solrClient.query(params);
solrComparator.validateResponse(controlRsp);
SOLR_RESPONSE_COMPARATOR.validateResponse(controlRsp);
params.remove("distrib");
if (setDistribParams)
@@ -805,7 +772,7 @@ public abstract class AbstractAlfrescoDistributedIT extends SolrITInitializer
QueryResponse rsp = queryRandomShard(params);
solrComparator.compareResponses(rsp, controlRsp);
SOLR_RESPONSE_COMPARATOR.compareResponses(rsp, controlRsp);
if (stress > 0)
{
@@ -827,7 +794,7 @@ public abstract class AbstractAlfrescoDistributedIT extends SolrITInitializer
QueryResponse rsp = client.query(new ModifiableSolrParams(params));
if (verifyStress)
{
solrComparator.compareResponses(rsp, controlRsp);
SOLR_RESPONSE_COMPARATOR.compareResponses(rsp, controlRsp);
}
} catch (SolrServerException | IOException e)
{
@@ -865,7 +832,7 @@ public abstract class AbstractAlfrescoDistributedIT extends SolrITInitializer
first = rsp;
} else
{
solrComparator.compareResponses(first, rsp);
SOLR_RESPONSE_COMPARATOR.compareResponses(first, rsp);
}
}
search-services/alfresco-search/src/test/java/org/alfresco/solr/AdminHandlerDistributedIT.java
+1 -1
View File
@@ -56,7 +56,7 @@ public class AdminHandlerDistributedIT extends AbstractAlfrescoDistributedIT
}
@AfterClass
private static void destroyData() throws Throwable
private static void destroyData()
{
dismissSolrServers();
}
search-services/alfresco-search/src/test/java/org/alfresco/solr/AlfrescoCollatableMLTextFieldTypeTest.java
+200
View File
@@ -0,0 +1,200 @@
/*
* Copyright (C) 2005-2014 Alfresco Software Limited.
*
* This file is part of Alfresco
*
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
*/
package org.alfresco.solr;
import static org.junit.Assert.assertEquals;
import static org.mockito.Mockito.reset;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
import static org.mockito.MockitoAnnotations.initMocks;
import java.text.Collator;
import java.util.Locale;
import org.alfresco.solr.AlfrescoCollatableMLTextFieldType.MLTextSortFieldComparator;
import org.apache.lucene.index.BinaryDocValues;
import org.apache.lucene.util.Bits;
import org.apache.lucene.util.BytesRef;
import org.junit.Before;
import org.junit.Test;
import org.mockito.InjectMocks;
import org.mockito.Mock;
/** Unit tests for {@link AlfrescoCollatableMLTextFieldType}. */
public class AlfrescoCollatableMLTextFieldTypeTest
{
private static final int NUM_HITS = 3;
private static final String FIELD = "field";
private static final Locale LOCALE = Locale.getDefault();
/** A document id. */
private static final int DOC = 0;
/** A value for the current bottom document. */
private static final String BOTTOM_STRING = "Bottom";
@InjectMocks
MLTextSortFieldComparator textSortFieldComparator = new MLTextSortFieldComparator(NUM_HITS, FIELD, LOCALE);
@Mock
BinaryDocValues mockDocTerms;
@Mock
Bits mockDocsWithField;
@Mock
Collator mockCollator;
@Before
public void setUp()
{
initMocks(this);
reset(mockDocTerms, mockDocsWithField);
textSortFieldComparator.bottom = BOTTOM_STRING;
}
/** Check that a zero length term is sorted before a populated field. */
@Test
public void testCompareBottom_termLengthZeroAndDocDoesntHaveField()
{
// Set up the document to have an empty term.
when(mockDocTerms.get(DOC)).thenReturn(new BytesRef());
when(mockDocsWithField.get(DOC)).thenReturn(false);
// Call the method under test.
int result = textSortFieldComparator.compareBottom(DOC);
assertEquals("Expected value for doc to be null, and so it to be sorted before BOTTOM_TERM", 1, result);
}
/**
* Check the behaviour of compareBottom when docsWithField is null (this happens when all documents contain the
* field).
*/
@Test
public void testCompareBottom_nullDocsWithField()
{
// Set docsWithField to null to simulate all documents containing the field.
Bits oldValue = textSortFieldComparator.docsWithField;
textSortFieldComparator.docsWithField = null;
// Set up the document to have an empty term.
when(mockDocTerms.get(DOC)).thenReturn(new BytesRef());
// Call the method under test.
textSortFieldComparator.compareBottom(DOC);
// Expect the EMPTY_TERM to be compared
verify(mockCollator).compare(BOTTOM_STRING, "");
// Reset docsWithField with the mock after the test.
textSortFieldComparator.docsWithField = oldValue;
}
/** Check that if the doc has a value then it is compared with the existing value. */
@Test
public void testCompareBottom_populatedTerm()
{
// Set up the document to have "Some value" for the field.
when(mockDocTerms.get(DOC)).thenReturn(new BytesRef("Some value"));
when(mockDocsWithField.get(DOC)).thenReturn(false);
// Call the method under test.
textSortFieldComparator.compareBottom(DOC);
verify(mockCollator).compare(BOTTOM_STRING, "Some value");
}
/** Check the behaviour if the multilanguage term is encoded. */
@Test
public void testCompareBottom_encodedTerm_localeFound()
{
// Create an encoded multilanguage string with Russian, US English and Thai with Thai digits.
String mlText = "\u0000ru\u0000First\u0000Ignored" +
"\u0000en_US\u0000Second\u0000IgnoredToo" +
"\u0000th_TH_TH\u0000Third\u0000AlsoIgnored";
// Set up the document to have an encoded value for the field.
when(mockDocTerms.get(DOC)).thenReturn(new BytesRef(mlText));
when(mockDocsWithField.get(DOC)).thenReturn(false);
// Check that the Russian text can be extracted.
textSortFieldComparator.collatorLocale = Locale.forLanguageTag("ru");
textSortFieldComparator.compareBottom(DOC);
verify(mockCollator).compare(BOTTOM_STRING, "First");
// Check that the English text can be extracted.
textSortFieldComparator.collatorLocale = Locale.forLanguageTag("en");
textSortFieldComparator.compareBottom(DOC);
verify(mockCollator).compare(BOTTOM_STRING, "Second");
// Check that the Thai text can be extracted.
textSortFieldComparator.collatorLocale = Locale.forLanguageTag("th");
textSortFieldComparator.compareBottom(DOC);
verify(mockCollator).compare(BOTTOM_STRING, "Third");
// Reset the locale for other tests.
textSortFieldComparator.collatorLocale = LOCALE;
}
/** Check the behaviour if the term has a locale but no text. */
@Test
public void testCompareBottom_badlyEncodedTerm()
{
// Set the value to have a locale but no text.
String mlText = "\u0000ru";
when(mockDocTerms.get(DOC)).thenReturn(new BytesRef(mlText));
// Call the method under test.
textSortFieldComparator.compareBottom(DOC);
// Check that an empty string is assumed.
verify(mockCollator).compare(BOTTOM_STRING, "");
}
@Test
public void testCompareValues_nullLessThanString()
{
int result = textSortFieldComparator.compareValues(null, "NotNull");
assertEquals("Expected null to be 'less' than string.", -1, result);
}
@Test
public void testCompareValues_stringGreaterThanNull()
{
int result = textSortFieldComparator.compareValues("NotNull", null);
assertEquals("Expected string to be 'greater' than null.", 1, result);
}
@Test
public void testCompareValues_nullEqualToNull()
{
int result = textSortFieldComparator.compareValues(null, null);
assertEquals("Expected two null values to be equal.", 0, result);
}
/** Check that when two non-null strings are compared then the underlying collator is used to get the result. */
@Test
public void testCompareValues_twoStringsCompared()
{
// An arbitrary value to be returned by the collator.
int comparisonResult = 10;
when(mockCollator.compare("NotNull1", "NotNull2")).thenReturn(comparisonResult);
// Call the method under test.
int result = textSortFieldComparator.compareValues("NotNull1", "NotNull2");
verify(mockCollator).compare("NotNull1", "NotNull2");
assertEquals("Expected result to be obtained from collator.", comparisonResult, result);
}
}
search-services/alfresco-search/src/test/java/org/alfresco/solr/AlfrescoCollatableTextFieldTypeTest.java
+168
View File
@@ -0,0 +1,168 @@
/*
* Copyright (C) 2005-2014 Alfresco Software Limited.
*
* This file is part of Alfresco
*
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
*/
package org.alfresco.solr;
import static org.junit.Assert.assertEquals;
import static org.mockito.Mockito.reset;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
import static org.mockito.MockitoAnnotations.initMocks;
import java.text.Collator;
import java.util.Locale;
import org.alfresco.solr.AlfrescoCollatableTextFieldType.TextSortFieldComparator;
import org.apache.lucene.index.BinaryDocValues;
import org.apache.lucene.util.Bits;
import org.apache.lucene.util.BytesRef;
import org.junit.Before;
import org.junit.Test;
import org.mockito.InjectMocks;
import org.mockito.Mock;
/** Unit tests for {@link AlfrescoCollatableTextFieldType}. */
public class AlfrescoCollatableTextFieldTypeTest
{
private static final int NUM_HITS = 3;
private static final String FIELD = "field";
private static final Locale LOCALE = Locale.getDefault();
/** A document id. */
private static final int DOC = 0;
/** A value for the current bottom document. */
private static final String BOTTOM_STRING = "Bottom";
@InjectMocks
TextSortFieldComparator textSortFieldComparator = new TextSortFieldComparator(NUM_HITS, FIELD, LOCALE);
@Mock
BinaryDocValues mockDocTerms;
@Mock
Bits mockDocsWithField;
@Mock
Collator mockCollator;
@Before
public void setUp()
{
initMocks(this);
reset(mockDocTerms, mockDocsWithField);
textSortFieldComparator.bottom = BOTTOM_STRING;
}
/** Check that a zero length term is sorted before a populated field. */
@Test
public void testCompareBottom_termLengthZeroAndDocDoesntHaveField()
{
// Set up the document to have an empty term.
when(mockDocTerms.get(DOC)).thenReturn(new BytesRef());
when(mockDocsWithField.get(DOC)).thenReturn(false);
// Call the method under test.
int result = textSortFieldComparator.compareBottom(DOC);
assertEquals("Expected value for doc to be null, and so it to be sorted before BOTTOM_TERM", 1, result);
}
/**
* Check the behaviour of compareBottom when docsWithField is null (this happens when all documents contain the
* field).
*/
@Test
public void testCompareBottom_nullDocsWithField()
{
// Set docsWithField to null to simulate all documents containing the field.
Bits oldValue = textSortFieldComparator.docsWithField;
textSortFieldComparator.docsWithField = null;
// Set up the document to have an empty term.
when(mockDocTerms.get(DOC)).thenReturn(new BytesRef());
// Call the method under test.
textSortFieldComparator.compareBottom(DOC);
// Expect the EMPTY_TERM to be compared
verify(mockCollator).compare(BOTTOM_STRING, "");
// Reset docsWithField with the mock after the test.
textSortFieldComparator.docsWithField = oldValue;
}
/** Check that if the doc has a value then it is compared with the existing value. */
@Test
public void testCompareBottom_populatedTerm()
{
// Set up the document to have "Some value" for the field.
when(mockDocTerms.get(DOC)).thenReturn(new BytesRef("Some value"));
when(mockDocsWithField.get(DOC)).thenReturn(false);
// Call the method under test.
textSortFieldComparator.compareBottom(DOC);
verify(mockCollator).compare(BOTTOM_STRING, "Some value");
}
/** Check the behaviour if the term is encoded. */
@Test
public void testCompareBottom_encodedTerm()
{
// Set up the document to have an encoded value for the field.
when(mockDocTerms.get(DOC)).thenReturn(new BytesRef("\u0000Value\u0000Ignored"));
when(mockDocsWithField.get(DOC)).thenReturn(false);
// Call the method under test.
textSortFieldComparator.compareBottom(DOC);
verify(mockCollator).compare(BOTTOM_STRING, "Value");
}
@Test
public void testCompareValues_nullLessThanString()
{
int result = textSortFieldComparator.compareValues(null, "NotNull");
assertEquals("Expected null to be 'less' than string.", -1, result);
}
@Test
public void testCompareValues_stringGreaterThanNull()
{
int result = textSortFieldComparator.compareValues("NotNull", null);
assertEquals("Expected string to be 'greater' than null.", 1, result);
}
@Test
public void testCompareValues_nullEqualToNull()
{
int result = textSortFieldComparator.compareValues(null, null);
assertEquals("Expected two null values to be equal.", 0, result);
}
/** Check that when two non-null strings are compared then the underlying collator is used to get the result. */
@Test
public void testCompareValues_twoStringsCompared()
{
// An arbitrary value to be returned by the collator.
int comparisonResult = 10;
when(mockCollator.compare("NotNull1", "NotNull2")).thenReturn(comparisonResult);
// Call the method under test.
int result = textSortFieldComparator.compareValues("NotNull1", "NotNull2");
verify(mockCollator).compare("NotNull1", "NotNull2");
assertEquals("Expected result to be obtained from collator.", comparisonResult, result);
}
}
search-services/alfresco-search/src/test/java/org/alfresco/solr/CoresCreateUpdateDistributedIT.java
+1 -1
View File
@@ -63,7 +63,7 @@ public class CoresCreateUpdateDistributedIT extends AbstractAlfrescoDistributedI
}
@After
private void destroyData() throws Throwable
private void destroyData()
{
dismissSolrServers();
}
search-services/alfresco-search/src/test/java/org/alfresco/solr/CoresCreateViaPropertyIT.java
+1 -1
View File
@@ -53,7 +53,7 @@ public class CoresCreateViaPropertyIT extends AbstractAlfrescoDistributedIT
}
@AfterClass
private static void destroyData() throws Throwable
private static void destroyData()
{
dismissSolrServers();
System.clearProperty(AlfrescoCoreAdminHandler.ALFRESCO_DEFAULTS);
search-services/alfresco-search/src/test/java/org/alfresco/solr/SolrITInitializer.java
+63 -82
View File
@@ -91,8 +91,8 @@ public abstract class SolrITInitializer extends SolrTestCaseJ4
//Standalone Tests
protected static SolrCore defaultCore;
protected static final int clientConnectionTimeout = DEFAULT_CONNECTION_TIMEOUT;;
protected static final int clientSoTimeout = 90000;;
protected static final int clientConnectionTimeout = DEFAULT_CONNECTION_TIMEOUT;
protected static final int clientSoTimeout = 90000;
protected static final String id = "id";
@@ -129,21 +129,22 @@ public abstract class SolrITInitializer extends SolrTestCaseJ4
jettyContainers = new HashMap<>();
nodeCnt = new AtomicInteger(0);
String serverName = testClassName;
currentTestName = serverName;
currentTestName = testClassName;
String[] coreNames = new String[]{DEFAULT_TEST_CORENAME};
distribSetUp(serverName);
RandomSupplier.RandVal.uniqueValues = new HashSet(); // reset random values
createServers(serverName, coreNames, numShards,solrcoreProperties);
System.setProperty("solr.solr.home", testDir.toPath().resolve(serverName).toString());
distribSetUp(testClassName);
RandomSupplier.RandVal.uniqueValues = new HashSet<>(); // reset random values
createServers(testClassName, coreNames, numShards,solrcoreProperties);
System.setProperty("solr.solr.home", testDir.toPath().resolve(testClassName).toString());
}
private static Properties addExplicitShardingProperty(Properties solrcoreProperties)
{
if(solrcoreProperties == null){
if(solrcoreProperties == null)
{
solrcoreProperties = new Properties();
}
if(solrcoreProperties.getProperty("shard.method")==null)
{
solrcoreProperties.put("shard.method", "EXPLICIT_ID");
@@ -151,7 +152,8 @@ public abstract class SolrITInitializer extends SolrTestCaseJ4
return solrcoreProperties;
}
public static void initSingleSolrServer(String testClassName, Properties solrcoreProperties) throws Throwable {
public static void initSingleSolrServer(String testClassName, Properties solrcoreProperties) throws Throwable
{
initSolrServers(0,testClassName,solrcoreProperties);
JettySolrRunner jsr = jettyContainers.get(testClassName);
@@ -163,7 +165,8 @@ public abstract class SolrITInitializer extends SolrTestCaseJ4
{
int i = 0;
extras = new String[solrcoreProperties.size()*2];
for (Map.Entry<Object, Object> prop:solrcoreProperties.entrySet()) {
for (Map.Entry<Object, Object> prop:solrcoreProperties.entrySet())
{
extras[i++] = "property."+prop.getKey();
extras[i++] = (String) prop.getValue();
}
@@ -185,7 +188,7 @@ public abstract class SolrITInitializer extends SolrTestCaseJ4
destroyServers();
distribTearDown();
boolean keepTests = Boolean.valueOf(System.getProperty("keep.tests"));
boolean keepTests = Boolean.parseBoolean(System.getProperty("keep.tests"));
if (!keepTests) FileUtils.deleteDirectory(testDir);
}
catch (Exception e)
@@ -203,7 +206,7 @@ public abstract class SolrITInitializer extends SolrTestCaseJ4
return System.getProperty("user.dir") + "/target/test-classes/test-files";
}
public static void distribSetUp(String serverName) throws Exception
public static void distribSetUp(String serverName)
{
SolrTestCaseJ4.resetExceptionIgnores(); // ignore anything with
// ignore_exception in it
@@ -213,7 +216,7 @@ public abstract class SolrITInitializer extends SolrTestCaseJ4
System.setProperty("solr.log.dir", testDir.toPath().resolve(serverName).toString());
}
public static void distribTearDown() throws Exception
public static void distribTearDown()
{
System.clearProperty("solr.directoryFactory");
System.clearProperty("solr.log.dir");
@@ -230,8 +233,6 @@ public abstract class SolrITInitializer extends SolrTestCaseJ4
/**
* Creates a JettySolrRunner (if one didn't exist already). DOES NOT START IT.
* @return
* @throws Exception
*/
protected static JettySolrRunner createJetty(String jettyKey, boolean basicAuth) throws Exception
{
@@ -243,19 +244,13 @@ public abstract class SolrITInitializer extends SolrTestCaseJ4
{
Path jettySolrHome = testDir.toPath().resolve(jettyKey);
seedSolrHome(jettySolrHome);
JettySolrRunner jetty = createJetty(jettySolrHome.toFile(), null, null, false, 0, getSchemaFile(), basicAuth);
return jetty;
return createJetty(jettySolrHome.toFile(), null, null, false, 0, getSchemaFile(), basicAuth);
}
}
/**
* Adds the core config information to the jetty file system.
* Its best to call this before calling start() on Jetty
* @param jettyKey
* @param sourceConfigName
* @param coreName
* @param additionalProperties
* @throws Exception
*/
protected static void addCoreToJetty(String jettyKey, String sourceConfigName, String coreName, Properties additionalProperties) throws Exception
{
@@ -289,17 +284,14 @@ public abstract class SolrITInitializer extends SolrTestCaseJ4
out.close();
in.close();
}
}
}
/**
* Starts jetty if its not already running
* @param jsr
* @throws Exception
*/
protected static void startJetty(JettySolrRunner jsr) throws Exception {
protected static void startJetty(JettySolrRunner jsr) throws Exception
{
if (!jsr.isRunning())
{
jsr.start();
@@ -310,37 +302,40 @@ public abstract class SolrITInitializer extends SolrTestCaseJ4
{
boolean basicAuth = additionalProperties != null ? Boolean.parseBoolean(additionalProperties.getProperty("BasicAuth", "false")) : false;
JettySolrRunner jsr = createJetty(jettyKey, basicAuth);
jettyContainers.put(jettyKey, jsr);
Properties properties = new Properties();
if(additionalProperties != null && additionalProperties.size() > 0) {
if(additionalProperties != null && additionalProperties.size() > 0)
{
properties.putAll(additionalProperties);
properties.remove("shard.method");
}
for (int i = 0; i < coreNames.length; i++)
for (String coreName : coreNames)
{
addCoreToJetty(jettyKey, coreNames[i], coreNames[i], properties);
addCoreToJetty(jettyKey, coreName, coreName, properties);
}
//Now start jetty
startJetty(jsr);
int jettyPort = jsr.getLocalPort();
for (int i = 0; i < coreNames.length; i++)
for (String coreName : coreNames)
{
String url = buildUrl(jettyPort) + "/" + coreNames[i];
String url = buildUrl(jettyPort) + "/" + coreName;
log.info(url);
solrCollectionNameToStandaloneClient.put(coreNames[i], createNewSolrClient(url));
solrCollectionNameToStandaloneClient.put(coreName, createNewSolrClient(url));
}
shardsArr = new String[numShards];
StringBuilder sb = new StringBuilder();
if (additionalProperties == null) {
if (additionalProperties == null)
{
additionalProperties = new Properties();
}
String[] ranges = {"0-100", "100-200", "200-300", "300-400"};
@@ -382,6 +377,7 @@ public abstract class SolrITInitializer extends SolrTestCaseJ4
solrHomes.add(jetty.getSolrHome());
jetty.stop();
}
for (SolrClient jClients : solrCollectionNameToStandaloneClient.values())
{
jClients.close();
@@ -409,25 +405,16 @@ public abstract class SolrITInitializer extends SolrTestCaseJ4
solrCollectionNameToStandaloneClient.clear();
}
public static JettySolrRunner createJetty(File solrHome, String dataDir, String shardList, boolean sslEnabled, int port, String schemaOverride, boolean basicAuth) throws Exception
public static JettySolrRunner createJetty(File solrHome, String dataDir, String shardList, boolean sslEnabled, int port, String schemaOverride, boolean basicAuth)
{
return createJetty(solrHome, dataDir, shardList, sslEnabled, port, schemaOverride, useExplicitNodeNames, basicAuth);
}
/**
* Create a solr jetty server.
*
* @param solrHome
* @param dataDir
* @param shardList
* @param port
* @param schemaOverride
* @param explicitCoreNodeName
* @return
* @throws Exception
*/
public static JettySolrRunner createJetty(File solrHome, String dataDir, String shardList, boolean sslEnabled, int port,
String schemaOverride, boolean explicitCoreNodeName, boolean basicAuth) throws Exception
String schemaOverride, boolean explicitCoreNodeName, boolean basicAuth)
{
Properties props = new Properties();
if (schemaOverride != null)
@@ -443,20 +430,21 @@ public abstract class SolrITInitializer extends SolrTestCaseJ4
{
props.setProperty("coreNodeName", Integer.toString(nodeCnt.incrementAndGet()));
}
SSLConfig sslConfig = new SSLConfig(sslEnabled, false, null, null, null, null);
JettyConfig config = null;
JettyConfig config;
if(basicAuth) {
System.out.println("###### adding basic auth ######");
if(basicAuth)
{
log.info("###### adding basic auth ######");
config = JettyConfig.builder().setContext("/solr").setPort(port).withFilter(BasicAuthFilter.class, "/sql/*").stopAtShutdown(true).withSSLConfig(sslConfig).build();
} else {
System.out.println("###### no basic auth ######");
log.info("###### no basic auth ######");
config = JettyConfig.builder().setContext("/solr").setPort(port).stopAtShutdown(true).withSSLConfig(sslConfig).build();
}
JettySolrRunner jetty = new JettySolrRunner(solrHome.getAbsolutePath(), props, config);
return jetty;
return new JettySolrRunner(solrHome.getAbsolutePath(), props, config);
}
/**
@@ -510,11 +498,8 @@ public abstract class SolrITInitializer extends SolrTestCaseJ4
*/
protected static void seedSolrHome(Path jettyHome) throws IOException
{
String solrxml = getSolrXml();
if (solrxml != null)
{
FileUtils.copyFile(new File(getTestFilesHome(), solrxml), jettyHome.resolve(getSolrXml()).toFile());
}
FileUtils.copyFile(new File(getTestFilesHome(), getSolrXml()), jettyHome.resolve(getSolrXml()).toFile());
//Add solr home conf folder with alfresco based configuration.
FileUtils.copyDirectory(new File(getTestFilesHome() + "/conf"), jettyHome.resolve("conf").toFile());
// Add alfresco data model def
@@ -547,36 +532,26 @@ public abstract class SolrITInitializer extends SolrTestCaseJ4
FileUtils.copyDirectory(coreSourceConfig.resolve("conf").toFile(), confDir.toFile());
}
protected void setupJettySolrHome(String coreName, Path jettyHome) throws IOException
public static class BasicAuthFilter implements Filter
{
seedSolrHome(jettyHome);
Properties coreProperties = new Properties();
coreProperties.setProperty("name", coreName);
coreProperties.setProperty("shard", "${shard:}");
coreProperties.setProperty("collection", "${collection:" + coreName + "}");
coreProperties.setProperty("config", "${solrconfig:solrconfig.xml}");
coreProperties.setProperty("schema", "${schema:schema.xml}");
coreProperties.setProperty("coreNodeName", "${coreNodeName:}");
writeCoreProperties(jettyHome.resolve("cores").resolve(coreName), coreProperties, coreName);
}
public static class BasicAuthFilter implements Filter {
public BasicAuthFilter() {
public BasicAuthFilter()
{
}
public void init(FilterConfig config) {
public void init(FilterConfig config)
{
}
public void doFilter(ServletRequest request, ServletResponse response, FilterChain filterChain)
throws IOException, ServletException {
throws IOException, ServletException
{
//Parse the basic auth filter
String auth = ((HttpServletRequest)request).getHeader("Authorization");
if(auth != null) {
if(auth != null)
{
auth = auth.replace("Basic ", "");
byte[] bytes = Base64.getDecoder().decode(auth);
String decodedBytes = new String(bytes);
@@ -584,18 +559,24 @@ public abstract class SolrITInitializer extends SolrTestCaseJ4
String user = pair[0];
String password = pair[1];
//Just look for the hard coded user and password.
if (user.equals("test") && password.equals("pass")) {
if (user.equals("test") && password.equals("pass"))
{
filterChain.doFilter(request, response);
} else {
}
else
{
((HttpServletResponse) response).sendError(HttpServletResponse.SC_FORBIDDEN);
}
} else {
}
else
{
((HttpServletResponse) response).sendError(HttpServletResponse.SC_FORBIDDEN);
}
}
public void destroy() {
public void destroy()
{
}
}
}
}
search-services/alfresco-search/src/test/java/org/alfresco/solr/SolrInformationServerTest.java
+3 -75
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2005-2014 Alfresco Software Limited.
* Copyright (C) 2005-2019 Alfresco Software Limited.
*
* This file is part of Alfresco
*
@@ -19,18 +19,15 @@
package org.alfresco.solr;
import java.io.IOException;
import java.util.Properties;
import org.alfresco.solr.client.SOLRAPIClient;
import org.alfresco.solr.content.SolrContentStore;
import org.apache.solr.common.SolrDocument;
import org.apache.solr.common.params.CommonParams;
import org.apache.solr.common.util.NamedList;
import org.apache.solr.common.util.SimpleOrderedMap;
import org.apache.solr.core.SolrCore;
import org.apache.solr.core.SolrResourceLoader;
import org.apache.solr.request.LocalSolrQueryRequest;
import org.apache.solr.request.SolrQueryRequest;
import org.apache.solr.request.SolrRequestHandler;
import org.apache.solr.response.SolrQueryResponse;
@@ -43,8 +40,6 @@ import org.mockito.junit.MockitoJUnitRunner;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNull;
import static org.junit.Assert.assertSame;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
@@ -99,6 +94,7 @@ public class SolrInformationServerTest
request = infoServer.newSolrQueryRequest();
}
@SuppressWarnings("unchecked")
@Test
public void testGetStateOk()
{
@@ -124,6 +120,7 @@ public class SolrInformationServerTest
/**
* GetState returns null in case the given id doesn't correspond to an existing state document.
*/
@SuppressWarnings("unchecked")
@Test
public void testGetStateWithStateNotFound_returnsNull()
{
@@ -143,73 +140,4 @@ public class SolrInformationServerTest
assertNull(document);
}
@Test
public void testIndexAcl() throws IOException
{
/*
// Source/expected data
List<AclReaders> aclReadersList = new ArrayList<AclReaders>();
aclReadersList.add(new AclReaders(101, Arrays.asList("r1", "r2", "r3"), Arrays.asList("d1", "d2"), 999,
"example.com"));
aclReadersList.add(new AclReaders(102, Arrays.asList("r4", "r5", "r6"), Arrays.asList("d3", "d4"), 999,
"another.test"));
aclReadersList.add(new AclReaders(103, Arrays.asList("GROUP_marketing", "simpleuser", "GROUP_EVERYONE",
"ROLE_GUEST", "ROLE_ADMINISTRATOR", "ROLE_OWNER", "ROLE_RANDOM"), Arrays.asList(
"GROUP_engineering", "justauser", "GROUP_EVERYONE", "ROLE_GUEST", "ROLE_ADMINISTRATOR",
"ROLE_OWNER", "ROLE_RANDOM"), 999, "tenant.test"));
aclReadersList.add(new AclReaders(104, Arrays.asList("GROUP_marketing", "simpleuser", "GROUP_EVERYONE",
"ROLE_GUEST", "ROLE_ADMINISTRATOR", "ROLE_OWNER", "ROLE_RANDOM"), Arrays.asList(
"GROUP_engineering", "justauser", "GROUP_EVERYONE", "ROLE_GUEST", "ROLE_ADMINISTRATOR",
"ROLE_OWNER", "ROLE_RANDOM"), 999, "" Zero-length tenant == no mangling ));
//final boolean willOverwrite = true;
// Invoke the method under test
//infoServer.indexAcl(aclReadersList, willOverwrite);
// Capture the AddUpdateCommand instances for further analysis.
//ArgumentCaptor<AddUpdateCommand> cmdArg = ArgumentCaptor.forClass(AddUpdateCommand.class);
// The processor processes as many add commands as there are items in the aclReaderList.
//verify(processor, times(aclReadersList.size())).processAdd(cmdArg.capture());
// Verify that the AddUpdateCommand is as expected.
//List<AddUpdateCommand> updates = cmdArg.getAllValues();
//assertEquals("Wrong number of updates", aclReadersList.size(), updates.size());
/*
for (int docIndex = 0; docIndex < updates.size(); docIndex++)
{
AddUpdateCommand update = updates.get(docIndex);
assertEquals("Overwrite flag was not correct value.", willOverwrite, update.overwrite);
SolrInputDocument inputDoc = update.getSolrInputDocument();
// Retrieve the original AclReaders object and compare with data in submitted SolrInputDocument
final AclReaders sourceAclReaders = aclReadersList.get(docIndex);
assertEquals(
AlfrescoSolrDataModel.getTenantId(sourceAclReaders.getTenantDomain()) + "!"
+ NumericEncoder.encode(sourceAclReaders.getId()) + "!ACL",
inputDoc.getFieldValue("id").toString());
assertEquals("0", inputDoc.getFieldValue("_version_").toString());
assertEquals(sourceAclReaders.getId(), inputDoc.getFieldValue(QueryConstants.FIELD_ACLID));
assertEquals(sourceAclReaders.getAclChangeSetId(), inputDoc.getFieldValue(QueryConstants.FIELD_INACLTXID));
if (sourceAclReaders.getId() == 103)
{
// Authorities *may* (e.g. GROUP, EVERYONE, GUEST) be mangled to include tenant information
final Collection<Object> docReaders = inputDoc.getFieldValues(QueryConstants.FIELD_READER);
assertEquals(Arrays.asList("GROUP_marketing@tenant.test", "simpleuser", "GROUP_EVERYONE@tenant.test",
"ROLE_GUEST@tenant.test", "ROLE_ADMINISTRATOR", "ROLE_OWNER", "ROLE_RANDOM"), docReaders);
final Collection<Object> docDenied = inputDoc.getFieldValues(QueryConstants.FIELD_DENIED);
assertEquals(Arrays.asList("GROUP_engineering@tenant.test", "justauser", "GROUP_EVERYONE@tenant.test",
"ROLE_GUEST@tenant.test", "ROLE_ADMINISTRATOR", "ROLE_OWNER", "ROLE_RANDOM"), docDenied);
}
else
{
// Simple case, no authority/tenant mangling.
assertEquals(sourceAclReaders.getReaders(), inputDoc.getFieldValues(QueryConstants.FIELD_READER));
assertEquals(sourceAclReaders.getDenied(), inputDoc.getFieldValues(QueryConstants.FIELD_DENIED));
}
}
*/
}
}
search-services/alfresco-search/src/test/java/org/alfresco/solr/TemplatesDistributedIT.java
+1 -1
View File
@@ -52,7 +52,7 @@ public class TemplatesDistributedIT extends AbstractAlfrescoDistributedIT
}
@AfterClass
private static void destroyData() throws Throwable
private static void destroyData()
{
dismissSolrServers();
}
search-services/alfresco-search/src/test/java/org/alfresco/solr/handler/ContentStoreReplicationIT.java
+2 -2
View File
@@ -141,8 +141,8 @@ public class ContentStoreReplicationIT extends AbstractAlfrescoDistributedIT
{
master.stop();
slave.stop();
FileUtils.forceDelete(new File(masterSolrHome.getParent().toUri()));
FileUtils.forceDelete(new File(slaveSolrHome.getParent().toUri()));
FileUtils.deleteQuietly(new File(masterSolrHome.getParent().toUri()));
FileUtils.deleteQuietly(new File(slaveSolrHome.getParent().toUri()));
SOLRAPIQueueClient.nodeMetaDataMap.clear();
SOLRAPIQueueClient.transactionQueue.clear();
search-services/alfresco-search/src/test/java/org/alfresco/solr/highlight/AlfrescoHighligherDistributedIT.java
+1 -1
View File
@@ -46,7 +46,7 @@ public class AlfrescoHighligherDistributedIT extends AbstractAlfrescoDistributed
}
@AfterClass
private static void destroyData() throws Throwable
private static void destroyData()
{
dismissSolrServers();
}
search-services/alfresco-search/src/test/java/org/alfresco/solr/lifecycle/SolrCoreLoadListenerTest.java
+50 -15
View File
@@ -18,6 +18,30 @@
*/
package org.alfresco.solr.lifecycle;
import static java.util.Arrays.asList;
import static org.alfresco.solr.SolrInformationServer.CASCADE_TRACKER_ENABLED;
import static org.alfresco.solr.tracker.Tracker.Type.ACL;
import static org.alfresco.solr.tracker.Tracker.Type.CASCADE;
import static org.alfresco.solr.tracker.Tracker.Type.CONTENT;
import static org.alfresco.solr.tracker.Tracker.Type.METADATA;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertTrue;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.ArgumentMatchers.same;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.never;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
import static org.mockito.MockitoAnnotations.initMocks;
import java.util.List;
import java.util.Properties;
import java.util.Set;
import java.util.stream.Collectors;
import org.alfresco.solr.SolrInformationServer;
import org.alfresco.solr.client.SOLRAPIClient;
import org.alfresco.solr.tracker.AclTracker;
@@ -26,6 +50,7 @@ import org.alfresco.solr.tracker.ContentTracker;
import org.alfresco.solr.tracker.MetadataTracker;
import org.alfresco.solr.tracker.SolrTrackerScheduler;
import org.alfresco.solr.tracker.Tracker;
import org.alfresco.solr.tracker.Tracker.Type;
import org.alfresco.solr.tracker.TrackerRegistry;
import org.apache.solr.core.SolrConfig;
import org.apache.solr.core.SolrCore;
@@ -36,20 +61,6 @@ import org.mockito.Mock;
import org.mockito.junit.MockitoJUnitRunner;
import org.xml.sax.InputSource;
import java.util.List;
import java.util.Properties;
import static java.util.Arrays.asList;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertTrue;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.ArgumentMatchers.same;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
/**
* Unit tests for the {@link SolrCoreLoadListener}.
*
@@ -83,6 +94,8 @@ public class SolrCoreLoadListenerTest
@Before
public void setUp()
{
initMocks(this);
listener = new SolrCoreLoadListener(core);
when(core.getName()).thenReturn(coreName);
@@ -104,7 +117,29 @@ public class SolrCoreLoadListenerTest
verify(scheduler).schedule(any(MetadataTracker.class), eq(coreName), same(coreProperties));
verify(scheduler).schedule(any(CascadeTracker.class), eq(coreName), same(coreProperties));
assertEquals(4, coreTrackers.size());
Set<Type> trackerTypes = coreTrackers.stream().map(Tracker::getType).collect(Collectors.toSet());
assertEquals("Unexpected trackers found.", Set.of(ACL, CONTENT, METADATA, CASCADE), trackerTypes);
}
@Test
public void testDisabledCascadeTracking()
{
coreProperties.put(CASCADE_TRACKER_ENABLED, "false");
List<Tracker> coreTrackers = listener.createAndScheduleCoreTrackers(core, registry, coreProperties, scheduler, api, informationServer);
verify(registry).register(eq(coreName), any(AclTracker.class));
verify(registry).register(eq(coreName), any(ContentTracker.class));
verify(registry).register(eq(coreName), any(MetadataTracker.class));
verify(registry, never()).register(eq(coreName), any(CascadeTracker.class));
verify(scheduler).schedule(any(AclTracker.class), eq(coreName), same(coreProperties));
verify(scheduler).schedule(any(ContentTracker.class), eq(coreName), same(coreProperties));
verify(scheduler).schedule(any(MetadataTracker.class), eq(coreName), same(coreProperties));
verify(scheduler, never()).schedule(any(CascadeTracker.class), eq(coreName), same(coreProperties));
Set<Type> trackerTypes = coreTrackers.stream().map(Tracker::getType).collect(Collectors.toSet());
assertEquals("Unexpected trackers found.", Set.of(ACL, CONTENT, METADATA), trackerTypes);
}
@Test
search-services/alfresco-search/src/test/java/org/alfresco/solr/query/AlfrescoSolrSortIT.java
+108
View File
@@ -0,0 +1,108 @@
/*
* Copyright (C) 2005-2014 Alfresco Software Limited.
*
* This file is part of Alfresco
*
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
*/
package org.alfresco.solr.query;
import org.alfresco.solr.AbstractAlfrescoDistributedIT;
import org.apache.lucene.util.LuceneTestCase;
import org.apache.solr.SolrTestCaseJ4;
import org.apache.solr.client.solrj.response.QueryResponse;
import org.apache.solr.common.SolrDocumentList;
import org.apache.solr.common.util.NamedList;
import org.junit.After;
import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Rule;
import org.junit.Test;
import static org.hamcrest.core.Is.is;
/**
* https://issues.alfresco.com/jira/browse/SEARCH-2012
*/
@SolrTestCaseJ4.SuppressSSL
@LuceneTestCase.SuppressCodecs({"Appending","Lucene3x","Lucene40","Lucene41","Lucene42","Lucene43", "Lucene44", "Lucene45","Lucene46","Lucene47","Lucene48","Lucene49"})
public class AlfrescoSolrSortIT extends AbstractAlfrescoDistributedIT
{
@BeforeClass
private static void initData() throws Throwable
{
initSolrServers(1, getClassName(), null);
}
@AfterClass
private static void destroyData()
{
dismissSolrServers();
}
@After
public void clearData() throws Exception
{
deleteByQueryAllClients("*:*");
}
@Test
public void AlfrescoCollatableFieldType_emptyValuesSortingAsc__shouldBeRankedFirst() throws Exception {
prepareIndexSegmentWithAllNonNullFieldValues("text@s__sort@{http://www.alfresco.org/model/content/1.0}title");
putHandleDefaults();
// Docs with id 1, 3, 5 and 6 should be first (note that these will be sorted by indexing time).
String[] expectedRanking = new String[]{"1","3","5","6","4","2"};
QueryResponse response = query(getDefaultTestClient(), true,
"{\"query\":\"(id:(1 2 3 4 5 6))\",\"locales\":[\"en\"], \"templates\": [{\"name\":\"t1\", \"template\":\"%cm:content\"}], \"authorities\": [\"joel\"], \"tenants\": []}",
params("qt", "/afts", "shards.qt", "/afts", "start", "0", "rows", "100", "sort", "text@s__sort@{http://www.alfresco.org/model/content/1.0}title asc"));
NamedList res = response.getResponse();
SolrDocumentList searchResults = (SolrDocumentList)res.get("response");
for(int i=0;i<searchResults.size();i++){
assertThat(searchResults.get(i).get("id"),is(expectedRanking[i]));
}
}
@Test
public void AlfrescoMLCollatableFieldType_emptyValuesSortingDesc_shouldRankThemLast() throws Exception {
prepareIndexSegmentWithAllNonNullFieldValues("mltext@m__sort@{http://www.alfresco.org/model/content/1.0}title");
putHandleDefaults();
// Docs with id 1, 3, 5 and 6 should be last (note that these will be sorted by indexing time).
String[] expectedRanking = new String[]{"2","4","1","3","5","6"};
QueryResponse response = query(getDefaultTestClient(), true,
"{\"query\":\"(id:(1 2 3 4 5 6))\",\"locales\":[\"en\"], \"templates\": [{\"name\":\"t1\", \"template\":\"%cm:content\"}], \"authorities\": [\"joel\"], \"tenants\": []}",
params("qt", "/afts", "shards.qt", "/afts", "start", "0", "rows", "100", "sort", "mltext@m__sort@{http://www.alfresco.org/model/content/1.0}title desc"));
NamedList res = response.getResponse();
SolrDocumentList searchResults = (SolrDocumentList)res.get("response");
for(int i=0;i<searchResults.size();i++){
assertThat(searchResults.get(i).get("id"),is(expectedRanking[i]));
}
}
/** Create six documents where the field is null for docs with id 1, 3, 5 and 6, and populated for docs with id 2 and 4. */
private void prepareIndexSegmentWithAllNonNullFieldValues(String field) throws Exception {
index(getDefaultTestClient(), true, "id", "1", "_version_", "0", field, "");
index(getDefaultTestClient(), true, "id", "2", "_version_", "0", field, "B");
index(getDefaultTestClient(), true, "id", "3", "_version_", "0", field, "");
index(getDefaultTestClient(), true, "id", "4", "_version_", "0", field, "A");
index(getDefaultTestClient(), true, "id", "5", "_version_", "0", field, "");
index(getDefaultTestClient(), true, "id", "6", "_version_", "0", field, "");
commit(getDefaultTestClient(), true);
}
}
search-services/alfresco-search/src/test/java/org/alfresco/solr/query/DistributedAlfrescoSolrFacetingIT.java
+1 -1
View File
@@ -48,7 +48,7 @@ import static org.hamcrest.core.Is.is;
}
@AfterClass
private static void destroyData() throws Throwable
private static void destroyData()
{
dismissSolrServers();
}
search-services/alfresco-search/src/test/java/org/alfresco/solr/query/DistributedAlfrescoSolrSpellcheckerIT.java
+1 -1
View File
@@ -42,7 +42,7 @@ public class DistributedAlfrescoSolrSpellcheckerIT extends AbstractAlfrescoDistr
}
@AfterClass
private static void destroyData() throws Throwable
private static void destroyData()
{
dismissSolrServers();
}
search-services/alfresco-search/src/test/java/org/alfresco/solr/tracker/AclTrackerTest.java
-282
View File
@@ -1,282 +0,0 @@
/*
* Copyright (C) 2005-2014 Alfresco Software Limited.
*
* This file is part of Alfresco
*
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
*/
package org.alfresco.solr.tracker;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertNotNull;
import static org.junit.Assert.assertTrue;
import static org.mockito.Mockito.doThrow;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.never;
import static org.mockito.Mockito.spy;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
import java.io.IOException;
import java.util.Collections;
import java.util.List;
import java.util.Properties;
import java.util.concurrent.ConcurrentLinkedQueue;
import org.alfresco.solr.IndexTrackingShutdownException;
import org.alfresco.solr.InformationServer;
import org.alfresco.solr.TrackerState;
import org.alfresco.solr.client.AclChangeSet;
import org.alfresco.solr.client.AclChangeSets;
import org.alfresco.solr.client.SOLRAPIClient;
import org.apache.commons.lang.reflect.FieldUtils;
import org.junit.Before;
import org.junit.Ignore;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.mockito.Mock;
import org.mockito.runners.MockitoJUnitRunner;
/**
* Unit tests for the {@link AclTracker} class.
*
* @author Matt Ward
*/
@RunWith(MockitoJUnitRunner.class)
public class AclTrackerTest
{
private static final Object CUSTOM_ALFRESCO_VERSION = "99.9.9";
// The class under test
private AclTracker tracker;
private @Mock SOLRAPIClient client;
private @Mock InformationServer informationServer;
private TrackerState trackerState;
@Before
public void setUp() throws Exception
{
trackerState = new TrackerState();
trackerState.setRunning(false); // Nothing would happen if it were already running.
Properties props = createProperties();
// Spy in order to stub some methods and verify others. Note: non-stubbed methods go to the real class.
tracker = spy(new AclTracker(props, client, "core-name", informationServer));
}
private Properties createProperties()
{
Properties props = new Properties();
props.put("alfresco.stores", "workspace://SpacesStore");
props.put("alfresco.version", CUSTOM_ALFRESCO_VERSION);
return props;
}
protected void testTrackChangesRan()
{
when(tracker.getTrackerState()).thenReturn(trackerState);
tracker.track();
assertFalse(trackerState.isRunning());
assertFalse(trackerState.isCheck());
}
// High level test of doTrack() workflow.
@Test
@Ignore("Superseded by AlfrescoSolrTrackerTest")
public void checkTrackingOperaionsPerformed() throws Throwable
{
testTrackChangesRan();
verify(tracker).purgeAclChangeSets();
verify(tracker).purgeAcls();
verify(tracker).reindexAclChangeSets();
verify(tracker).reindexAcls();
verify(tracker).indexAclChangeSets();
verify(tracker).indexAcls();
verify(tracker).trackRepository();
}
// Tests the purgeAclChangeSets() call made in AclTracker.doTrack()
// TODO: the other operations in doTrack().
@Test
@Ignore("Superseded by AlfrescoSolrTrackerTest")
public void checkTrackingWhenAclChangeSetsToPurge() throws IllegalAccessException, IOException
{
@SuppressWarnings("unchecked")
ConcurrentLinkedQueue<Long> aclChangeSetsToPurge = (ConcurrentLinkedQueue<Long>)
FieldUtils.readField(tracker, "aclChangeSetsToPurge", true);
aclChangeSetsToPurge.add(101L);
aclChangeSetsToPurge.add(102L);
aclChangeSetsToPurge.add(103L);
// Invoke the behaviour we're testing
testTrackChangesRan();
// aclChangeSetsToPurge
verify(informationServer).deleteByAclChangeSetId(101L);
verify(informationServer).deleteByAclChangeSetId(102L);
verify(informationServer).deleteByAclChangeSetId(103L);
// TODO: verify checkShutdown
verify(informationServer).commit();
}
@Test
@Ignore("Superseded by AlfrescoSolrTrackerTest")
public void checkTrackingWhenAclsToPurge() throws IllegalAccessException, IOException
{
@SuppressWarnings("unchecked")
ConcurrentLinkedQueue<Long> aclsToPurge = (ConcurrentLinkedQueue<Long>)
FieldUtils.readField(tracker, "aclsToPurge", true);
aclsToPurge.add(201L);
aclsToPurge.add(202L);
aclsToPurge.add(203L);
// Invoke the behaviour we're testing
testTrackChangesRan();
// aclsToPurge
verify(informationServer).deleteByAclId(201L);
verify(informationServer).deleteByAclId(202L);
verify(informationServer).deleteByAclId(203L);
// TODO: verify checkShutdown
verify(informationServer).commit();
}
// TODO: Commented out, approach not working
// Rethink or finish later.
/*@Test
public void checkTrackingWhenAclChangeSetsToReindex() throws IllegalAccessException, IOException, AuthenticationException, JSONException
{
@SuppressWarnings("unchecked")
ConcurrentLinkedQueue<Long> aclChangeSetsToReindex = (ConcurrentLinkedQueue<Long>)
FieldUtils.readField(tracker, "aclChangeSetsToReindex", true);
aclChangeSetsToReindex.add(301L);
aclChangeSetsToReindex.add(302L);
aclChangeSetsToReindex.add(303L);
// tracker will loop through list of ACLs to reindex, and get changesets for each changeset ID.
when(client.getAclChangeSets(null, 201L, null, 202L, 1)).thenReturn(mockChangeSets(201L));
when(client.getAclChangeSets(null, 202L, null, 203L, 1)).thenReturn(mockChangeSets(202L));
when(client.getAclChangeSets(null, 203L, null, 204L, 1)).thenReturn(mockChangeSets(203L));
when(client.getAcls(Collections.singletonList(new AclChangeSet(201L, 0L, 1)), null, Integer.MAX_VALUE)).thenReturn(Collections.singletonList(new Acl(201L, 0L)));
when(client.getAcls(Collections.singletonList(new AclChangeSet(202L, 0L, 1)), null, Integer.MAX_VALUE)).thenReturn(Collections.singletonList(new Acl(202L, 0L)));
when(client.getAcls(Collections.singletonList(new AclChangeSet(203L, 0L, 1)), null, Integer.MAX_VALUE)).thenReturn(Collections.singletonList(new Acl(203L, 0L)));
// TODO: ...more...
// Invoke the behaviour we're testing
testTrackChangesRan();
verify(informationServer).deleteByAclChangeSetId(301L);
verify(informationServer).deleteByAclChangeSetId(302L);
verify(informationServer).deleteByAclChangeSetId(303L);
// TODO: verify checkShutdown
verify(informationServer).commit();
}*/
private AclChangeSets mockChangeSets(long id)
{
List<AclChangeSet> changeSets = Collections.singletonList(new AclChangeSet(id, 0L, 1));
AclChangeSets acs = mock(AclChangeSets.class);
when(acs.getAclChangeSets()).thenReturn(changeSets);
return acs;
}
@Test
@Ignore("Superseded by AlfrescoSolrTrackerTest")
public void trackingAbortsWhenAlreadyRunning() throws Throwable
{
trackerState.setRunning(true);
// Prove running state, before attempt to track()
assertTrue(trackerState.isRunning());
FieldUtils.writeField(tracker, "state", trackerState, true);
tracker.track();
// Still running - these values are unaffected.
assertTrue(trackerState.isRunning());
assertFalse(trackerState.isCheck());
// Prove doTrack() was not called
verify(tracker, never()).doTrack();
}
@Test
@Ignore("Superseded by AlfrescoSolrTrackerTest")
public void willRollbackOnThrowableDuringTracking() throws Throwable
{
doThrow(new RuntimeException("Simulated problem during tracking")).when(tracker).doTrack();
testTrackChangesRan();
verify(informationServer).rollback();
}
@Test
@Ignore("Superseded by AlfrescoSolrTrackerTest")
public void willRollbackOnIndexTrackingShutdownException() throws Throwable
{
doThrow(new IndexTrackingShutdownException()).when(tracker).doTrack();
testTrackChangesRan();
verify(informationServer).rollback();
}
@Ignore("Not yet implemented.")
@Test
public void canCheckIndex()
{
// TODO
// tracker.checkIndex(fromTx, toTx, fromAclTx, toAclTx, fromTime, toTime);
}
@Test
@Ignore("Superseded by AlfrescoSolrTrackerTest")
public void canGetAlfrescoVersion()
{
// Check we're testing something useful
assertNotNull(CUSTOM_ALFRESCO_VERSION);
// Check alfresco version retrieved from properties
assertEquals(CUSTOM_ALFRESCO_VERSION, tracker.getAlfrescoVersion());
}
@Test
@Ignore("Superseded by AlfrescoSolrTrackerTest")
public void canClose() throws IllegalAccessException
{
ThreadHandler threadHandler = (ThreadHandler) FieldUtils.readField(tracker, "threadHandler", true);
threadHandler = spy(threadHandler);
FieldUtils.writeField(tracker, "threadHandler", threadHandler, true);
tracker.shutdown();
// AclTracker specific
verify(threadHandler).shutDownThreadPool();
// Applicable to all AbstractAclTracker
verify(client).close();
}
}
search-services/alfresco-search/src/test/java/org/alfresco/solr/tracker/CascadeTrackerIT.java → search-services/alfresco-search/src/test/java/org/alfresco/solr/tracker/CascadingIT.java
+1 -1
View File
@@ -52,7 +52,7 @@ import java.util.List;
@LuceneTestCase.SuppressCodecs({"Appending","Lucene3x","Lucene40","Lucene41","Lucene42","Lucene43", "Lucene44", "Lucene45","Lucene46","Lucene47","Lucene48","Lucene49"})
@SolrTestCaseJ4.SuppressSSL
public class CascadeTrackerIT extends AbstractAlfrescoSolrIT
public class CascadingIT extends AbstractAlfrescoSolrIT
{
private static long MAX_WAIT_TIME = 80000;
search-services/alfresco-search/src/test/java/org/alfresco/solr/tracker/ContentPropertyValueTrackerIT.java
+125
View File
@@ -0,0 +1,125 @@
/*
* Copyright (C) 2005-2019 Alfresco Software Limited.
*
* This file is part of Alfresco
*
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
*/
package org.alfresco.solr.tracker;
import org.alfresco.model.ContentModel;
import org.alfresco.solr.AbstractAlfrescoDistributedIT;
import org.alfresco.solr.client.Acl;
import org.alfresco.solr.client.AclChangeSet;
import org.alfresco.solr.client.AclReaders;
import org.alfresco.solr.client.ContentPropertyValue;
import org.alfresco.solr.client.Node;
import org.alfresco.solr.client.NodeMetaData;
import org.alfresco.solr.client.StringPropertyValue;
import org.alfresco.solr.client.Transaction;
import org.apache.lucene.index.Term;
import org.apache.lucene.search.TermQuery;
import org.apache.lucene.util.LuceneTestCase;
import org.apache.solr.SolrTestCaseJ4;
import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Test;
import java.util.Locale;
import static org.alfresco.solr.AlfrescoSolrUtils.MAX_WAIT_TIME;
import static org.alfresco.solr.AlfrescoSolrUtils.getAcl;
import static org.alfresco.solr.AlfrescoSolrUtils.getAclChangeSet;
import static org.alfresco.solr.AlfrescoSolrUtils.getAclReaders;
import static org.alfresco.solr.AlfrescoSolrUtils.getNode;
import static org.alfresco.solr.AlfrescoSolrUtils.getNodeMetaData;
import static org.alfresco.solr.AlfrescoSolrUtils.getTransaction;
import static org.alfresco.solr.AlfrescoSolrUtils.indexAclChangeSet;
import static org.alfresco.solr.AlfrescoSolrUtils.list;
/**
* @author Elia Porciani
*/
@SolrTestCaseJ4.SuppressSSL
@LuceneTestCase.SuppressCodecs({"Appending","Lucene3x","Lucene40","Lucene41","Lucene42","Lucene43", "Lucene44", "Lucene45","Lucene46","Lucene47","Lucene48","Lucene49"})
public class ContentPropertyValueTrackerIT extends AbstractAlfrescoDistributedIT
{
final private String authorField = "text@s__lt@{http://www.alfresco.org/model/content/1.0}author";
@BeforeClass
private static void initData() throws Throwable
{
initSolrServers(1, "ContentPropertyValueTrackerIT", null);
}
@AfterClass
private static void destroyData()
{
dismissSolrServers();
}
/**
*
* MNT-21076 (SEARCH-1906)
*
* This test aims to test that a document with more than one (not indexed) ContentPropertyValue
* is not removed from the index after any update. The fix prevents an NPE during an update because the
* not indexed ContentPropertyValue is searched and not found in the cached document.
*
* The second ContentPropertyValue must not be indexed.
* @throws Exception
*/
@Test
public void testDataWithMultipleContentValueIsIndexedAfterUpdateTest() throws Exception
{
putHandleDefaults();
AclChangeSet aclChangeSet = getAclChangeSet(1, 1);
Acl acl = getAcl(aclChangeSet);
// Arbitrary acl data.
AclReaders aclReaders = getAclReaders(aclChangeSet, acl, list("joel"), list("phil"), null);
indexAclChangeSet(aclChangeSet,
list(acl),
list(aclReaders));
Transaction txn = getTransaction(0, 1);
Node fileNode = getNode(txn, acl, Node.SolrApiNodeStatus.UPDATED);
NodeMetaData fileMetaData = getNodeMetaData(fileNode, txn, acl, "mike", null, false);
String author = "Mario";
// Here we set the PROP_TITLE as a ContentPropertyValue.
// The value is not indexed because the related indexedField is not found (it does not exist).
fileMetaData.getProperties()
.put(ContentModel.PROP_TITLE, new ContentPropertyValue(Locale.CANADA, 100, "UTF8", "txt", 10l));
fileMetaData.getProperties().put(ContentModel.PROP_AUTHOR, new StringPropertyValue(author));
indexTransaction(txn,
list(fileNode),
list(fileMetaData));
// Check the document is correctly indexed
waitForDocCount(new TermQuery(new Term(authorField, author)), 1, MAX_WAIT_TIME);
// Update the author.
Transaction txn1 = getTransaction(0, 1);
String authorAfterUpdate = "Luigi";
fileMetaData.getProperties().put(ContentModel.PROP_AUTHOR, new StringPropertyValue(authorAfterUpdate));
indexTransaction(txn1,
list(fileNode),
list(fileMetaData));
// Check the document is still indexed after the update.
waitForDocCount(new TermQuery(new Term(authorField, authorAfterUpdate)), 1, MAX_WAIT_TIME);
}
}
search-services/alfresco-search/src/test/java/org/alfresco/solr/tracker/DistributedAclIdAlfrescoSolrTrackerIT.java
+1 -1
View File
@@ -67,7 +67,7 @@ public class DistributedAclIdAlfrescoSolrTrackerIT extends AbstractAlfrescoDistr
}
@AfterClass
private static void destroyData() throws Throwable
private static void destroyData()
{
dismissSolrServers();
}
search-services/alfresco-search/src/test/java/org/alfresco/solr/tracker/DistributedAlfrescoSolrJsonIT.java
+1 -1
View File
@@ -59,7 +59,7 @@ public class DistributedAlfrescoSolrJsonIT extends AbstractAlfrescoDistributedIT
}
@AfterClass
private static void destroyData() throws Throwable
private static void destroyData()
{
dismissSolrServers();
}
search-services/alfresco-search/src/test/java/org/alfresco/solr/tracker/DistributedAlfrescoSolrTrackerIT.java
+1 -1
View File
@@ -66,7 +66,7 @@ public class DistributedAlfrescoSolrTrackerIT extends AbstractAlfrescoDistribute
}
@AfterClass
private static void destroyData() throws Throwable
private static void destroyData()
{
dismissSolrServers();
}
search-services/alfresco-search/src/test/java/org/alfresco/solr/tracker/DistributedAlfrescoSolrTrackerRaceIT.java
+1 -1
View File
@@ -64,7 +64,7 @@ public class DistributedAlfrescoSolrTrackerRaceIT extends AbstractAlfrescoDistri
}
@AfterClass
private static void destroyData() throws Throwable
private static void destroyData()
{
dismissSolrServers();
}
search-services/alfresco-search/src/test/java/org/alfresco/solr/tracker/DistributedCascadeTrackerIT.java → search-services/alfresco-search/src/test/java/org/alfresco/solr/tracker/DistributedCascadeIT.java
+1 -1
View File
@@ -55,7 +55,7 @@ import java.util.Properties;
*/
@SolrTestCaseJ4.SuppressSSL
@LuceneTestCase.SuppressCodecs({"Appending","Lucene3x","Lucene40","Lucene41","Lucene42","Lucene43", "Lucene44", "Lucene45","Lucene46","Lucene47","Lucene48","Lucene49"})
public class DistributedCascadeTrackerIT extends AbstractAlfrescoDistributedIT
public class DistributedCascadeIT extends AbstractAlfrescoDistributedIT
{
private Node parentFolder;
private NodeMetaData parentFolderMetadata;
search-services/alfresco-search/src/test/java/org/alfresco/solr/tracker/DistributedDateQuarterAlfrescoSolrTrackerIT.java
+1 -1
View File
@@ -43,7 +43,7 @@ public class DistributedDateQuarterAlfrescoSolrTrackerIT extends DistributedDate
}
@AfterClass
private static void destroyData() throws Throwable
private static void destroyData()
{
dismissSolrServers();
}
search-services/alfresco-search/src/test/java/org/alfresco/solr/tracker/DistributedDateSplitYearAlfrescoSolrTrackerIT.java
+1 -1
View File
@@ -42,7 +42,7 @@ public class DistributedDateSplitYearAlfrescoSolrTrackerIT extends DistributedDa
}
@AfterClass
private static void destroyData() throws Throwable
private static void destroyData()
{
dismissSolrServers();
}
search-services/alfresco-search/src/test/java/org/alfresco/solr/tracker/DistributedDbidRangeAlfrescoSolrTrackerIT.java
+1 -1
View File
@@ -62,7 +62,7 @@ public class DistributedDbidRangeAlfrescoSolrTrackerIT extends AbstractAlfrescoD
}
@AfterClass
private static void destroyData() throws Throwable
private static void destroyData()
{
dismissSolrServers();
}
search-services/alfresco-search/src/test/java/org/alfresco/solr/tracker/DistributedExpandDbidRangeAlfrescoSolrTrackerIT.java
+1 -1
View File
@@ -65,7 +65,7 @@ public class DistributedExpandDbidRangeAlfrescoSolrTrackerIT extends AbstractAlf
}
@AfterClass
private static void destroyData() throws Throwable
private static void destroyData()
{
dismissSolrServers();
}
search-services/alfresco-search/src/test/java/org/alfresco/solr/tracker/DistributedExplicitShardIdWithStaticPropertyRouterIT.java
+1 -1
View File
@@ -66,7 +66,7 @@ public class DistributedExplicitShardIdWithStaticPropertyRouterIT extends Abstra
}
@AfterClass
private static void destroyData() throws Throwable
private static void destroyData()
{
dismissSolrServers();
}
search-services/alfresco-search/src/test/java/org/alfresco/solr/tracker/DistributedExplicitShardRoutingTrackerIT.java
+1 -1
View File
@@ -71,7 +71,7 @@ public class DistributedExplicitShardRoutingTrackerIT extends AbstractAlfrescoDi
}
@AfterClass
private static void destroyData() throws Throwable
private static void destroyData()
{
dismissSolrServers();
}
search-services/alfresco-search/src/test/java/org/alfresco/solr/tracker/DistributedPropertyBasedAlfrescoSolrTrackerIT.java
+1 -1
View File
@@ -82,7 +82,7 @@ public class DistributedPropertyBasedAlfrescoSolrTrackerIT extends AbstractAlfre
}
@AfterClass
private static void destroyData() throws Throwable
private static void destroyData()
{
dismissSolrServers();
}
search-services/alfresco-search/src/test/java/org/alfresco/solr/tracker/MetadataTrackerTest.java
+1 -1
View File
@@ -120,7 +120,7 @@ public class MetadataTrackerTest
this.metadataTracker.doTrack();
InOrder inOrder = inOrder(srv);
inOrder.verify(srv).indexNodes(nodes, true, false);
inOrder.verify(srv).indexNodes(nodes, true);
inOrder.verify(srv).indexTransaction(tx, true);
inOrder.verify(srv).commit();
}
search-services/alfresco-search/src/test/java/org/alfresco/solr/tracker/SolrTrackerSchedulerTest.java
-1
View File
@@ -78,7 +78,6 @@ public class SolrTrackerSchedulerTest
props.put("shard.method", "SHARD_METHOD_DBID");
props.put("alfresco.template", "");
props.put("alfresco.index.transformContent", "true");
props.put("alfresco.version", "5.0.0");
}
@After
search-services/alfresco-search/src/test/java/org/alfresco/solr/transformer/CachedDocTransformerIT.java
+1 -1
View File
@@ -70,7 +70,7 @@ public class CachedDocTransformerIT extends AbstractAlfrescoDistributedIT
}
@AfterClass
private static void destroyData() throws Throwable
private static void destroyData()
{
dismissSolrServers();
}
search-services/alfresco-solrclient-lib/pom.xml
+5 -5
View File
@@ -22,8 +22,8 @@
</distributionManagement>
<properties>
<dependency.alfresco-data-model.version>8.53</dependency.alfresco-data-model.version>
<dependency.jackson.version>2.10.1</dependency.jackson.version>
<dependency.alfresco-data-model.version>8.87</dependency.alfresco-data-model.version>
<dependency.jackson.version>2.10.2</dependency.jackson.version>
</properties>
<dependencies>
@@ -61,19 +61,19 @@
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<version>4.13</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<version>3.1.0</version>
<version>3.2.4</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.29</version>
<version>1.7.30</version>
</dependency>
</dependencies>
search-services/alfresco-solrclient-lib/src/main/java/org/alfresco/solr/client/SOLRAPIClient.java
+134 -12
View File
@@ -103,7 +103,9 @@ public class SOLRAPIClient
private static final String GET_NODES_URL = "api/solr/nodes";
private static final String GET_CONTENT = "api/solr/textContent";
private static final String GET_MODEL = "api/solr/model";
private static final String GET_MODELS_DIFF = "api/solr/modelsdiff";
private static final String GET_MODELS_DIFF = "api/solr/modelsdiff";
private static final String GET_NEXT_TX_COMMIT_TIME = "api/solr/nextTransaction";
private static final String GET_TX_INTERVAL_COMMIT_TIME = "api/solr/transactionInterval";
private static final String CHECKSUM_HEADER = "XAlfresco-modelChecksum";
@@ -111,17 +113,34 @@ public class SOLRAPIClient
private SOLRDeserializer deserializer;
private DictionaryService dictionaryService;
private JsonFactory jsonFactory;
private NamespaceDAO namespaceDAO;
private NamespaceDAO namespaceDAO;
/**
* This option enables ("Accept-Encoding": "gzip") header for compression
* in GET_CONTENT requests. Additional configuration is required in
* Alfresco Repository Tomcat Connector or HTTP Web Proxy to deal
* with compressed requests.
*/
private boolean compression;
public SOLRAPIClient(AlfrescoHttpClient repositoryHttpClient,
DictionaryService dictionaryService,
NamespaceDAO namespaceDAO)
{
this(repositoryHttpClient, dictionaryService, namespaceDAO, false);
}
public SOLRAPIClient(AlfrescoHttpClient repositoryHttpClient,
DictionaryService dictionaryService,
NamespaceDAO namespaceDAO)
NamespaceDAO namespaceDAO,
boolean compression)
{
this.repositoryHttpClient = repositoryHttpClient;
this.dictionaryService = dictionaryService;
this.namespaceDAO = namespaceDAO;
this.deserializer = new SOLRDeserializer(namespaceDAO);
this.jsonFactory = new JsonFactory();
this.jsonFactory = new JsonFactory();
this.compression = compression;
}
/**
@@ -1118,20 +1137,24 @@ public class SOLRAPIClient
GetRequest req = new GetRequest(url.toString());
Map<String, String> headers = new HashMap<>();
if(modifiedSince != null)
{
Map<String, String> headers = new HashMap<String, String>(1, 1.0f);
headers.put("If-Modified-Since", String.valueOf(DateUtil.formatDate(new Date(modifiedSince))));
req.setHeaders(headers);
}
if (compression)
{
headers.put("Accept-Encoding", "gzip");
}
req.setHeaders(headers);
Response response = repositoryHttpClient.sendRequest(req);
if(response.getStatus() != Status.STATUS_NOT_MODIFIED && response.getStatus() != Status.STATUS_NO_CONTENT && response.getStatus() != Status.STATUS_OK)
{
throw new AlfrescoRuntimeException("GetTextContentResponse return status is " + response.getStatus());
}
}
return new GetTextContentResponse(response);
}
@@ -1228,7 +1251,99 @@ public class SOLRAPIClient
}
return diffs;
}
}
/**
* Returns the minimum and the maximum commit time for transactions in a node id range.
*
* @param coreName alfresco, archive
* @param fromCommitTime initial transaction commit time
* @return Time of the next transaction
* @throws IOException
* @throws AuthenticationException
* @throws NoSuchMethodException
*/
public Long getNextTxCommitTime(String coreName, Long fromCommitTime) throws AuthenticationException, IOException, NoSuchMethodException
{
StringBuilder url = new StringBuilder(GET_NEXT_TX_COMMIT_TIME);
url.append("?").append("fromCommitTime").append("=").append(fromCommitTime);
GetRequest get = new GetRequest(url.toString());
Response response = null;
JSONObject json = null;
try
{
response = repositoryHttpClient.sendRequest(get);
if (response.getStatus() != HttpStatus.SC_OK)
{
throw new NoSuchMethodException(coreName + " - GetNextTxCommitTime return status is "
+ response.getStatus() + " when invoking " + url);
}
Reader reader = new BufferedReader(new InputStreamReader(response.getContentAsStream(), "UTF-8"));
json = new JSONObject(new JSONTokener(reader));
}
finally
{
if (response != null)
{
response.release();
}
}
if (log.isDebugEnabled())
{
log.debug(json.toString());
}
return Long.parseLong(json.get("nextTransactionCommitTimeMs").toString());
}
/**
* Returns the minimum and the maximum commit time for transactions in a node id range.
*
* @param coreName alfresco, archive
* @param fromNodeId Id of the initial node
* @param toNodeId Id of the final node
* @return Time of the first transaction, time of the last transaction
* @throws IOException
* @throws AuthenticationException
* @throws NoSuchMethodException
*/
public Pair<Long, Long> getTxIntervalCommitTime(String coreName, Long fromNodeId, Long toNodeId)
throws AuthenticationException, IOException, NoSuchMethodException
{
StringBuilder url = new StringBuilder(GET_TX_INTERVAL_COMMIT_TIME);
url.append("?").append("fromNodeId").append("=").append(fromNodeId);
url.append("&").append("toNodeId").append("=").append(toNodeId);
GetRequest get = new GetRequest(url.toString());
Response response = null;
JSONObject json = null;
try
{
response = repositoryHttpClient.sendRequest(get);
if (response.getStatus() != HttpStatus.SC_OK)
{
throw new NoSuchMethodException(coreName + " - GetTxIntervalCommitTime return status is "
+ response.getStatus() + " when invoking " + url);
}
Reader reader = new BufferedReader(new InputStreamReader(response.getContentAsStream(), "UTF-8"));
json = new JSONObject(new JSONTokener(reader));
}
finally
{
if (response != null)
{
response.release();
}
}
if (log.isDebugEnabled())
{
log.debug(json.toString());
}
return new Pair<Long, Long>(Long.parseLong(json.get("minTransactionCommitTimeMs").toString()),
Long.parseLong(json.get("maxTransactionCommitTimeMs").toString()));
}
/*
* type conversions from serialized JSON values to SOLR-consumable objects
@@ -1481,7 +1596,8 @@ public class SOLRAPIClient
private SolrApiContentStatus status;
private String transformException;
private String transformStatusStr;
private Long transformDuration;
private Long transformDuration;
private String contentEncoding;
public GetTextContentResponse(Response response) throws IOException
{
@@ -1491,7 +1607,8 @@ public class SOLRAPIClient
this.transformStatusStr = response.getHeader("X-Alfresco-transformStatus");
this.transformException = response.getHeader("X-Alfresco-transformException");
String tmp = response.getHeader("X-Alfresco-transformDuration");
this.transformDuration = (tmp != null ? Long.valueOf(tmp) : null);
this.transformDuration = (tmp != null ? Long.valueOf(tmp) : null);
this.contentEncoding = response.getHeader("Content-Encoding");
setStatus();
}
@@ -1557,6 +1674,11 @@ public class SOLRAPIClient
public Long getTransformDuration()
{
return transformDuration;
}
public String getContentEncoding()
{
return contentEncoding;
}
}
search-services/alfresco-solrclient-lib/src/main/java/org/alfresco/solr/client/SOLRAPIClientFactory.java
+2 -1
View File
@@ -143,6 +143,7 @@ public class SOLRAPIClientFactory
alfrescoHost = props.getProperty("alfresco.host", "localhost");
alfrescoPort = Integer.parseInt(props.getProperty("alfresco.port", "8080"));
alfrescoPortSSL = Integer.parseInt(props.getProperty("alfresco.port.ssl", "8443"));
boolean compression = Boolean.parseBoolean(props.getProperty("solr.request.content.compress", "false"));
SOLRAPIClient client = getCachedClient(alfrescoHost, alfrescoPort, alfrescoPortSSL);
if (client == null)
@@ -171,7 +172,7 @@ public class SOLRAPIClientFactory
maxHostConnections = Integer.parseInt(props.getProperty("alfresco.maxHostConnections", "40"));
socketTimeout = Integer.parseInt(props.getProperty("alfresco.socketTimeout", "60000"));
client = new SOLRAPIClient(getRepoClient(keyResourceLoader), dictionaryService, namespaceDAO);
client = new SOLRAPIClient(getRepoClient(keyResourceLoader), dictionaryService, namespaceDAO, compression);
setCachedClient(alfrescoHost, alfrescoPort, alfrescoPortSSL, client);
}
search-services/packaging/pom.xml
+6 -3
View File
@@ -33,7 +33,7 @@
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<version>4.13</version>
<scope>test</scope>
</dependency>
</dependencies>
@@ -60,7 +60,7 @@
<plugin>
<groupId>com.googlecode.maven-download-plugin</groupId>
<artifactId>download-maven-plugin</artifactId>
<version>1.3.0</version><!-- 1.4.0 gives class not found issues with "CachedFileEntry": https://github.com/maven-download-plugin/maven-download-plugin/issues/80 -->
<version>1.5.0</version>
<executions>
<execution>
<id>unpack-solr-war</id>
@@ -111,7 +111,8 @@
<version>${project.version}</version>
<classifier>libs</classifier>
<outputDirectory>${project.build.directory}/solr-libs</outputDirectory>
<excludes>**/jackson-dataformat-smile-*.jar,**/asm-3.3.1.jar,**/jackson-core-asl-*.jar,**/jackson-mapper-asl-*.jar,**/dom4j-1.6.1.jar,**/annotations-1.0.0.jar</excludes>
<excludes>**/jackson-dataformat-smile-*.jar,**/asm-3.3.1.jar,**/jackson-core-asl-*.jar,**/jackson-mapper-asl-*.jar,**/dom4j-1.6.1.jar,
**/annotations-1.0.0.jar,**/spring-context-support-*.jar,**/spring-web-*.jar,**/woodstox-core-asl-4.4.1.jar</excludes>
</artifactItem>
</artifactItems>
</configuration>
@@ -157,7 +158,9 @@
<delete file="${project.build.directory}/solr-${solr.version}/server/solr-webapp/webapp/WEB-INF/lib/jackson-core-asl-1.9.13.jar" />
<delete file="${project.build.directory}/solr-${solr.version}/server/solr-webapp/webapp/WEB-INF/lib/jackson-mapper-asl-1.9.13.jar" />
<delete file="${project.build.directory}/solr-${solr.version}/server/solr-webapp/webapp/WEB-INF/lib/dom4j-1.6.1.jar" />
<delete file="${project.build.directory}/solr-${solr.version}/server/solr-webapp/webapp/WEB-INF/lib/woodstox-core-asl-4.4.1.jar" />
</target>
</configuration>
</execution>
<!-- Set the default solr home directory to solrhome -->
search-services/packaging/src/docker/6.x/docker-compose.yml
+25
View File
@@ -37,6 +37,10 @@ services:
search:
image: quay.io/alfresco/search-services:${SEARCH_TAG}
environment:
#Replication properties
- REPLICATION_TYPE=master
#- REPLICATION_AFTER=commit,startup- SOLR_ALFRESCO_HOST=alfresco
#- REPLICATION_CONFIG_FILES=schema.xml,stopwords.txt- SOLR_ALFRESCO_PORT=8080
#Solr needs to know how to register itself with Alfresco
- SOLR_ALFRESCO_HOST=alfresco
- SOLR_ALFRESCO_PORT=8080
@@ -51,6 +55,27 @@ services:
- ENABLE_SPELLCHECK=${SEARCH_ENABLE_SPELLCHECK}
ports:
- 8083:8983 #Browser port
#search_slave:
# image: quay.io/alfresco/search-services:${SEARCH_TAG}
# environment:
# #Replication properties
# - REPLICATION_TYPE=slave
# - REPLICATION_MASTER_HOST=search
# - REPLICATION_MASTER_PORT=8983
# #- REPLICATION_MASTER_PROTOCOL=http
# #- REPLICATION_CORE_NAME=alfresco
# #- REPLICATION_POLL_INTERVAL=00:00:60
# #Solr needs to know how to register itself with Alfresco
# - SOLR_ALFRESCO_HOST=alfresco
# - SOLR_ALFRESCO_PORT=8080
# #Alfresco needs to know how to call solr
# - SOLR_SOLR_HOST=search
# - SOLR_SOLR_PORT=8983
# #Create the default alfresco and archive cores
# - SOLR_CREATE_ALFRESCO_DEFAULTS=alfresco,archive
# ports:
# - 8084:8983 #Browser port
activemq:
image: alfresco/alfresco-activemq:5.15.6
ports:
search-services/packaging/src/docker/Dockerfile
+1 -1
View File
@@ -1,6 +1,6 @@
# Alfresco Search Services ${project.version} Docker Image
FROM alfresco/alfresco-base-java:11.0.1-openjdk-centos-7-6784d76a7b81
FROM alfresco/alfresco-base-java:11.0.1-openjdk-centos-7-7a6031154417
LABEL creator="Gethin James" maintainer="Alfresco Search Services Team"
ENV DIST_DIR /opt/alfresco-search-services
search-services/packaging/src/docker/search_config_setup.sh
+65 -2
View File
@@ -1,5 +1,69 @@
#!/bin/bash
set -e
# By default its going to deploy "Master" setup configuration with "REPLICATION_TYPE=master".
# Slave replica service can be enabled using "REPLICATION_TYPE=slave" environment value.
RERANK_TEMPLATE_PATH=$PWD/solrhome/templates/rerank/conf
NORERANK_TEMPLATE_PATH=$PWD/solrhome/templates/noRerank/conf
SOLR_RERANK_CONFIG_FILE=$RERANK_TEMPLATE_PATH/solrconfig.xml
SOLR_NORERANK_CONFIG_FILE=$NORERANK_TEMPLATE_PATH/solrconfig.xml
SOLR_RERANK_CORE_FILE=$RERANK_TEMPLATE_PATH/solrcore.properties
SOLR_NORERANK_CORE_FILE=$NORERANK_TEMPLATE_PATH/solrcore.properties
if [[ $REPLICATION_TYPE == "master" ]]; then
findStringMaster='<requestHandler name="\/replication" class="org\.alfresco\.solr\.handler\.AlfrescoReplicationHandler">'
replaceStringMaster="\n\t<lst name=\"master\"> \n"
if [[ $REPLICATION_AFTER == "" ]]; then
REPLICATION_AFTER=commit,startup
fi
if [[ $REPLICATION_CONFIG_FILES == "" ]]; then
REPLICATION_CONFIG_FILES=schema.xml,stopwords.txt
fi
for i in $(echo $REPLICATION_AFTER | sed "s/,/ /g")
do
replaceStringMaster+="\t\t<str name=\"replicateAfter\">"$i"<\/str> \n"
done
if [[ ! -z "$REPLICATION_CONFIG_FILES" ]]; then
replaceStringMaster+="\t\t<str name=\"confFiles\">$REPLICATION_CONFIG_FILES<\/str> \n"
fi
replaceStringMaster+="\t<\/lst>"
sed -i "s/$findStringMaster/$findStringMaster$replaceStringMaster/g" $SOLR_RERANK_CONFIG_FILE $SOLR_NORERANK_CONFIG_FILE
sed -i "s/enable.alfresco.tracking=true/enable.alfresco.tracking=true\nenable.master=true\nenable.slave=false/g" $SOLR_RERANK_CORE_FILE $SOLR_NORERANK_CORE_FILE
fi
if [[ $REPLICATION_TYPE == "slave" ]]; then
if [[ $REPLICATION_MASTER_PROTOCOL != https ]]; then
REPLICATION_MASTER_PROTOCOL=http
fi
if [[ $REPLICATION_MASTER_HOST == "" ]]; then
REPLICATION_MASTER_HOST=localhost
fi
if [[ $REPLICATION_MASTER_PORT == "" ]]; then
REPLICATION_MASTER_PORT=8083
fi
if [[ $REPLICATION_POLL_INTERVAL == "" ]]; then
REPLICATION_POLL_INTERVAL=00:00:30
fi
sed -i 's/<requestHandler name="\/replication" class="org\.alfresco\.solr\.handler\.AlfrescoReplicationHandler">/<requestHandler name="\/replication" class="org\.alfresco\.solr\.handler\.AlfrescoReplicationHandler">\
<lst name="slave">\
<str name="masterUrl">'$REPLICATION_MASTER_PROTOCOL':\/\/'$REPLICATION_MASTER_HOST':'$REPLICATION_MASTER_PORT'\/solr\/${solr.core.name}<\/str>\
<str name="pollInterval">'$REPLICATION_POLL_INTERVAL'<\/str>\
<\/lst>/g' $SOLR_RERANK_CONFIG_FILE $SOLR_NORERANK_CONFIG_FILE
sed -i "s/enable.alfresco.tracking=true/enable.alfresco.tracking=false\nenable.master=false\nenable.slave=true/g" $SOLR_RERANK_CORE_FILE $SOLR_NORERANK_CORE_FILE
fi
SOLR_IN_FILE=$PWD/solr.in.sh
@@ -21,8 +85,7 @@ fi
# By default Docker Image is using TLS Mutual Authentication (SSL) for communications with Repository
# Plain HTTP can be enabled by setting ALFRESCO_SECURE_COMMS to 'none'
if [[ "none" == "$ALFRESCO_SECURE_COMMS" ]]; then
sed -i 's/alfresco.secureComms=https/alfresco.secureComms=none/' ${PWD}/solrhome/templates/rerank/conf/solrcore.properties
sed -i 's/alfresco.secureComms=https/alfresco.secureComms=none/' ${PWD}/solrhome/templates/noRerank/conf/solrcore.properties
sed -i 's/alfresco.secureComms=https/alfresco.secureComms=none/' $SOLR_RERANK_CORE_FILE $SOLR_NORERANK_CORE_FILE
# Apply also the setting to existing SOLR cores property files when existing
if [[ -f ${PWD}/solrhome/alfresco/conf/solrcore.properties ]]; then
sed -i 's/alfresco.secureComms=https/alfresco.secureComms=none/' ${PWD}/solrhome/alfresco/conf/solrcore.properties
search-services/packaging/src/main/resources/licenses/3rd-party/Apache-like-XPP.txt → search-services/packaging/src/main/resources/licenses/3rd-party/BSDlike-XPP.txt
Vendored
+58 -58
View File
@@ -1,58 +1,58 @@
LICENSE FOR THE Extreme! Lab PullParser
------------------------------------------------------------------------
Copyright © 2002 The Trustees of Indiana University.
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
1) All redistributions of source code must retain the above
copyright notice, the list of authors in the original source
code, this list of conditions and the disclaimer listed in this
license;
2) All redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the disclaimer
listed in this license in the documentation and/or other
materials provided with the distribution;
3) Any documentation included with all redistributions must include
the following acknowledgement:
"This product includes software developed by the Indiana
University Extreme! Lab. For further information please visit
http://www.extreme.indiana.edu/"
Alternatively, this acknowledgment may appear in the software
itself, and wherever such third-party acknowledgments normally
appear.
4) The name "Indiana Univeristy" and "Indiana Univeristy
Extreme! Lab" shall not be used to endorse or promote
products derived from this software without prior written
permission from Indiana University. For written permission,
please contact http://www.extreme.indiana.edu/.
5) Products derived from this software may not use "Indiana
Univeristy" name nor may "Indiana Univeristy" appear in their name,
without prior written permission of the Indiana University.
Indiana University provides no reassurances that the source code
provided does not infringe the patent or any other intellectual
property rights of any other entity. Indiana University disclaims any
liability to any recipient for claims brought by any other entity
based on infringement of intellectual property rights or otherwise.
LICENSEE UNDERSTANDS THAT SOFTWARE IS PROVIDED "AS IS" FOR WHICH
NO WARRANTIES AS TO CAPABILITIES OR ACCURACY ARE MADE. INDIANA
UNIVERSITY GIVES NO WARRANTIES AND MAKES NO REPRESENTATION THAT
SOFTWARE IS FREE OF INFRINGEMENT OF THIRD PARTY PATENT, COPYRIGHT, OR
OTHER PROPRIETARY RIGHTS.  INDIANA UNIVERSITY MAKES NO WARRANTIES THAT
SOFTWARE IS FREE FROM "BUGS", "VIRUSES", "TROJAN HORSES", "TRAP
DOORS", "WORMS", OR OTHER HARMFUL CODE. LICENSEE ASSUMES THE ENTIRE
RISK AS TO THE PERFORMANCE OF SOFTWARE AND/OR ASSOCIATED MATERIALS,
AND TO THE PERFORMANCE AND VALIDITY OF INFORMATION GENERATED USING
SOFTWARE.
LICENSE FOR THE Extreme! Lab
------------------------------------------------------------------------
Copyright © 2003 The Trustees of Indiana University.
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
1) All redistributions of source code must retain the above
copyright notice, the list of authors in the original source
code, this list of conditions and the disclaimer listed in this
license;
2) All redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the disclaimer
listed in this license in the documentation and/or other
materials provided with the distribution;
3) Any documentation included with all redistributions must include
the following acknowledgement:
"This product includes software developed by the Indiana
University Extreme! Lab. For further information please visit
http://www.extreme.indiana.edu/"
Alternatively, this acknowledgment may appear in the software
itself, and wherever such third-party acknowledgments normally
appear.
4) The name "Indiana University" and "Indiana University
Extreme! Lab" shall not be used to endorse or promote
products derived from this software without prior written
permission from Indiana University. For written permission,
please contact http://www.extreme.indiana.edu/.
5) Products derived from this software may not use "Indiana
University" name nor may "Indiana University" appear in their name,
without prior written permission of the Indiana University.
Indiana University provides no reassurances that the source code
provided does not infringe the patent or any other intellectual
property rights of any other entity. Indiana University disclaims any
liability to any recipient for claims brought by any other entity
based on infringement of intellectual property rights or otherwise.
LICENSEE UNDERSTANDS THAT SOFTWARE IS PROVIDED "AS IS" FOR WHICH
NO WARRANTIES AS TO CAPABILITIES OR ACCURACY ARE MADE. INDIANA
UNIVERSITY GIVES NO WARRANTIES AND MAKES NO REPRESENTATION THAT
SOFTWARE IS FREE OF INFRINGEMENT OF THIRD PARTY PATENT, COPYRIGHT, OR
OTHER PROPRIETARY RIGHTS.  INDIANA UNIVERSITY MAKES NO WARRANTIES THAT
SOFTWARE IS FREE FROM "BUGS", "VIRUSES", "TROJAN HORSES", "TRAP
DOORS", "WORMS", OR OTHER HARMFUL CODE. LICENSEE ASSUMES THE ENTIRE
RISK AS TO THE PERFORMANCE OF SOFTWARE AND/OR ASSOCIATED MATERIALS,
AND TO THE PERFORMANCE AND VALIDITY OF INFORMATION GENERATED USING
SOFTWARE.
search-services/packaging/src/main/resources/licenses/notice.txt
+57 -62
View File
@@ -22,8 +22,8 @@ antlr-3.5.2.jar http://www.antlr.org/
jaxen-1.2.0.jar http://www.cafeconleche.org/jaxen/
=== Apache variant License ===
xpp3-1.1.3_8.jar http://www.extreme.indiana.edu/xgws/xsoap/xpp/
=== BSD variant License ===
xpp3-1.1.4c.jar http://www.extreme.indiana.edu/dist/java-repository/xpp3/licenses/LICENSE.txt
=== JSON ===
@@ -32,43 +32,38 @@ json-20160212.jar http://code.google.com/p/json-simple/
=== Apache 2.0 ===
xml-resolver-1.2.jar https://github.com/FasterXML/jackson
neethi-3.0.3.jar http://ws.apache.org/commons/neethi/
neethi-3.1.1.jar http://ws.apache.org/commons/neethi/
commons-logging-1.2.jar http://jakarta.apache.org/commons/
commons-lang3-3.9.jar http://jakarta.apache.org/commons/
mybatis-3.3.0.jar http://www.mybatis.org/
chemistry-opencmis-commons-impl-1.1.0.jar http://chemistry.apache.org/
chemistry-opencmis-commons-api-1.1.0.jar http://chemistry.apache.org/
xmlschema-core-2.2.1.jar http://ws.apache.org/commons/XmlSchema/
xmlschema-core-2.2.5.jar http://ws.apache.org/commons/XmlSchema/
HikariCP-java7-2.4.13.jar https://github.com/brettwooldridge/HikariCP
cxf-core-3.0.12.jar https://cxf.apache.org/
cxf-rt-bindings-soap-3.0.12.jar https://cxf.apache.org/
cxf-rt-bindings-xml-3.0.12.jar https://cxf.apache.org/
cxf-rt-databinding-jaxb-3.0.12.jar https://cxf.apache.org/
cxf-rt-frontend-jaxws-3.0.12.jar https://cxf.apache.org/
cxf-rt-frontend-simple-3.0.12.jar https://cxf.apache.org/
cxf-rt-transports-http-3.0.12.jar https://cxf.apache.org/
cxf-rt-ws-addr-3.0.12.jar https://cxf.apache.org/
cxf-rt-ws-policy-3.0.12.jar https://cxf.apache.org/
cxf-rt-wsdl-3.0.12.jar https://cxf.apache.org/
mybatis-spring-1.2.5.jar http://www.mybatis.org/
cxf-core-3.2.12.jar https://cxf.apache.org/
cxf-rt-bindings-soap-3.2.12.jar https://cxf.apache.org/
cxf-rt-bindings-xml-3.2.12.jar https://cxf.apache.org/
cxf-rt-databinding-jaxb-3.2.12.jar https://cxf.apache.org/
cxf-rt-frontend-jaxws-3.2.12.jar https://cxf.apache.org/
cxf-rt-frontend-simple-3.2.12.jar https://cxf.apache.org/
cxf-rt-transports-http-3.2.12.jar https://cxf.apache.org/
cxf-rt-ws-addr-3.2.12.jar https://cxf.apache.org/
cxf-rt-ws-policy-3.2.12.jar https://cxf.apache.org/
cxf-rt-wsdl-3.2.12.jar https://cxf.apache.org/
chemistry-opencmis-server-support-1.0.0.jar http://chemistry.apache.org/
chemistry-opencmis-server-bindings-1.0.0.jar http://chemistry.apache.org/
quartz-2.3.1.jar http://quartz-scheduler.org/
jackson-core-2.10.1.jar https://github.com/FasterXML/jackson
jackson-annotations-2.10.1.jar https://github.com/FasterXML/jackson
jackson-databind-2.10.1.jar https://github.com/FasterXML/jackson
quartz-2.3.2.jar http://quartz-scheduler.org/
jackson-core-2.10.2.jar https://github.com/FasterXML/jackson
jackson-annotations-2.10.2.jar https://github.com/FasterXML/jackson
jackson-databind-2.10.2.jar https://github.com/FasterXML/jackson
commons-httpclient-3.1-HTTPCLIENT-1265.jar http://jakarta.apache.org/commons/
spring-aop-5.1.8.RELEASE.jar http://projects.spring.io/spring-framework/
spring-beans-5.1.8.RELEASE.jar http://projects.spring.io/spring-framework/
spring-context-5.1.8.RELEASE.jar http://projects.spring.io/spring-framework/
spring-context-support-5.1.8.RELEASE.jar http://projects.spring.io/spring-framework/
spring-core-5.1.8.RELEASE.jar http://projects.spring.io/spring-framework/
spring-expression-5.1.8.RELEASE.jar http://projects.spring.io/spring-framework/
spring-jcl-5.1.8.RELEASE.jar http://projects.spring.io/spring-framework/
spring-jdbc-5.1.8.RELEASE.jar http://projects.spring.io/spring-framework/
spring-orm-5.1.8.RELEASE.jar http://projects.spring.io/spring-framework/
spring-tx-5.1.8.RELEASE.jar http://projects.spring.io/spring-framework/
spring-web-5.1.8.RELEASE.jar http://projects.spring.io/spring-framework/
spring-aop-5.2.3.RELEASE.jar http://projects.spring.io/spring-framework/
spring-beans-5.2.3.RELEASE.jar http://projects.spring.io/spring-framework/
spring-context-5.2.3.RELEASE.jar http://projects.spring.io/spring-framework/
spring-core-5.2.3.RELEASE.jar http://projects.spring.io/spring-framework/
spring-expression-5.2.3.RELEASE.jar http://projects.spring.io/spring-framework/
spring-jdbc-5.2.3.RELEASE.jar http://projects.spring.io/spring-framework/
spring-orm-5.2.3.RELEASE.jar http://projects.spring.io/spring-framework/
spring-tx-5.2.3.RELEASE.jar http://projects.spring.io/spring-framework/
xercesImpl-2.12.0-alfresco-patched-20191004.jar http://xerces.apache.org/xerces2-j
guessencoding-1.4.jar http://docs.codehaus.org/display/GUESSENC/
xml-apis-1.4.01.jar https://github.com/FasterXML/jackson
@@ -87,6 +82,7 @@ jetty-servlets-9.3.27.v20190418.jar https://www.eclipse.org/jetty/licenses.html
jetty-util-9.3.27.v20190418.jar https://www.eclipse.org/jetty/licenses.html
jetty-webapp-9.3.27.v20190418.jar https://www.eclipse.org/jetty/licenses.html
jetty-xml-9.3.27.v20190418.jar https://www.eclipse.org/jetty/licenses.html
woodstox-core-5.0.3.jar https://github.com/FasterXML/woodstox
=== CDDL 1.0 ===
@@ -115,8 +111,8 @@ asm-commons-5.1.jar
aspectjrt-1.8.0.jar
attributes-binder-1.3.1.jar
avatica-core-1.9.0.jar
bcmail-jdk15-1.45.jar
bcprov-jdk15-1.45.jar
bcmail-jdk15on-1.47.jar
bcprov-jdk15on-1.47.jar
boilerpipe-1.1.0.jar
caffeine-2.4.0.jar
calcite-core-1.11.0.jar
@@ -131,7 +127,7 @@ commons-compiler-2.7.6.jar
commons-compress-1.14.jar
commons-configuration-1.6.jar
commons-exec-1.3.jar
commons-fileupload-1.3.2.jar
commons-fileupload-1.3.3.jar
commons-io-2.5.jar
commons-lang-2.6.jar
commons-math3-3.4.1.jar
@@ -168,28 +164,28 @@ jul-to-slf4j-1.7.7.jar
juniversalchardet-1.0.3.jar
langdetect-1.1-20120112.jar
log4j-1.2.17.jar
lucene-analyzers-common-6.6.5-patched.1.jar
lucene-analyzers-icu-6.6.5-patched.1.jar
lucene-analyzers-kuromoji-6.6.5-patched.1.jar
lucene-analyzers-morfologik-6.6.5-patched.1.jar
lucene-analyzers-phonetic-6.6.5-patched.1.jar
lucene-analyzers-smartcn-6.6.5-patched.1.jar
lucene-analyzers-stempel-6.6.5-patched.1.jar
lucene-backward-codecs-6.6.5-patched.1.jar
lucene-classification-6.6.5-patched.1.jar
lucene-codecs-6.6.5-patched.1.jar
lucene-core-6.6.5-patched.1.jar
lucene-expressions-6.6.5-patched.1.jar
lucene-grouping-6.6.5-patched.1.jar
lucene-highlighter-6.6.5-patched.1.jar
lucene-join-6.6.5-patched.1.jar
lucene-memory-6.6.5-patched.1.jar
lucene-misc-6.6.5-patched.1.jar
lucene-queries-6.6.5-patched.1.jar
lucene-queryparser-6.6.5-patched.1.jar
lucene-sandbox-6.6.5-patched.1.jar
lucene-spatial-extras-6.6.5-patched.1.jar
lucene-suggest-6.6.5-patched.1.jar
lucene-analyzers-common-6.6.5-patched.2.jar
lucene-analyzers-icu-6.6.5-patched.2.jar
lucene-analyzers-kuromoji-6.6.5-patched.2.jar
lucene-analyzers-morfologik-6.6.5-patched.2.jar
lucene-analyzers-phonetic-6.6.5-patched.2.jar
lucene-analyzers-smartcn-6.6.5-patched.2.jar
lucene-analyzers-stempel-6.6.5-patched.2.jar
lucene-backward-codecs-6.6.5-patched.2.jar
lucene-classification-6.6.5-patched.2.jar
lucene-codecs-6.6.5-patched.2.jar
lucene-core-6.6.5-patched.2.jar
lucene-expressions-6.6.5-patched.2.jar
lucene-grouping-6.6.5-patched.2.jar
lucene-highlighter-6.6.5-patched.2.jar
lucene-join-6.6.5-patched.2.jar
lucene-memory-6.6.5-patched.2.jar
lucene-misc-6.6.5-patched.2.jar
lucene-queries-6.6.5-patched.2.jar
lucene-queryparser-6.6.5-patched.2.jar
lucene-sandbox-6.6.5-patched.2.jar
lucene-spatial-extras-6.6.5-patched.2.jar
lucene-suggest-6.6.5-patched.2.jar
metadata-extractor-2.9.1.jar
metrics-core-3.2.2.jar
metrics-ganglia-3.2.2.jar
@@ -213,11 +209,11 @@ rome-1.5.1.jar
simple-xml-2.7.1.jar
slf4j-api-1.7.7.jar
slf4j-log4j12-1.7.7.jar
solr-analysis-extras-6.6.5-patched.1.jar
solr-clustering-6.6.5-patched.1.jar
solr-core-6.6.5-patched.1.jar
solr-langid-6.6.5-patched.1.jar
solr-solrj-6.6.5-patched.1.jar
solr-analysis-extras-6.6.5-patched.2.jar
solr-clustering-6.6.5-patched.2.jar
solr-core-6.6.5-patched.2.jar
solr-langid-6.6.5-patched.2.jar
solr-solrj-6.6.5-patched.2.jar
spatial4j-0.6.jar
start.jar
stax2-api-3.1.4.jar
@@ -229,7 +225,6 @@ tika-parsers-1.16.jar
tika-xmp-1.16.jar
vorbis-java-core-0.8.jar
vorbis-java-tika-0.8.jar
woodstox-core-asl-4.4.1.jar
xmlbeans-2.6.0.jar
xmpcore-5.1.2.jar
xz-1.6.jar
search-services/pom.xml
+2 -1
View File
@@ -14,7 +14,8 @@
<packaging>pom</packaging>
<name>Alfresco Solr Search parent</name>
<properties>
<slf4j.version>1.7.29</slf4j.version>
<slf4j.version>1.7.30</slf4j.version>
<cxf.version>3.2.12</cxf.version>
</properties>
<modules>
<module>alfresco-solrclient-lib</module>