From 24eca8c0541cd758f973188495e55ff8ffd18b3d Mon Sep 17 00:00:00 2001 From: Alan Davis Date: Sun, 11 Oct 2020 08:23:32 +0100 Subject: [PATCH] Merge changes on master 7.0.0-A7 projects since 7.0.0-A3 to release/master (#30) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Does not include chemistry-opencmis 1.1.0 (downgraded to 1.0.0) or jackson-databind 2.11.2 (downgraded to 2.10.1) due to test failures. Original upgrades by dependabot. * Fix latest T-Engine versions and remove redundantly versions in pom.xml files * Changes from acs-packaging * From alfresco-core (included json change after 7.0.0-A7) Bump commons-io from 2.7 to 2.8.0 (#160) (cherry picked from commit c5a8f5ffb3988bd87719aa43b7089851607b7308) MNT-21879 : [Security] Multiple json-java vulnerabilities - updated org.json.json version (cherry picked from commit edaffa350b29157bdcd1de3737fa491b88093378) [revert] MNT-21879 : [Security] Multiple json-java vulnerabilities - updated org.json.json version (cherry picked from commit 48e2cffb287adb5c5a1d5092ae7fde895c8d0836) * from alfresco-data-model SEARCH-2362: Consider changing properties from mandatory/enforced/pro… (#313) * SEARCH-2362: Consider changing properties from mandatory/enforced/protected to NON mandatory/enforced/protected as incremental changes * SEARCH-2362 Update tests to check whole set of returned diffs at once. Co-authored-by: Tom Page (cherry picked from commit 576601564251038311627f230f286b96ea950383) Bump dependency.cxf.version from 3.3.7 to 3.4.0 (#309) (cherry picked from commit dc689a911d83b608d49572cada5c2d16441c1eea) Bump chemistry-opencmis-commons-impl from 1.0.0 to 1.1.0 (#305) (cherry picked from commit 98fa7b66851aeada1bb6ea05bd6e747e3c55b342) Bump commons-compress from 1.19 to 1.20 (#303) (cherry picked from commit f6a950d4a36f05083536638cc5ff7866436f191d) Bump pdfbox from 2.0.17 to 2.0.20 (#304) (cherry picked from commit 435b786817ee18c28b84d093c6fbfbc330434e4c) * From alfresco-remote-api MNT-21702 : Kerberos SSO fallback mechanism for WebDAV (#765) Reinstated 'part' of the reverted code change originally made in MNT-16931 to handle fallback correctly for WebDAV in a kerberos environment. (cherry picked from commit 4091cb9d4b1cd643f9678a4c5c9654963a72c266) [MNT-21758] Cannot map an AOS / WebDAV drive with Kerberos SSO enabled (#784) - Add multi catch for Illegal Argument and Not Found when findWebScript is called. (cherry picked from commit 59517076d2dfc0477870007a7d136cdf907e322f) SEARCH-2363: Escape special characters when serializing ChildAssociationRefs and AssociationRefs objects (#788) (cherry picked from commit f941a6e283517c0949f57fefdb1e7e8d7cfb46c4) ACS-580: Direct access urls - revert initial api version (#792) (cherry picked from commit 7f4959fdfaeaf1b7debd78df5dcacd4a5b9e809a) * From alfresco-repository SEARCH-2289: Restore the support for SOLR 4 in ACS 6.2+ deployments. (#1063) This configuration is recommended for upgrading scenarios, where you need to upgrade ACS and to re-index the repository with SOLR 6 while using the system with SOLR 4. (cherry picked from commit e8bba3ca13880be782c2e4c1d82223e70c370f28) [MNT-21847] - Aync permissions fail when new nodes are created (#1188) Fix: *Changed method setFixedAcls on class ADMAccessControlListDAO to continue to propagate through children to apply the correct acl not only when the current child acl matches the shared acl to replace but also when the current child acl matches the new shared acl Unit Test: *Refactored the unit test FixedAclUpdaterTest to be able to add in a new test without repeating code: separating the operations that set the permissions from the one that triggers the job into separate methods *As it was if one test failed, leaving aspects to be processed, the test would run indefinitely (it was programmed to keep running the job while there where nodes with the aspect). Added a verification to stop triggering the job if the number of nodes with the pendingFixAcl did not change between executions. *Also, if one test failed, it would leave nodes with pendingFixAcl aspect in the database, and the other tests that ran after would also fail, not completing the goal of processing all nodes with the aspect. If a test fails, the folder structure it ran is now deleted so no nodes with the aspect from that structure are processed by the other tests. *Added a test to find the first folder in a tree where permissions where set async that has the pendingFixAcl aspect and that creates a new node in it to verify the issue (cherry picked from commit 443e5e226430a2760492fb82214ad520e7e1cb75) Bump dependency.pdfbox.version from 2.0.20 to 2.0.21 (#1170) (cherry picked from commit b93d73dec5f0e94d83f62721842b92e617dfba79) Bump rhino from 1.7.12 to 1.7.13 (#1185) (cherry picked from commit a5b5f072c64511999d3d27087a1a82f949371ac8) Bump spring-security-core from 5.2.1.RELEASE to 5.3.4.RELEASE (#1167) (cherry picked from commit 54f68d823333254cef74343288e99107494b1e28) REPO-5339 Improve concurrency of SimplePermissionReference (#1189) * REPO-5339 Improve concurrency of SimplePermissionReference * REPO-5339 Change locks to synchronised block * [REPO-5339] Improve concurrency of SimplePermissionReference - Move to ConcurrentHashMap - Capacity set to hold Repo + RM + slight overage for custom permissions - Getting new SimplePermissionReference will not block while getting existing value. - If a new value needs to be added to the Map, it will block when adding - If blocked it will wait, in order, until it can attempt to add - If it was blocked before it will check first if another operation added before adding as an atomic action * [REPO-5339] Improve concurrency of SimplePermissionReference - Simplify getPermissionReference by removing Future * [REPO-5339] Thread Lock detected for AclReadersGet invocations - Add load capacity and concurrency level Co-authored-by: Jared Ottley (cherry picked from commit 39ded1cc0a364d24f737584eed22bef0918a5359) [MNT-21766]: Significant degradation of performance as file and site count increase (#1217) * [MNT-21766]: Significant degradation of performance as file and site count increase - Added the following indexes on alf_transaction: idx_alf_txn_ctms_sc, cols (commit_time_ms) idx_alf_txn_id_ctms, cols (id, commit_time_ms) - Added the following indexes on alf_node: idx_alf_node_ver, cols (version) idx_alf_node_txn, cols (transaction_id) - Created patch patch.db-V6.3-add-indexes-node-transaction - Updated version.schema to 14002 - Added system property system.new-node-transaction-indexes.ignored, set as true by default to not apply the patch automatically - Created the MySQL update dbscripts to add the new indexes - and also drop and recreate the pre-existing index idx_alf_txn_ctms on alf_transaction that was inconsistent with the other DBMS: was indexing only commit_time_ms when on all other DBMS index idx_alf_txn_ctms was on (commit_time_ms, id) - Created the Postgres update dbscripts to add the new indexes (cherry picked from commit d70746f63a6e6a2154d2e37c577955a17c42256e) SEARCH-2450 Don't expose properties from solr endpoint if the model says not to index them. (#1228) (cherry picked from commit 2fea6c9484def402e2bfd7c3717071ff8df69c01) Fix/MNT-21800 CMIS Web Service Check Out returns error (#1232) * MNT-21800 : CMIS Web Service Check Out returns error Integrate path into codebase. (cherry picked from commit 51a7793668ba1c969c5de37603190244e3839cd0) * Revert dependency.opencmis.version change 1.1.0 back to 1.0.0 done by dependabot on 08/09/2020 as the upgraded failed in alfresco-remote-api tests which are now part of the same repo. * Trial build with previous rino version * Revert cxf upgrade to 3.4.0 from 3.7.7 as dependabot failed to apply it in acs-packaging Revert org-json to 20160212 from 20090211 as dependabot failed to apply it in alfresco-repository * Revert "Trial build with previous rino version" This reverts commit 40f8120d5df837033c3d63f66b10420e1fdd2df1. * Remove READMEs that have been moved to the top level. * Fix location of keystore Only copy win64 and linux pdf-renderer to dist zip Sort out what get included in dist jar and zip * jackson databind revert --- .travis.yml | 8 +- core/README.md | 42 -- data-model/README.md | 47 --- .../repo/dictionary/M2PropertyDefinition.java | 34 +- .../repo/dictionary/AbstractModelTest.java | 52 +++ .../repo/dictionary/DiffModelTest.java | 362 +++++++++--------- packaging/docker-alfresco/Dockerfile | 2 +- packaging/tests/environment/.env | 4 +- .../email/imap/ImapDeleteMessagesTests.java | 5 +- pom.xml | 25 +- remote-api/.travis.yml | 0 remote-api/README.md | 41 -- remote-api/pom.xml | 50 --- .../repo/web/scripts/solr/SOLRSerializer.java | 4 +- .../BaseKerberosAuthenticationFilter.java | 17 +- .../auth/BaseSSOAuthenticationFilter.java | 2 +- .../java/org/alfresco/rest/api/Nodes.java | 6 - .../org/alfresco/rest/api/Renditions.java | 4 - .../org/alfresco/rest/api/impl/NodesImpl.java | 66 ---- .../rest/api/impl/RenditionsImpl.java | 15 - .../api/model/DirectAccessUrlRequest.java | 58 --- .../api/nodes/NodeRenditionsRelation.java | 16 - .../nodes/NodeVersionRenditionsRelation.java | 18 - .../rest/api/nodes/NodeVersionsRelation.java | 21 - .../rest/api/nodes/NodesEntityResource.java | 13 - .../web/scripts/solr/SOLRSerializerTest.java | 120 ++++-- .../rest/api/tests/AbstractBaseApiTest.java | 45 --- .../alfresco/rest/api/tests/NodeApiTest.java | 69 ---- .../tests/NodeVersionRenditionsApiTest.java | 111 ------ .../rest/api/tests/NodeVersionsApiTest.java | 87 ----- .../rest/api/tests/RenditionsTest.java | 120 ------ .../tests/client/data/DirectAccessUrl.java | 49 --- .../client/data/DirectAccessUrlRequest.java | 50 --- repository/.travis.yml | 0 repository/README.md | 41 -- repository/pom.xml | 53 +-- .../opencmis/TransactionAwareHolder.java | 18 +- .../permissions/ADMAccessControlListDAO.java | 4 +- .../impl/SimplePermissionReference.java | 108 ++---- .../repo/solr/SOLRTrackingComponentImpl.java | 10 +- .../AlfrescoCreate-RepoTables.sql | 6 +- .../Schema-Reference-ALF.xml | 22 ++ .../AlfrescoCreate-RepoTables.sql | 4 + .../Schema-Reference-ALF.xml | 21 + .../alfresco/dbscripts/db-schema-context.xml | 1 + .../add-indexes-node-transaction.sql | 35 ++ .../add-indexes-node-transaction.sql | 32 ++ .../messages/patch-service.properties | 4 +- .../alfresco/patch/patch-services-context.xml | 12 + .../resources/alfresco/repository.properties | 7 +- .../Search/solr4/solr-search-context.xml | 11 +- .../permissions/FixedAclUpdaterTest.java | 84 +++- .../solr/SOLRTrackingComponentUnitTest.java | 115 ++++++ 53 files changed, 786 insertions(+), 1365 deletions(-) delete mode 100644 core/README.md delete mode 100644 data-model/README.md delete mode 100644 remote-api/.travis.yml delete mode 100644 remote-api/README.md delete mode 100644 remote-api/src/main/java/org/alfresco/rest/api/model/DirectAccessUrlRequest.java delete mode 100644 remote-api/src/test/java/org/alfresco/rest/api/tests/client/data/DirectAccessUrl.java delete mode 100644 remote-api/src/test/java/org/alfresco/rest/api/tests/client/data/DirectAccessUrlRequest.java delete mode 100644 repository/.travis.yml delete mode 100644 repository/README.md create mode 100644 repository/src/main/resources/alfresco/dbscripts/upgrade/6.3/org.alfresco.repo.domain.dialect.MySQLInnoDBDialect/add-indexes-node-transaction.sql create mode 100644 repository/src/main/resources/alfresco/dbscripts/upgrade/6.3/org.alfresco.repo.domain.dialect.PostgreSQLDialect/add-indexes-node-transaction.sql create mode 100644 repository/src/test/java/org/alfresco/repo/solr/SOLRTrackingComponentUnitTest.java diff --git a/.travis.yml b/.travis.yml index 359fcf58ca..c7804b1eaa 100644 --- a/.travis.yml +++ b/.travis.yml @@ -80,14 +80,14 @@ jobs: before_script: - docker run -d -p 5433:5432 -e POSTGRES_PASSWORD=alfresco -e POSTGRES_USER=alfresco -e POSTGRES_DB=alfresco postgres:11.7 postgres -c 'max_connections=300' - docker run -d -p 61616:61616 -p 5672:5672 alfresco/alfresco-activemq:5.15.8 - - docker run -d -p 8090:8090 -e JAVA_OPTS=" -Xms256m -Xmx256m" alfresco/alfresco-transform-core-aio:2.3.4 + - docker run -d -p 8090:8090 -e JAVA_OPTS=" -Xms256m -Xmx256m" alfresco/alfresco-transform-core-aio:2.3.5 script: travis_wait 20 mvn -B test -pl repository -Dtest=AppContext03TestSuite -Ddb.driver=org.postgresql.Driver -Ddb.name=alfresco -Ddb.url=jdbc:postgresql://localhost:5433/alfresco -Ddb.username=alfresco -Ddb.password=alfresco -Dalfresco-pdf-renderer.url=http://localhost:8090/ -Djodconverter.url=http://localhost:8090/ -Dimg.url=http://localhost:8090/ -Dtika.url=http://localhost:8090/ -Dtransform.misc.url=http://localhost:8090/ - name: "Repository - AppContext04TestSuite" before_script: - docker run -d -p 5433:5432 -e POSTGRES_PASSWORD=alfresco -e POSTGRES_USER=alfresco -e POSTGRES_DB=alfresco postgres:11.7 postgres -c 'max_connections=300' - docker run -d -p 61616:61616 -p 5672:5672 alfresco/alfresco-activemq:5.15.8 - - docker run -d -p 8090:8090 -e JAVA_OPTS=" -Xms256m -Xmx256m" alfresco/alfresco-transform-core-aio:2.3.4 + - docker run -d -p 8090:8090 -e JAVA_OPTS=" -Xms256m -Xmx256m" alfresco/alfresco-transform-core-aio:2.3.5 script: travis_wait 20 mvn -B test -pl repository -Dtest=AppContext04TestSuite -Ddb.driver=org.postgresql.Driver -Ddb.name=alfresco -Ddb.url=jdbc:postgresql://localhost:5433/alfresco -Ddb.username=alfresco -Ddb.password=alfresco - name: "Repository - AppContext05TestSuite" @@ -104,7 +104,7 @@ jobs: before_script: - docker run -d -p 5433:5432 -e POSTGRES_PASSWORD=alfresco -e POSTGRES_USER=alfresco -e POSTGRES_DB=alfresco postgres:11.7 postgres -c 'max_connections=300' - docker run -d -p 61616:61616 -p 5672:5672 alfresco/alfresco-activemq:5.15.8 - - docker run -d -p 8090:8090 -e JAVA_OPTS=" -Xms256m -Xmx256m" alfresco/alfresco-transform-core-aio:2.3.4 + - docker run -d -p 8090:8090 -e JAVA_OPTS=" -Xms256m -Xmx256m" alfresco/alfresco-transform-core-aio:2.3.5 script: travis_wait 20 mvn -B test -pl repository -Dtest=AppContext06TestSuite -Ddb.driver=org.postgresql.Driver -Ddb.name=alfresco -Ddb.url=jdbc:postgresql://localhost:5433/alfresco -Ddb.username=alfresco -Ddb.password=alfresco -Dalfresco-pdf-renderer.url=http://localhost:8090/ -Djodconverter.url=http://localhost:8090/ -Dimg.url=http://localhost:8090/ -Dtika.url=http://localhost:8090/ -Dtransform.misc.url=http://localhost:8090/ - name: "Repository - AppContextExtraTestSuite" @@ -117,7 +117,7 @@ jobs: before_script: - docker run -d -p 5433:5432 -e POSTGRES_PASSWORD=alfresco -e POSTGRES_USER=alfresco -e POSTGRES_DB=alfresco postgres:11.7 postgres -c 'max_connections=300' - docker run -d -p 61616:61616 -p 5672:5672 alfresco/alfresco-activemq:5.15.8 - - docker run -d -p 8090:8090 -e JAVA_OPTS=" -Xms256m -Xmx256m" alfresco/alfresco-transform-core-aio:2.3.4 + - docker run -d -p 8090:8090 -e JAVA_OPTS=" -Xms256m -Xmx256m" alfresco/alfresco-transform-core-aio:2.3.5 script: travis_wait 20 mvn -B test -pl repository -Dtest=MiscContextTestSuite -Ddb.driver=org.postgresql.Driver -Ddb.name=alfresco -Ddb.url=jdbc:postgresql://localhost:5433/alfresco -Ddb.username=alfresco -Ddb.password=alfresco -Dalfresco-pdf-renderer.url=http://localhost:8090/ -Djodconverter.url=http://localhost:8090/ -Dimg.url=http://localhost:8090/ -Dtika.url=http://localhost:8090/ -Dtransform.misc.url=http://localhost:8090/ - name: "Repository - MySQL tests" diff --git a/core/README.md b/core/README.md deleted file mode 100644 index 13462dd056..0000000000 --- a/core/README.md +++ /dev/null @@ -1,42 +0,0 @@ -### Alfresco Core -[![Build Status](https://travis-ci.com/Alfresco/alfresco-core.svg?branch=master)](https://travis-ci.com/Alfresco/alfresco-core) - -Alfresco Core is a library packaged as a jar file which is part of [Alfresco Content Services Repository](https://community.alfresco.com/docs/DOC-6385-project-overview-repository). -The library contains the following: -* Various helpers and utils -* Canned queries interface and supporting classes -* Generic encryption supporting classes - -Version 7 of the library uses Spring 5, Quartz 2.3 and does not have Hibernate dependency. - -### Building and testing -The project can be built and tested by running Maven command: -~~~ -mvn clean install -~~~ - -### Artifacts -The artifacts can be obtained by: -* downloading from [Alfresco repository](https://artifacts.alfresco.com/nexus/content/groups/public) -* getting as Maven dependency by adding the dependency to your pom file: -~~~ - - org.alfresco - alfresco-core - version - -~~~ -and Alfresco repository: -~~~ - - alfresco-maven-repo - https://artifacts.alfresco.com/nexus/content/groups/public - -~~~ -The SNAPSHOT version of the artifact is **never** published. - -### Old version history -The history for older versions can be found in [Alfresco SVN](https://svn.alfresco.com/repos/alfresco-open-mirror/services/alfresco-core/) - -### Contributing guide -Please use [this guide](CONTRIBUTING.md) to make a contribution to the project. diff --git a/data-model/README.md b/data-model/README.md deleted file mode 100644 index 3d9f56bc6c..0000000000 --- a/data-model/README.md +++ /dev/null @@ -1,47 +0,0 @@ -### Alfresco Data Model -[![Build Status](https://travis-ci.com/Alfresco/alfresco-data-model.svg?branch=master)](https://travis-ci.com/Alfresco/alfresco-data-model) - -Data model is a library packaged as a jar file which is part of [Alfresco Content Services Repository](https://community.alfresco.com/docs/DOC-6385-project-overview-repository). -The library contains the following: -* Dictionary, Repository and Search Services interfaces -* Models for data types and Dictionary implementation -* Parsers - -Please note that the data model uses version 2 of the Jackson libraries. -The upgrade from version 1 was not backward compatible, any projects -that are dependent on data model using Jackson 1.x should use the data-model 6.N branch. - -Version 8.0 of data-model depends on alfresco-core 7.0 which is based on Spring 5. - - -### Building and testing -The project can be built and tested by running Maven command: -~~~ -mvn clean install -~~~ - -### Artifacts -The artifacts can be obtained by: -* downloading from [Alfresco repository](https://artifacts.alfresco.com/nexus/content/groups/public) -* getting as Maven dependency by adding the dependency to your pom file: -~~~ - - org.alfresco - alfresco-data-model - version - -~~~ -and Alfresco repository: -~~~ - - alfresco-maven-repo - https://artifacts.alfresco.com/nexus/content/groups/public - -~~~ -The SNAPSHOT version of the artifact is **never** published. - -### Old version history -The history for older versions can be found in [Alfresco SVN](https://svn.alfresco.com/repos/alfresco-open-mirror/alfresco/HEAD/root/projects/data-model) - -### Contributing guide -Please use [this guide](CONTRIBUTING.md) to make a contribution to the project. diff --git a/data-model/src/main/java/org/alfresco/repo/dictionary/M2PropertyDefinition.java b/data-model/src/main/java/org/alfresco/repo/dictionary/M2PropertyDefinition.java index e69a2c765a..d7883f3348 100644 --- a/data-model/src/main/java/org/alfresco/repo/dictionary/M2PropertyDefinition.java +++ b/data-model/src/main/java/org/alfresco/repo/dictionary/M2PropertyDefinition.java @@ -552,19 +552,43 @@ import org.springframework.util.StringUtils; // check mandatory if (isMandatory() != propDef.isMandatory()) { - isUpdated = true; + // Change from mandatory to NON mandatory is an incremental change + if (isMandatory() && !propDef.isMandatory()) + { + isUpdatedIncrementally = true; + } + else + { + isUpdated = true; + } } // check mandatory enforced if (isMandatoryEnforced() != propDef.isMandatoryEnforced()) - { - isUpdated = true; + { + // Change from mandatory enforced to NON mandatory enforced is an incremental change + if (isMandatoryEnforced() && ! propDef.isMandatoryEnforced()) + { + isUpdatedIncrementally = true; + } + else + { + isUpdated = true; + } } // check protected if (isProtected() != propDef.isProtected()) - { - isUpdated = true; + { + // Change from protected to NON protected is an incremental change + if (isProtected() && !propDef.isProtected()) + { + isUpdatedIncrementally = true; + } + else + { + isUpdated = true; + } } // diff --git a/data-model/src/test/java/org/alfresco/repo/dictionary/AbstractModelTest.java b/data-model/src/test/java/org/alfresco/repo/dictionary/AbstractModelTest.java index 3b81760c58..305120d150 100644 --- a/data-model/src/test/java/org/alfresco/repo/dictionary/AbstractModelTest.java +++ b/data-model/src/test/java/org/alfresco/repo/dictionary/AbstractModelTest.java @@ -896,6 +896,58 @@ public class AbstractModelTest extends TestCase " " + ""; + public static final String MODEL8_XML = + "" + + + " " + + " " + + " " + + + " " + + " " + + " " + + + " " + + + " " + + " " + + " " + + " Prop A1 " + + " d:text " + + " true " + + " " + + " " + + " " + + + " " + + + ""; + public static final String MODEL8_CHANGE_MANDATORY_PROPERTIES_ASPECTS_XML = + "" + + + " " + + " " + + " " + + + " " + + " " + + " " + + + " " + + + " " + + " " + + " " + + " Prop A1 " + + " d:text " + + " false " + + " " + + " " + + " " + + + " " + + + ""; public AbstractModelTest() diff --git a/data-model/src/test/java/org/alfresco/repo/dictionary/DiffModelTest.java b/data-model/src/test/java/org/alfresco/repo/dictionary/DiffModelTest.java index ca5308fc73..10cd6c0ec1 100644 --- a/data-model/src/test/java/org/alfresco/repo/dictionary/DiffModelTest.java +++ b/data-model/src/test/java/org/alfresco/repo/dictionary/DiffModelTest.java @@ -25,12 +25,32 @@ */ package org.alfresco.repo.dictionary; +import static java.util.function.Function.identity; +import static java.util.stream.Collectors.averagingDouble; +import static java.util.stream.Collectors.toMap; + +import static org.alfresco.repo.dictionary.M2ModelDiff.DIFF_CREATED; +import static org.alfresco.repo.dictionary.M2ModelDiff.DIFF_DELETED; +import static org.alfresco.repo.dictionary.M2ModelDiff.DIFF_UNCHANGED; +import static org.alfresco.repo.dictionary.M2ModelDiff.DIFF_UPDATED; +import static org.alfresco.repo.dictionary.M2ModelDiff.DIFF_UPDATED_INC; +import static org.alfresco.repo.dictionary.M2ModelDiff.TYPE_ASPECT; +import static org.alfresco.repo.dictionary.M2ModelDiff.TYPE_ASSOCIATION; +import static org.alfresco.repo.dictionary.M2ModelDiff.TYPE_PROPERTY; +import static org.alfresco.repo.dictionary.M2ModelDiff.TYPE_TYPE; + import java.io.ByteArrayInputStream; import java.util.ArrayList; +import java.util.HashMap; import java.util.List; +import java.util.Map; import java.util.concurrent.LinkedBlockingQueue; import java.util.concurrent.ThreadPoolExecutor; import java.util.concurrent.TimeUnit; +import java.util.function.Function; +import java.util.stream.Collectors; + +import com.google.common.collect.Maps; import junit.framework.TestCase; @@ -40,8 +60,10 @@ import org.alfresco.repo.tenant.TenantService; import org.alfresco.service.namespace.NamespaceException; import org.alfresco.service.namespace.QName; import org.alfresco.util.DynamicallySizedThreadPoolExecutor; +import org.alfresco.util.Pair; import org.alfresco.util.TraceableThreadFactory; import org.alfresco.util.cache.DefaultAsynchronouslyRefreshedCacheRegistry; +import org.apache.commons.collections4.map.UnmodifiableMap; public class DiffModelTest extends AbstractModelTest { @@ -90,16 +112,11 @@ public class DiffModelTest extends AbstractModelTest CompiledModel previousVersion = dictionaryDAO.getCompiledModel(modelName); List modelDiffs = dictionaryDAO.diffModel(previousVersion, null); - - for (M2ModelDiff modelDiff : modelDiffs) - { - System.out.println(modelDiff.toString()); - } - - assertEquals(6, modelDiffs.size()); - - assertEquals(3, countDiffs(modelDiffs, M2ModelDiff.TYPE_TYPE, M2ModelDiff.DIFF_DELETED)); - assertEquals(3, countDiffs(modelDiffs, M2ModelDiff.TYPE_ASPECT, M2ModelDiff.DIFF_DELETED)); + + Map, Integer> expected = Map.of( + new Pair(TYPE_TYPE, DIFF_DELETED), 3, + new Pair(TYPE_ASPECT, DIFF_DELETED), 3); + assertEquals("Unexpected set of diffs found.", expected, getAllDiffCounts(modelDiffs)); } @SuppressWarnings("unused") @@ -143,16 +160,11 @@ public class DiffModelTest extends AbstractModelTest CompiledModel newVersion = dictionaryDAO.getCompiledModel(modelName); List modelDiffs = dictionaryDAO.diffModel(null, newVersion); - - for (M2ModelDiff modelDiff : modelDiffs) - { - System.out.println(modelDiff.toString()); - } - - assertEquals(6, modelDiffs.size()); - - assertEquals(3, countDiffs(modelDiffs, M2ModelDiff.TYPE_TYPE, M2ModelDiff.DIFF_CREATED)); - assertEquals(3, countDiffs(modelDiffs, M2ModelDiff.TYPE_ASPECT, M2ModelDiff.DIFF_CREATED)); + + Map, Integer> expected = Map.of( + new Pair(TYPE_TYPE, DIFF_CREATED), 3, + new Pair(TYPE_ASPECT, DIFF_CREATED), 3); + assertEquals("Unexpected set of diffs found.", expected, getAllDiffCounts(modelDiffs)); } public void testDuplicateModels() @@ -189,28 +201,20 @@ public class DiffModelTest extends AbstractModelTest CompiledModel newVersion = dictionaryDAO.getCompiledModel(modelName); List modelDiffs = dictionaryDAO.diffModel(previousVersion, newVersion); - - for (M2ModelDiff M2ModelDiff : modelDiffs) - { - System.out.println(M2ModelDiff.toString()); - } - - assertEquals(16, modelDiffs.size()); - - assertEquals(1, countDiffs(modelDiffs, M2ModelDiff.TYPE_TYPE, M2ModelDiff.DIFF_CREATED)); - assertEquals(2, countDiffs(modelDiffs, M2ModelDiff.TYPE_TYPE, M2ModelDiff.DIFF_UNCHANGED)); - assertEquals(0, countDiffs(modelDiffs, M2ModelDiff.TYPE_TYPE, M2ModelDiff.DIFF_UPDATED)); - assertEquals(1, countDiffs(modelDiffs, M2ModelDiff.TYPE_TYPE, M2ModelDiff.DIFF_DELETED)); - - assertEquals(1, countDiffs(modelDiffs, M2ModelDiff.TYPE_ASPECT, M2ModelDiff.DIFF_CREATED)); - assertEquals(2, countDiffs(modelDiffs, M2ModelDiff.TYPE_ASPECT, M2ModelDiff.DIFF_UNCHANGED)); - assertEquals(0, countDiffs(modelDiffs, M2ModelDiff.TYPE_ASPECT, M2ModelDiff.DIFF_UPDATED)); - assertEquals(1, countDiffs(modelDiffs, M2ModelDiff.TYPE_ASPECT, M2ModelDiff.DIFF_DELETED)); - - assertEquals(0, countDiffs(modelDiffs, M2ModelDiff.TYPE_PROPERTY, M2ModelDiff.DIFF_CREATED)); - assertEquals(6, countDiffs(modelDiffs, M2ModelDiff.TYPE_PROPERTY, M2ModelDiff.DIFF_UNCHANGED)); - assertEquals(1, countDiffs(modelDiffs, M2ModelDiff.TYPE_PROPERTY, M2ModelDiff.DIFF_UPDATED)); - assertEquals(1, countDiffs(modelDiffs, M2ModelDiff.TYPE_PROPERTY, M2ModelDiff.DIFF_DELETED)); + + Map, Integer> expected = Map.of( + new Pair(TYPE_TYPE, DIFF_CREATED), 1, + new Pair(TYPE_TYPE, DIFF_UNCHANGED), 2, + new Pair(TYPE_TYPE, DIFF_DELETED), 1, + + new Pair(TYPE_ASPECT, DIFF_CREATED), 1, + new Pair(TYPE_ASPECT, DIFF_UNCHANGED), 2, + new Pair(TYPE_ASPECT, DIFF_DELETED), 1, + + new Pair(TYPE_PROPERTY, DIFF_UNCHANGED), 6, + new Pair(TYPE_PROPERTY, DIFF_UPDATED), 1, + new Pair(TYPE_PROPERTY, DIFF_DELETED), 1); + assertEquals("Unexpected set of diffs found.", expected, getAllDiffCounts(modelDiffs)); } public void testIncUpdatePropertiesAdded() @@ -226,18 +230,13 @@ public class DiffModelTest extends AbstractModelTest CompiledModel newVersion = dictionaryDAO.getCompiledModel(modelName); List modelDiffs = dictionaryDAO.diffModel(previousVersion, newVersion); - - for (M2ModelDiff modelDiff : modelDiffs) - { - System.out.println(modelDiff.toString()); - } - - assertEquals(8, modelDiffs.size()); - - assertEquals(1, countDiffs(modelDiffs, M2ModelDiff.TYPE_TYPE, M2ModelDiff.DIFF_UNCHANGED)); - assertEquals(1, countDiffs(modelDiffs, M2ModelDiff.TYPE_ASPECT, M2ModelDiff.DIFF_UNCHANGED)); - assertEquals(4, countDiffs(modelDiffs, M2ModelDiff.TYPE_PROPERTY, M2ModelDiff.DIFF_UNCHANGED)); - assertEquals(2, countDiffs(modelDiffs, M2ModelDiff.TYPE_PROPERTY, M2ModelDiff.DIFF_CREATED)); + + Map, Integer> expected = Map.of( + new Pair(TYPE_TYPE, DIFF_UNCHANGED), 1, + new Pair(TYPE_ASPECT, DIFF_UNCHANGED), 1, + new Pair(TYPE_PROPERTY, DIFF_UNCHANGED), 4, + new Pair(TYPE_PROPERTY, DIFF_CREATED), 2); + assertEquals("Unexpected set of diffs found.", expected, getAllDiffCounts(modelDiffs)); } public void testIncUpdateTypesAndAspectsAdded() @@ -253,21 +252,14 @@ public class DiffModelTest extends AbstractModelTest CompiledModel newVersion = dictionaryDAO.getCompiledModel(modelName); List modelDiffs = dictionaryDAO.diffModel(previousVersion, newVersion); - - for (M2ModelDiff modelDiff : modelDiffs) - { - System.out.println(modelDiff.toString()); - } - - assertEquals(8, modelDiffs.size()); - - assertEquals(1, countDiffs(modelDiffs, M2ModelDiff.TYPE_TYPE, M2ModelDiff.DIFF_UNCHANGED)); - assertEquals(1, countDiffs(modelDiffs, M2ModelDiff.TYPE_ASPECT, M2ModelDiff.DIFF_UNCHANGED)); - - assertEquals(1, countDiffs(modelDiffs, M2ModelDiff.TYPE_TYPE, M2ModelDiff.DIFF_CREATED)); - assertEquals(1, countDiffs(modelDiffs, M2ModelDiff.TYPE_ASPECT, M2ModelDiff.DIFF_CREATED)); - - assertEquals(4, countDiffs(modelDiffs, M2ModelDiff.TYPE_PROPERTY, M2ModelDiff.DIFF_UNCHANGED)); + + Map, Integer> expected = Map.of( + new Pair(TYPE_TYPE, DIFF_UNCHANGED), 1, + new Pair(TYPE_ASPECT, DIFF_UNCHANGED), 1, + new Pair(TYPE_TYPE, DIFF_CREATED), 1, + new Pair(TYPE_ASPECT, DIFF_CREATED), 1, + new Pair(TYPE_PROPERTY, DIFF_UNCHANGED), 4); + assertEquals("Unexpected set of diffs found.", expected, getAllDiffCounts(modelDiffs)); } public void testIncUpdateAssociationsAdded() @@ -283,22 +275,14 @@ public class DiffModelTest extends AbstractModelTest CompiledModel newVersion = dictionaryDAO.getCompiledModel(modelName); List modelDiffs = dictionaryDAO.diffModel(previousVersion, newVersion); - - for (M2ModelDiff modelDiff : modelDiffs) - { - System.out.println(modelDiff.toString()); - } - - assertEquals(12, modelDiffs.size()); - - assertEquals(1, countDiffs(modelDiffs, M2ModelDiff.TYPE_TYPE, M2ModelDiff.DIFF_UPDATED_INC)); - assertEquals(1, countDiffs(modelDiffs, M2ModelDiff.TYPE_TYPE, M2ModelDiff.DIFF_UNCHANGED)); - - assertEquals(2, countDiffs(modelDiffs, M2ModelDiff.TYPE_ASPECT, M2ModelDiff.DIFF_UNCHANGED)); - - assertEquals(6, countDiffs(modelDiffs, M2ModelDiff.TYPE_PROPERTY, M2ModelDiff.DIFF_UNCHANGED)); - - assertEquals(2, countDiffs(modelDiffs, M2ModelDiff.TYPE_ASSOCIATION, M2ModelDiff.DIFF_CREATED)); + + Map, Integer> expected = Map.of( + new Pair(TYPE_TYPE, DIFF_UPDATED_INC), 1, + new Pair(TYPE_TYPE, DIFF_UNCHANGED), 1, + new Pair(TYPE_ASPECT, DIFF_UNCHANGED), 2, + new Pair(TYPE_PROPERTY, DIFF_UNCHANGED), 6, + new Pair(TYPE_ASSOCIATION, DIFF_CREATED), 2); + assertEquals("Unexpected set of diffs found.", expected, getAllDiffCounts(modelDiffs)); } public void testIncUpdateTitleDescription() @@ -314,17 +298,12 @@ public class DiffModelTest extends AbstractModelTest CompiledModel newVersion = dictionaryDAO.getCompiledModel(modelName); List modelDiffs = dictionaryDAO.diffModel(previousVersion, newVersion); - - for (M2ModelDiff modelDiff : modelDiffs) - { - System.out.println(modelDiff.toString()); - } - - assertEquals(4, modelDiffs.size()); - - assertEquals(1, countDiffs(modelDiffs, M2ModelDiff.TYPE_TYPE, M2ModelDiff.DIFF_UPDATED_INC)); - assertEquals(1, countDiffs(modelDiffs, M2ModelDiff.TYPE_ASPECT, M2ModelDiff.DIFF_UNCHANGED)); - assertEquals(2, countDiffs(modelDiffs, M2ModelDiff.TYPE_PROPERTY, M2ModelDiff.DIFF_UPDATED_INC)); + + Map, Integer> expected = Map.of( + new Pair(TYPE_TYPE, DIFF_UPDATED_INC), 1, + new Pair(TYPE_ASPECT, DIFF_UNCHANGED), 1, + new Pair(TYPE_PROPERTY, DIFF_UPDATED_INC), 2); + assertEquals("Unexpected set of diffs found.", expected, getAllDiffCounts(modelDiffs)); } public void testNonIncUpdatePropertiesRemoved() @@ -340,18 +319,13 @@ public class DiffModelTest extends AbstractModelTest CompiledModel newVersion = dictionaryDAO.getCompiledModel(modelName); List modelDiffs = dictionaryDAO.diffModel(previousVersion, newVersion); - - for (M2ModelDiff modelDiff : modelDiffs) - { - System.out.println(modelDiff.toString()); - } - - assertEquals(8, modelDiffs.size()); - - assertEquals(1, countDiffs(modelDiffs, M2ModelDiff.TYPE_TYPE, M2ModelDiff.DIFF_UNCHANGED)); - assertEquals(1, countDiffs(modelDiffs, M2ModelDiff.TYPE_ASPECT, M2ModelDiff.DIFF_UNCHANGED)); - assertEquals(4, countDiffs(modelDiffs, M2ModelDiff.TYPE_PROPERTY, M2ModelDiff.DIFF_UNCHANGED)); - assertEquals(2, countDiffs(modelDiffs, M2ModelDiff.TYPE_PROPERTY, M2ModelDiff.DIFF_DELETED)); + + Map, Integer> expected = Map.of( + new Pair(TYPE_TYPE, DIFF_UNCHANGED), 1, + new Pair(TYPE_ASPECT, DIFF_UNCHANGED), 1, + new Pair(TYPE_PROPERTY, DIFF_UNCHANGED), 4, + new Pair(TYPE_PROPERTY, DIFF_DELETED), 2); + assertEquals("Unexpected set of diffs found.", expected, getAllDiffCounts(modelDiffs)); } public void testNonIncUpdateTypesAndAspectsRemoved() @@ -367,21 +341,14 @@ public class DiffModelTest extends AbstractModelTest CompiledModel newVersion = dictionaryDAO.getCompiledModel(modelName); List modelDiffs = dictionaryDAO.diffModel(previousVersion, newVersion); - - for (M2ModelDiff modelDiff : modelDiffs) - { - System.out.println(modelDiff.toString()); - } - - assertEquals(8, modelDiffs.size()); - - assertEquals(1, countDiffs(modelDiffs, M2ModelDiff.TYPE_TYPE, M2ModelDiff.DIFF_UNCHANGED)); - assertEquals(1, countDiffs(modelDiffs, M2ModelDiff.TYPE_ASPECT, M2ModelDiff.DIFF_UNCHANGED)); - - assertEquals(1, countDiffs(modelDiffs, M2ModelDiff.TYPE_TYPE, M2ModelDiff.DIFF_DELETED)); - assertEquals(1, countDiffs(modelDiffs, M2ModelDiff.TYPE_ASPECT, M2ModelDiff.DIFF_DELETED)); - - assertEquals(4, countDiffs(modelDiffs, M2ModelDiff.TYPE_PROPERTY, M2ModelDiff.DIFF_UNCHANGED)); + + Map, Integer> expected = Map.of( + new Pair(TYPE_TYPE, DIFF_UNCHANGED), 1, + new Pair(TYPE_ASPECT, DIFF_UNCHANGED), 1, + new Pair(TYPE_TYPE, DIFF_DELETED), 1, + new Pair(TYPE_ASPECT, DIFF_DELETED), 1, + new Pair(TYPE_PROPERTY, DIFF_UNCHANGED), 4); + assertEquals("Unexpected set of diffs found.", expected, getAllDiffCounts(modelDiffs)); } public void testNonIncUpdateDefaultAspectAdded() @@ -397,17 +364,12 @@ public class DiffModelTest extends AbstractModelTest CompiledModel newVersion = dictionaryDAO.getCompiledModel(modelName); List modelDiffs = dictionaryDAO.diffModel(previousVersion, newVersion); - - for (M2ModelDiff modelDiff : modelDiffs) - { - System.out.println(modelDiff.toString()); - } - - assertEquals(4, modelDiffs.size()); - - assertEquals(1, countDiffs(modelDiffs, M2ModelDiff.TYPE_TYPE, M2ModelDiff.DIFF_UPDATED)); - assertEquals(1, countDiffs(modelDiffs, M2ModelDiff.TYPE_ASPECT, M2ModelDiff.DIFF_UNCHANGED)); - assertEquals(2, countDiffs(modelDiffs, M2ModelDiff.TYPE_PROPERTY, M2ModelDiff.DIFF_UNCHANGED)); + + Map, Integer> expected = Map.of( + new Pair(TYPE_TYPE, DIFF_UPDATED), 1, + new Pair(TYPE_ASPECT, DIFF_UNCHANGED), 1, + new Pair(TYPE_PROPERTY, DIFF_UNCHANGED), 2); + assertEquals("Unexpected set of diffs found.", expected, getAllDiffCounts(modelDiffs)); } public void testNonIncUpdateAssociationsRemoved() @@ -423,22 +385,14 @@ public class DiffModelTest extends AbstractModelTest CompiledModel newVersion = dictionaryDAO.getCompiledModel(modelName); List modelDiffs = dictionaryDAO.diffModel(previousVersion, newVersion); - - for (M2ModelDiff modelDiff : modelDiffs) - { - System.out.println(modelDiff.toString()); - } - - assertEquals(12, modelDiffs.size()); - - assertEquals(1, countDiffs(modelDiffs, M2ModelDiff.TYPE_TYPE, M2ModelDiff.DIFF_UPDATED)); - assertEquals(1, countDiffs(modelDiffs, M2ModelDiff.TYPE_TYPE, M2ModelDiff.DIFF_UNCHANGED)); - - assertEquals(2, countDiffs(modelDiffs, M2ModelDiff.TYPE_ASPECT, M2ModelDiff.DIFF_UNCHANGED)); - - assertEquals(6, countDiffs(modelDiffs, M2ModelDiff.TYPE_PROPERTY, M2ModelDiff.DIFF_UNCHANGED)); - - assertEquals(2, countDiffs(modelDiffs, M2ModelDiff.TYPE_ASSOCIATION, M2ModelDiff.DIFF_DELETED)); + + Map, Integer> expected = Map.of( + new Pair(TYPE_TYPE, DIFF_UPDATED), 1, + new Pair(TYPE_TYPE, DIFF_UNCHANGED), 1, + new Pair(TYPE_ASPECT, DIFF_UNCHANGED), 2, + new Pair(TYPE_PROPERTY, DIFF_UNCHANGED), 6, + new Pair(TYPE_ASSOCIATION, DIFF_DELETED), 2); + assertEquals("Unexpected set of diffs found.", expected, getAllDiffCounts(modelDiffs)); } public void testIncUpdatePropertiesAddedToMandatoryAspect() @@ -454,16 +408,11 @@ public class DiffModelTest extends AbstractModelTest CompiledModel newVersion = dictionaryDAO.getCompiledModel(modelName); List modelDiffs = dictionaryDAO.diffModel(previousVersion, newVersion); - - for (M2ModelDiff modelDiff : modelDiffs) - { - System.out.println(modelDiff.toString()); - } - - assertEquals(3, modelDiffs.size()); - - assertEquals(2, countDiffs(modelDiffs, M2ModelDiff.TYPE_ASPECT, M2ModelDiff.DIFF_UNCHANGED)); - assertEquals(1, countDiffs(modelDiffs, M2ModelDiff.TYPE_PROPERTY, M2ModelDiff.DIFF_CREATED)); + + Map, Integer> expected = Map.of( + new Pair(TYPE_ASPECT, DIFF_UNCHANGED), 2, + new Pair(TYPE_PROPERTY, DIFF_CREATED), 1); + assertEquals("Unexpected set of diffs found.", expected, getAllDiffCounts(modelDiffs)); } public void testNonIncUpdatePropertiesRemovedFromMandatoryAspect() @@ -479,30 +428,71 @@ public class DiffModelTest extends AbstractModelTest CompiledModel newVersion = dictionaryDAO.getCompiledModel(modelName); List modelDiffs = dictionaryDAO.diffModel(previousVersion, newVersion); - - for (M2ModelDiff modelDiff : modelDiffs) - { - System.out.println(modelDiff.toString()); - } - - assertEquals(3, modelDiffs.size()); - - assertEquals(2, countDiffs(modelDiffs, M2ModelDiff.TYPE_ASPECT, M2ModelDiff.DIFF_UNCHANGED)); - assertEquals(1, countDiffs(modelDiffs, M2ModelDiff.TYPE_PROPERTY, M2ModelDiff.DIFF_DELETED)); - } - - private int countDiffs(List M2ModelDiffs, String elementType, String diffType) - { - int count = 0; - for (M2ModelDiff modelDiff : M2ModelDiffs) - { - if (modelDiff.getDiffType().equals(diffType) && modelDiff.getElementType().equals(elementType)) - { - count++; - } - } - return count; - } - -} + Map, Integer> expected = Map.of( + new Pair(TYPE_ASPECT, DIFF_UNCHANGED), 2, + new Pair(TYPE_PROPERTY, DIFF_DELETED), 1); + assertEquals("Unexpected set of diffs found.", expected, getAllDiffCounts(modelDiffs)); + } + + /** + * Changing a property from mandatory/enforced/protected to NON mandatory/enforced/protected + * is an incremental change and it should be allowed. + */ + public void testIncChangeMandatoryProperties() + { + ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(AbstractModelTest.MODEL8_XML.getBytes()); + M2Model model = M2Model.createModel(byteArrayInputStream); + QName modelName = dictionaryDAO.putModel(model); + CompiledModel previousVersion = dictionaryDAO.getCompiledModel(modelName); + + byteArrayInputStream = new ByteArrayInputStream(AbstractModelTest.MODEL8_CHANGE_MANDATORY_PROPERTIES_ASPECTS_XML.getBytes()); + model = M2Model.createModel(byteArrayInputStream); + modelName = dictionaryDAO.putModel(model); + CompiledModel newVersion = dictionaryDAO.getCompiledModel(modelName); + + List modelDiffs = dictionaryDAO.diffModel(previousVersion, newVersion); + + Map, Integer> expected = Map.of( + new Pair(TYPE_ASPECT, DIFF_UNCHANGED), 1, + new Pair(TYPE_PROPERTY, DIFF_UPDATED_INC), 1); + assertEquals("Unexpected set of diffs found.", expected, getAllDiffCounts(modelDiffs)); + } + + /** + * Changing a property from NOT mandatory/enforced/protected to mandatory/enforced/protected + * is considered to be a non incremental change. + */ + public void testNonIncChangeMandatoryProperties() + { + ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(AbstractModelTest.MODEL8_CHANGE_MANDATORY_PROPERTIES_ASPECTS_XML.getBytes()); + M2Model model = M2Model.createModel(byteArrayInputStream); + QName modelName = dictionaryDAO.putModel(model); + CompiledModel previousVersion = dictionaryDAO.getCompiledModel(modelName); + + byteArrayInputStream = new ByteArrayInputStream(AbstractModelTest.MODEL8_XML.getBytes()); + model = M2Model.createModel(byteArrayInputStream); + modelName = dictionaryDAO.putModel(model); + CompiledModel newVersion = dictionaryDAO.getCompiledModel(modelName); + + List modelDiffs = dictionaryDAO.diffModel(previousVersion, newVersion); + + Map, Integer> expected = Map.of( + new Pair(TYPE_ASPECT, DIFF_UNCHANGED), 1, + new Pair(TYPE_PROPERTY, DIFF_UPDATED), 1); + assertEquals("Unexpected set of diffs found.", expected, getAllDiffCounts(modelDiffs)); + } + + /** + * Count the diffs grouping by element type and diff type. + * + * @param m2ModelDiffs The list of diffs returned from the dictionaryDAO. + * @return A map from (elementType, diffType) to the number of occurrences of matching diffs in the list. + */ + private Map, Integer> getAllDiffCounts(List m2ModelDiffs) + { + return m2ModelDiffs.stream() + .map(modelDiff -> new Pair<>(modelDiff.getElementType(), modelDiff.getDiffType())) + .collect(toMap(identity(), pair -> 1, Integer::sum)); + } +} diff --git a/packaging/docker-alfresco/Dockerfile b/packaging/docker-alfresco/Dockerfile index ad3546211b..7da3eb0ded 100644 --- a/packaging/docker-alfresco/Dockerfile +++ b/packaging/docker-alfresco/Dockerfile @@ -34,7 +34,7 @@ COPY ${resource_path}/war ${TOMCAT_DIR}/webapps COPY ${resource_path}/connector/* ${TOMCAT_DIR}/lib/ COPY ${resource_path}/alfresco-mmt/* ${TOMCAT_DIR}/alfresco-mmt/ COPY ${resource_path}/dependency/licenses/ /licenses/ -COPY ${resource_path}/dependency/keystore/metadata-keystore ${TOMCAT_DIR}/shared/classes/alfresco/extension/keystore/ +COPY ${resource_path}/dependency/keystore/metadata-keystore/keystore ${TOMCAT_DIR}/shared/classes/alfresco/extension/keystore # Change the value of the shared.loader= property to the following: # shared.loader=${catalina.base}/shared/classes diff --git a/packaging/tests/environment/.env b/packaging/tests/environment/.env index 389332f38b..b60ffc632e 100644 --- a/packaging/tests/environment/.env +++ b/packaging/tests/environment/.env @@ -1,4 +1,4 @@ -TRANSFORMERS_TAG=2.3.4 -SOLR6_TAG=1.4.2 +TRANSFORMERS_TAG=2.3.5 +SOLR6_TAG=2.0.0 POSTGRES_TAG=11.7 ACTIVEMQ_TAG=5.15.8 diff --git a/packaging/tests/tas-email/src/test/java/org/alfresco/email/imap/ImapDeleteMessagesTests.java b/packaging/tests/tas-email/src/test/java/org/alfresco/email/imap/ImapDeleteMessagesTests.java index fe8c70f873..3834b10276 100644 --- a/packaging/tests/tas-email/src/test/java/org/alfresco/email/imap/ImapDeleteMessagesTests.java +++ b/packaging/tests/tas-email/src/test/java/org/alfresco/email/imap/ImapDeleteMessagesTests.java @@ -21,7 +21,8 @@ public class ImapDeleteMessagesTests extends EmailTest adminSite = dataSite.usingAdmin().createIMAPSite(); } - @TestRail(section = { TestGroup.PROTOCOLS, TestGroup.IMAP }, executionType = ExecutionType.SANITY, + // TODO uncomment once https://issues.alfresco.com/jira/browse/MNT-21648 is solved + /* @TestRail(section = { TestGroup.PROTOCOLS, TestGroup.IMAP }, executionType = ExecutionType.SANITY, description = "Verify message can be deleted from IMAP client by admin") @Test(groups = { TestGroup.PROTOCOLS, TestGroup.IMAP, TestGroup.SANITY }) public void adminShouldDeleteMessage() throws Exception @@ -31,7 +32,7 @@ public class ImapDeleteMessagesTests extends EmailTest .and().usingResource(testFile).assertThat().existsInRepo().deleteMessage() .and().assertThat().doesNotContainMessages(testFile) .then().usingResource(testFile).assertThat().doesNotExistInRepo(); - } + } */ @TestRail(section = { TestGroup.PROTOCOLS, TestGroup.IMAP }, executionType = ExecutionType.SANITY, description = "Verify deleting message via IMAP client by user with MANAGER role") diff --git a/pom.xml b/pom.xml index 05d07cad2f..05632ae218 100644 --- a/pom.xml +++ b/pom.xml @@ -56,20 +56,20 @@ 6.2 0.0.8 - 5.2.8.RELEASE + 5.2.9.RELEASE 3.5.2 2.11.2 2.10.1 3.3.7 1.0.0 - 2.0.20 + 2.0.21 8.8 1.66 - 3.4.6 + 3.5.11 20090211 1.4-DBCP330 28.2-jre - 2.7 + 2.8.0 2.8.5 4.5.12 4.4.13 @@ -81,7 +81,7 @@ 2.5.9 1.6.2 1.24.1 - 5.2.1.RELEASE + 5.3.4.RELEASE 7.7.10 4.1.2 1.4 @@ -93,9 +93,9 @@ 4.0.3 3.2.0 - 1.3.1 + 1.4.0-M1 - 42.2.15 + 42.2.16 8.0.19 2.6.2 @@ -103,7 +103,7 @@ 3.3.0 1.1.4 1.47 - 1.13 + 1.14 1.8 1.6 1.5 @@ -174,11 +174,6 @@ win64 tgz - - org.alfresco - alfresco-pdf-renderer - ${dependency.alfresco-pdf-renderer.version} - org.alfresco alfresco-server-root @@ -243,7 +238,7 @@ commons-codec commons-codec - 1.14 + 1.15 commons-io @@ -525,7 +520,7 @@ org.apache.commons commons-compress - 1.19 + 1.20 diff --git a/remote-api/.travis.yml b/remote-api/.travis.yml deleted file mode 100644 index e69de29bb2..0000000000 diff --git a/remote-api/README.md b/remote-api/README.md deleted file mode 100644 index 8065f12c9d..0000000000 --- a/remote-api/README.md +++ /dev/null @@ -1,41 +0,0 @@ -### Alfresco Remote API -[![Build Status](https://travis-ci.com/Alfresco/alfresco-remote-api.svg?branch=master)](https://travis-ci.com/Alfresco/alfresco-remote-api) - -Remote API is a library packaged as a jar file which is part of [Alfresco Content Services Repository](https://community.alfresco.com/docs/DOC-6385-project-overview-repository). -The library contains the following: -* REST API framework -* WebScript implementations including [V1 REST APIs](https://community.alfresco.com/community/ecm/blog/2017/05/02/v1-rest-api-10-things-you-should-know) -* [OpenCMIS](https://chemistry.apache.org/java/opencmis.html) implementations - -### Building and testing -The project can be built by running Maven command: -~~~ -mvn clean install -~~~ -The tests are combined in test classes split by test type or Spring application context used in the test, see classes in _src/test/java/org/alfresco_. All of these classes as well as individual tests can be run by specifying the test class name and a set of DB connection properties, for example: -~~~ -mvn clean test -Dtest=SomeTest -Ddb.driver=org.postgresql.Driver -Ddb.name=alfresco -Ddb.url=jdbc:postgresql:alfresco -Ddb.username=alfresco -Ddb.password=alfresco -~~~ - -### Artifacts -The artifacts can be obtained by: -* downloading from [Alfresco repository](https://artifacts.alfresco.com/nexus/content/groups/public) -* getting as Maven dependency by adding the dependency to your pom file: -~~~ - - org.alfresco - alfresco-remote-api - version - -~~~ -and Alfresco Maven repository: -~~~ - - alfresco-maven-repo - https://artifacts.alfresco.com/nexus/content/groups/public - -~~~ -The SNAPSHOT version of the artifact is **never** published. - -### Contributing guide -Please use [this guide](CONTRIBUTING.md) to make a contribution to the project. diff --git a/remote-api/pom.xml b/remote-api/pom.xml index beadad0fa3..1bf9b1d0c9 100644 --- a/remote-api/pom.xml +++ b/remote-api/pom.xml @@ -221,7 +221,6 @@ org.alfresco alfresco-pdf-renderer - ${dependency.alfresco-pdf-renderer.version} win64 tgz @@ -266,7 +265,6 @@ org.alfresco alfresco-pdf-renderer - ${dependency.alfresco-pdf-renderer.version} linux tgz @@ -303,54 +301,6 @@ ${project.build.directory}/test-binaries/alfresco-pdf-renderer/alfresco-pdf-renderer - - osx-alfresco-pdf-renderer-test - - - mac - - - - - org.alfresco - alfresco-pdf-renderer - ${dependency.alfresco-pdf-renderer.version} - osx - tgz - - - - - - false - maven-antrun-plugin - - - extract-alfresco-pdf-renderer-test - generate-test-resources - - run - - - ${skipTests} - - - - - - - - - - - - - - - - ${project.build.directory}/test-binaries/alfresco-pdf-renderer/alfresco-pdf-renderer - - diff --git a/remote-api/src/main/java/org/alfresco/repo/web/scripts/solr/SOLRSerializer.java b/remote-api/src/main/java/org/alfresco/repo/web/scripts/solr/SOLRSerializer.java index b483948b12..ecffd579a7 100644 --- a/remote-api/src/main/java/org/alfresco/repo/web/scripts/solr/SOLRSerializer.java +++ b/remote-api/src/main/java/org/alfresco/repo/web/scripts/solr/SOLRSerializer.java @@ -438,7 +438,7 @@ import org.springframework.extensions.webscripts.json.JSONUtils; { public String convert(ChildAssociationRef source) { - return source.toString(); + return org.json.simple.JSONObject.escape(source.toString()); } }); @@ -454,7 +454,7 @@ import org.springframework.extensions.webscripts.json.JSONUtils; { public String convert(AssociationRef source) { - return source.toString(); + return org.json.simple.JSONObject.escape(source.toString()); } }); diff --git a/remote-api/src/main/java/org/alfresco/repo/webdav/auth/BaseKerberosAuthenticationFilter.java b/remote-api/src/main/java/org/alfresco/repo/webdav/auth/BaseKerberosAuthenticationFilter.java index 2b0bfd7096..b08ea6d271 100644 --- a/remote-api/src/main/java/org/alfresco/repo/webdav/auth/BaseKerberosAuthenticationFilter.java +++ b/remote-api/src/main/java/org/alfresco/repo/webdav/auth/BaseKerberosAuthenticationFilter.java @@ -366,7 +366,22 @@ public abstract class BaseKerberosAuthenticationFilter extends BaseSSOAuthentica } // Send back a request for SPNEGO authentication - logonStartAgain(context, req, resp, true); + + // MNT-21702 fixing Kerberos SSO fallback machanism for WebDAV + if (req.getRequestURL().toString().contains("webdav")) + { + if ( getLogger().isDebugEnabled()) { + getLogger().debug("WebDAV request, fallback"); + } + logonStartAgain(context, req, resp, false); + } + else + { + if ( getLogger().isDebugEnabled()) { + getLogger().debug("Non-WebDAV request, don't fallback"); + } + logonStartAgain(context, req, resp, true); + } return false; } diff --git a/remote-api/src/main/java/org/alfresco/repo/webdav/auth/BaseSSOAuthenticationFilter.java b/remote-api/src/main/java/org/alfresco/repo/webdav/auth/BaseSSOAuthenticationFilter.java index 3c4bcdb90f..a99a30e68b 100644 --- a/remote-api/src/main/java/org/alfresco/repo/webdav/auth/BaseSSOAuthenticationFilter.java +++ b/remote-api/src/main/java/org/alfresco/repo/webdav/auth/BaseSSOAuthenticationFilter.java @@ -208,7 +208,7 @@ public abstract class BaseSSOAuthenticationFilter extends BaseAuthenticationFilt { match = container.getRegistry().findWebScript(req.getMethod(), getScriptUrl(req)); } - catch (NotFoundException notFoundEx) + catch (NotFoundException | IllegalArgumentException Ex) { getLogger().debug(req.getMethod() + " " + getScriptUrl(req) + "not found in Public API Container."); } diff --git a/remote-api/src/main/java/org/alfresco/rest/api/Nodes.java b/remote-api/src/main/java/org/alfresco/rest/api/Nodes.java index 5ea2836315..48f1332d83 100644 --- a/remote-api/src/main/java/org/alfresco/rest/api/Nodes.java +++ b/remote-api/src/main/java/org/alfresco/rest/api/Nodes.java @@ -37,7 +37,6 @@ import org.alfresco.rest.api.model.Document; import org.alfresco.rest.api.model.Folder; import org.alfresco.rest.api.model.LockInfo; import org.alfresco.rest.api.model.Node; -import org.alfresco.rest.api.model.DirectAccessUrlRequest; import org.alfresco.rest.api.model.PathInfo; import org.alfresco.rest.api.model.UserInfo; import org.alfresco.rest.framework.resource.content.BasicContentInfo; @@ -45,7 +44,6 @@ import org.alfresco.rest.framework.resource.content.BinaryResource; import org.alfresco.rest.framework.resource.parameters.CollectionWithPagingInfo; import org.alfresco.rest.framework.resource.parameters.Parameters; import org.alfresco.service.cmr.repository.ChildAssociationRef; -import org.alfresco.service.cmr.repository.DirectAccessUrl; import org.alfresco.service.cmr.repository.NodeRef; import org.alfresco.service.cmr.repository.StoreRef; import org.alfresco.service.namespace.QName; @@ -268,10 +266,6 @@ public interface Nodes */ Node unlock(String nodeId, Parameters parameters); - DirectAccessUrl requestContentUrl(String nodeId, DirectAccessUrlRequest directAccessUrlRequest); - - DirectAccessUrl requestContentUrl(NodeRef nodeRef, DirectAccessUrlRequest directAccessUrlRequest); - /** * Convert from node properties (map of QName to Serializable) retrieved from * the respository to a map of String to Object that can be formatted/expressed diff --git a/remote-api/src/main/java/org/alfresco/rest/api/Renditions.java b/remote-api/src/main/java/org/alfresco/rest/api/Renditions.java index ca179312fa..3464bad52d 100644 --- a/remote-api/src/main/java/org/alfresco/rest/api/Renditions.java +++ b/remote-api/src/main/java/org/alfresco/rest/api/Renditions.java @@ -26,14 +26,12 @@ package org.alfresco.rest.api; -import org.alfresco.rest.api.model.DirectAccessUrlRequest; import org.alfresco.rest.api.model.Rendition; import org.alfresco.rest.framework.core.exceptions.ConstraintViolatedException; import org.alfresco.rest.framework.core.exceptions.NotFoundException; import org.alfresco.rest.framework.resource.content.BinaryResource; import org.alfresco.rest.framework.resource.parameters.CollectionWithPagingInfo; import org.alfresco.rest.framework.resource.parameters.Parameters; -import org.alfresco.service.cmr.repository.DirectAccessUrl; import org.alfresco.service.cmr.repository.NodeRef; import java.util.List; @@ -188,7 +186,5 @@ public interface Renditions * @return the rendition stream */ BinaryResource getContentNoValidation(NodeRef nodeRef, String versionId, String renditionId, Parameters parameters); - - DirectAccessUrl requestContentUrl(String nodeId, String versionId, String renditionId, DirectAccessUrlRequest directAccessUrlRequest); } diff --git a/remote-api/src/main/java/org/alfresco/rest/api/impl/NodesImpl.java b/remote-api/src/main/java/org/alfresco/rest/api/impl/NodesImpl.java index 600c276337..6604b6cc73 100644 --- a/remote-api/src/main/java/org/alfresco/rest/api/impl/NodesImpl.java +++ b/remote-api/src/main/java/org/alfresco/rest/api/impl/NodesImpl.java @@ -37,7 +37,6 @@ import java.util.ArrayList; import java.util.Arrays; import java.util.Collection; import java.util.Collections; -import java.util.Date; import java.util.HashMap; import java.util.HashSet; import java.util.LinkedHashSet; @@ -90,7 +89,6 @@ import org.alfresco.rest.api.model.Document; import org.alfresco.rest.api.model.Folder; import org.alfresco.rest.api.model.LockInfo; import org.alfresco.rest.api.model.Node; -import org.alfresco.rest.api.model.DirectAccessUrlRequest; import org.alfresco.rest.api.model.NodePermissions; import org.alfresco.rest.api.model.PathInfo; import org.alfresco.rest.api.model.PathInfo.ElementInfo; @@ -141,7 +139,6 @@ import org.alfresco.service.cmr.repository.ContentData; import org.alfresco.service.cmr.repository.ContentIOException; import org.alfresco.service.cmr.repository.ContentService; import org.alfresco.service.cmr.repository.ContentWriter; -import org.alfresco.service.cmr.repository.DirectAccessUrl; import org.alfresco.service.cmr.repository.DuplicateChildNodeNameException; import org.alfresco.service.cmr.repository.InvalidNodeRefException; import org.alfresco.service.cmr.repository.MimetypeService; @@ -169,7 +166,6 @@ import org.alfresco.util.PropertyCheck; import org.apache.commons.lang3.StringUtils; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; -import org.joda.time.DateTime; import org.springframework.dao.ConcurrencyFailureException; import org.springframework.extensions.surf.util.Content; import org.springframework.extensions.webscripts.servlet.FormData; @@ -3374,68 +3370,6 @@ public class NodesImpl implements Nodes return getFolderOrDocument(nodeId, parameters); } - @Override - public DirectAccessUrl requestContentUrl(String nodeId, DirectAccessUrlRequest nodeDirectAccess) - { - NodeRef nodeRef = validateOrLookupNode(nodeId, null); - return requestContentUrl(nodeRef, nodeDirectAccess); - } - - private void checkExpiryDate(Date expiryDate) - { - DateTime now = DateTime.now(); - if (now.isAfter(expiryDate.getTime())) - { - throw new InvalidArgumentException("Invalid expiry date. Expiry date can't be in the past."); - } - } - - @Override - public DirectAccessUrl requestContentUrl(NodeRef nodeRef, DirectAccessUrlRequest directAccessUrlRequest) - { - if (nodeRef == null) - { - throw new InvalidArgumentException("Missing nodeRef"); - } - - Date expiresAt = null; - if (directAccessUrlRequest != null) - { - if (directAccessUrlRequest.getExpiresAt() != null && directAccessUrlRequest.getValidFor() != null) - { - throw new InvalidArgumentException("Direct access url can not have both expiresAt and validFor set."); - } - - if (directAccessUrlRequest.getExpiresAt() != null) - { - checkExpiryDate(directAccessUrlRequest.getExpiresAt()); - - expiresAt = directAccessUrlRequest.getExpiresAt(); - } - else if (directAccessUrlRequest.getValidFor() != null) - { - Date expiration = new Date(); - long expTimeMillis = expiration.getTime(); - - expTimeMillis += (directAccessUrlRequest.getValidFor() * 1000); - expiration.setTime(expTimeMillis); - - checkExpiryDate(expiration); - - expiresAt = expiration; - } - } - - DirectAccessUrl directAccessUrl = contentService.getDirectAccessUrl(nodeRef, expiresAt); - - if (directAccessUrl == null) - { - throw new DisabledServiceException("Direct access url isn't available."); - } - - return directAccessUrl; - } - /** * Checks if same permission is sent more than once * @param locallySetPermissions diff --git a/remote-api/src/main/java/org/alfresco/rest/api/impl/RenditionsImpl.java b/remote-api/src/main/java/org/alfresco/rest/api/impl/RenditionsImpl.java index 8e65103edc..3c1b4f0353 100644 --- a/remote-api/src/main/java/org/alfresco/rest/api/impl/RenditionsImpl.java +++ b/remote-api/src/main/java/org/alfresco/rest/api/impl/RenditionsImpl.java @@ -39,7 +39,6 @@ import org.alfresco.rest.antlr.WhereClauseParser; import org.alfresco.rest.api.Nodes; import org.alfresco.rest.api.Renditions; import org.alfresco.rest.api.model.ContentInfo; -import org.alfresco.rest.api.model.DirectAccessUrlRequest; import org.alfresco.rest.api.model.Rendition; import org.alfresco.rest.api.model.Rendition.RenditionStatus; import org.alfresco.rest.framework.core.exceptions.ApiException; @@ -62,7 +61,6 @@ import org.alfresco.rest.workflow.api.impl.MapBasedQueryWalker; import org.alfresco.service.ServiceRegistry; import org.alfresco.service.cmr.repository.ChildAssociationRef; import org.alfresco.service.cmr.repository.ContentData; -import org.alfresco.service.cmr.repository.DirectAccessUrl; import org.alfresco.service.cmr.repository.MimetypeService; import org.alfresco.service.cmr.repository.NodeRef; import org.alfresco.service.cmr.repository.NodeService; @@ -718,18 +716,5 @@ public class RenditionsImpl implements Renditions, ResourceLoaderAware return status; } - @Override - public DirectAccessUrl requestContentUrl(String nodeId, String versionId, String renditionId, DirectAccessUrlRequest directAccessUrlRequest) - { - final NodeRef validatedNodeRef = validateNode(StoreRef.STORE_REF_WORKSPACE_SPACESSTORE, nodeId, versionId, null); - NodeRef renditionNodeRef = getRenditionByName(validatedNodeRef, renditionId, null); - - if (renditionNodeRef == null) - { - throw new NotFoundException("The rendition with id: " + renditionId + " was not found."); - } - - return nodes.requestContentUrl(renditionNodeRef, directAccessUrlRequest); - } } diff --git a/remote-api/src/main/java/org/alfresco/rest/api/model/DirectAccessUrlRequest.java b/remote-api/src/main/java/org/alfresco/rest/api/model/DirectAccessUrlRequest.java deleted file mode 100644 index ff25083d7a..0000000000 --- a/remote-api/src/main/java/org/alfresco/rest/api/model/DirectAccessUrlRequest.java +++ /dev/null @@ -1,58 +0,0 @@ -/* - * #%L - * Alfresco Remote API - * %% - * Copyright (C) 2005 - 2020 Alfresco Software Limited - * %% - * This file is part of the Alfresco software. - * If the software was purchased under a paid Alfresco license, the terms of - * the paid license agreement will prevail. Otherwise, the software is - * provided under the following open source license terms: - * - * Alfresco is free software: you can redistribute it and/or modify - * it under the terms of the GNU Lesser General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * Alfresco is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public License - * along with Alfresco. If not, see . - * #L% - */ -package org.alfresco.rest.api.model; - -import java.util.Date; - -public class DirectAccessUrlRequest -{ - private Date expiresAt; - private Long validFor; - - public DirectAccessUrlRequest() - { - } - - public Date getExpiresAt() - { - return expiresAt; - } - - public void setExpiresAt(Date expiresAt) - { - this.expiresAt = expiresAt; - } - - public Long getValidFor() - { - return validFor; - } - - public void setValidFor(Long validFor) - { - this.validFor = validFor; - } -} diff --git a/remote-api/src/main/java/org/alfresco/rest/api/nodes/NodeRenditionsRelation.java b/remote-api/src/main/java/org/alfresco/rest/api/nodes/NodeRenditionsRelation.java index d406fdfff8..f9702f1c62 100644 --- a/remote-api/src/main/java/org/alfresco/rest/api/nodes/NodeRenditionsRelation.java +++ b/remote-api/src/main/java/org/alfresco/rest/api/nodes/NodeRenditionsRelation.java @@ -27,28 +27,21 @@ package org.alfresco.rest.api.nodes; import org.alfresco.rest.api.Renditions; -import org.alfresco.rest.api.model.DirectAccessUrlRequest; import org.alfresco.rest.api.model.Rendition; import org.alfresco.rest.framework.BinaryProperties; -import org.alfresco.rest.framework.Operation; import org.alfresco.rest.framework.WebApiDescription; -import org.alfresco.rest.framework.WebApiParam; -import org.alfresco.rest.framework.core.ResourceParameter; import org.alfresco.rest.framework.resource.RelationshipResource; import org.alfresco.rest.framework.resource.actions.interfaces.RelationshipResourceAction; import org.alfresco.rest.framework.resource.actions.interfaces.RelationshipResourceBinaryAction; import org.alfresco.rest.framework.resource.content.BinaryResource; import org.alfresco.rest.framework.resource.parameters.CollectionWithPagingInfo; import org.alfresco.rest.framework.resource.parameters.Parameters; -import org.alfresco.rest.framework.webscripts.WithResponse; -import org.alfresco.service.cmr.repository.DirectAccessUrl; import org.alfresco.service.cmr.repository.NodeRef; import org.alfresco.service.cmr.repository.StoreRef; import org.alfresco.util.PropertyCheck; import org.springframework.beans.factory.InitializingBean; import org.springframework.extensions.webscripts.Status; -import javax.servlet.http.HttpServletResponse; import java.util.List; /** @@ -109,13 +102,4 @@ public class NodeRenditionsRelation implements RelationshipResourceAction.Read. * #L% */ package org.alfresco.repo.web.scripts.solr; -import static org.junit.Assert.*; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; import java.util.Date; +import org.alfresco.model.ContentModel; import org.alfresco.repo.web.scripts.solr.SOLRSerializer.SOLRTypeConverter; +import org.alfresco.service.cmr.dictionary.DictionaryService; +import org.alfresco.service.cmr.repository.AssociationRef; +import org.alfresco.service.cmr.repository.ChildAssociationRef; +import org.alfresco.service.cmr.repository.NodeRef; +import org.alfresco.service.namespace.NamespaceService; +import org.alfresco.service.namespace.QName; import org.alfresco.util.ISO8601DateFormat; +import org.json.JSONException; +import org.json.JSONObject; import org.junit.Test; +import org.mockito.Mockito; -public class SOLRSerializerTest +public class SOLRSerializerTest { - @Test + @Test public void testDateSerializer() { - SOLRTypeConverter typeConverter = new SOLRTypeConverter(null); - - trip(typeConverter, "1912-01-01T00:40:00-06:00", "1912-01-01T06:40:00.000Z"); - trip(typeConverter, "1812-01-01T00:40:00-06:00", "1812-01-01T06:40:00.000Z"); - trip(typeConverter, "1845-01-01T00:40:00-06:00", "1845-01-01T06:40:00.000Z"); - trip(typeConverter, "1846-01-01T00:40:00-06:00", "1846-01-01T06:40:00.000Z"); - trip(typeConverter, "1847-01-01T00:40:00-06:00", "1847-01-01T06:40:00.000Z"); - trip(typeConverter, "1848-01-01T00:40:00-06:00", "1848-01-01T06:40:00.000Z"); - - + SOLRTypeConverter typeConverter = new SOLRTypeConverter(null); + + trip(typeConverter, "1912-01-01T00:40:00-06:00", "1912-01-01T06:40:00.000Z"); + trip(typeConverter, "1812-01-01T00:40:00-06:00", "1812-01-01T06:40:00.000Z"); + trip(typeConverter, "1845-01-01T00:40:00-06:00", "1845-01-01T06:40:00.000Z"); + trip(typeConverter, "1846-01-01T00:40:00-06:00", "1846-01-01T06:40:00.000Z"); + trip(typeConverter, "1847-01-01T00:40:00-06:00", "1847-01-01T06:40:00.000Z"); + trip(typeConverter, "1848-01-01T00:40:00-06:00", "1848-01-01T06:40:00.000Z"); + } - - - private void trip( SOLRTypeConverter typeConverter, String iso, String zulu) - { - Date testDate = ISO8601DateFormat.parse(iso); - String strDate = typeConverter.INSTANCE.convert(String.class, testDate); - assertEquals(zulu, strDate); - } + + private void trip(SOLRTypeConverter typeConverter, String iso, String zulu) + { + Date testDate = ISO8601DateFormat.parse(iso); + String strDate = typeConverter.INSTANCE.convert(String.class, testDate); + assertEquals(zulu, strDate); + } + + /** + * Test SOLR Serialization including values with special characters for ChildAssociationRef + */ + @Test + public void testChildAssociationRefToJSONString() + { + SOLRSerializer solrSerializer = new SOLRSerializer(); + solrSerializer.setDictionaryService(Mockito.mock(DictionaryService.class)); + solrSerializer.setNamespaceService(Mockito.mock(NamespaceService.class)); + solrSerializer.init(); + + // Create a Child QName including special character \ + QName childQName = QName.createQName("hello", "wo\rld"); + + ChildAssociationRef childAssociationRef = new ChildAssociationRef(ContentModel.ASSOC_CONTAINS, + new NodeRef("workspace://SpacesStore/parent"), childQName, + new NodeRef("workspace://SpacesStore/child")); + String validJsonString = solrSerializer.serializeToJSONString(childAssociationRef); + String jsonObjectString = String.format("{ \"key\": \"%s\" }", validJsonString); + + try + { + new JSONObject(jsonObjectString); + } + catch (JSONException e) + { + assertTrue("JSON String " + jsonObjectString + " is not a valid JSON", false); + } + } + + /** + * Test SOLR Serialization including values with special characters for AssociationRef + */ + @Test + public void testAssociationRefToJSONString() + { + SOLRSerializer solrSerializer = new SOLRSerializer(); + solrSerializer.setDictionaryService(Mockito.mock(DictionaryService.class)); + solrSerializer.setNamespaceService(Mockito.mock(NamespaceService.class)); + solrSerializer.init(); + AssociationRef associationRef = new AssociationRef( + new NodeRef("workspace://SpacesStore/wo\rld"), + ContentModel.ASSOC_ATTACHMENTS, + new NodeRef("workspace://SpacesStore/hello")); + String validJsonString = solrSerializer.serializeToJSONString(associationRef); + String jsonObjectString = String.format("{ \"key\": \"%s\" }", validJsonString); + + try + { + new JSONObject(jsonObjectString); + } + catch (JSONException e) + { + assertTrue("JSON String " + jsonObjectString + " is not a valid JSON", false); + } + } + } diff --git a/remote-api/src/test/java/org/alfresco/rest/api/tests/AbstractBaseApiTest.java b/remote-api/src/test/java/org/alfresco/rest/api/tests/AbstractBaseApiTest.java index c6399e74a4..4938ecf42e 100644 --- a/remote-api/src/test/java/org/alfresco/rest/api/tests/AbstractBaseApiTest.java +++ b/remote-api/src/test/java/org/alfresco/rest/api/tests/AbstractBaseApiTest.java @@ -51,7 +51,6 @@ import org.alfresco.rest.api.tests.client.data.ContentInfo; import org.alfresco.rest.api.tests.client.data.Document; import org.alfresco.rest.api.tests.client.data.Folder; import org.alfresco.rest.api.tests.client.data.Node; -import org.alfresco.rest.api.tests.client.data.DirectAccessUrl; import org.alfresco.rest.api.tests.client.data.Rendition; import org.alfresco.rest.api.tests.client.data.SiteMember; import org.alfresco.rest.api.tests.client.data.SiteRole; @@ -847,30 +846,6 @@ public abstract class AbstractBaseApiTest extends EnterpriseTestApi return RestApiUtil.parseRestApiEntry(response.getJsonResponse(), Document.class); } - protected DirectAccessUrl requestContentUrl(String nodeId, String body, int expectedStatus) throws Exception - { - HttpResponse response = post(getNodeOperationUrl(nodeId, "request-content-url"), body, null, null, null, expectedStatus); - - if (response.getJsonResponse().get("error") != null) - { - return null; - } - - return RestApiUtil.parseRestApiEntry(response.getJsonResponse(), DirectAccessUrl.class); - } - - protected DirectAccessUrl requestContentUrl(String nodeId, String versionId, String body, int expectedStatus) throws Exception - { - HttpResponse response = post(getNodeVersionsOperationUrl(nodeId, versionId, "request-content-url"), body, null, null, null, expectedStatus); - - if (response.getJsonResponse().get("error") != null) - { - return null; - } - - return RestApiUtil.parseRestApiEntry(response.getJsonResponse(), DirectAccessUrl.class); - } - /** * This test helper method uses "update binary content" to create one or more new versions. The file must already exist. * @@ -1042,21 +1017,6 @@ public abstract class AbstractBaseApiTest extends EnterpriseTestApi return URL_NODES + "/" + nodeId + "/" + URL_RENDITIONS; } - protected String getNodeRenditionOperationUrl(String nodeId, String renditionId, String operation) - { - return URL_NODES + "/" + nodeId + "/" + URL_RENDITIONS+ "/" + renditionId + "/" + operation; - } - - protected String getNodeVersionsUrl(String nodeId, String versionId) - { - return URL_NODES + "/" + nodeId + "/" + URL_VERSIONS + "/" + versionId; - } - - protected String getNodeVersionsOperationUrl(String nodeId, String versionId, String operation) - { - return URL_NODES + "/" + nodeId + "/" + URL_VERSIONS + "/" + versionId + "/" + operation; - } - protected String getNodeVersionsUrl(String nodeId) { return URL_NODES + "/" + nodeId + "/" + URL_VERSIONS; @@ -1067,11 +1027,6 @@ public abstract class AbstractBaseApiTest extends EnterpriseTestApi return URL_NODES + "/" + nodeId + "/" + URL_VERSIONS + "/" + versionId + "/" + URL_RENDITIONS; } - protected String getNodeVersionRenditionsOperationUrl(String nodeId, String versionId, String renditionId, String operation) - { - return URL_NODES + "/" + nodeId + "/" + URL_VERSIONS + "/" + versionId + "/" + URL_RENDITIONS + "/" + renditionId + "/" + operation; - } - protected String getNodeChildrenUrl(String nodeId) { return URL_NODES + "/" + nodeId + "/" + URL_CHILDREN; diff --git a/remote-api/src/test/java/org/alfresco/rest/api/tests/NodeApiTest.java b/remote-api/src/test/java/org/alfresco/rest/api/tests/NodeApiTest.java index 45ca3ba657..74d70932cb 100644 --- a/remote-api/src/test/java/org/alfresco/rest/api/tests/NodeApiTest.java +++ b/remote-api/src/test/java/org/alfresco/rest/api/tests/NodeApiTest.java @@ -76,7 +76,6 @@ import org.alfresco.rest.api.tests.client.PublicApiClient.Paging; import org.alfresco.rest.api.tests.client.PublicApiHttpClient.BinaryPayload; import org.alfresco.rest.api.tests.client.data.Association; import org.alfresco.rest.api.tests.client.data.ContentInfo; -import org.alfresco.rest.api.tests.client.data.DirectAccessUrlRequest; import org.alfresco.rest.api.tests.client.data.Document; import org.alfresco.rest.api.tests.client.data.Folder; import org.alfresco.rest.api.tests.client.data.Node; @@ -103,7 +102,6 @@ import org.alfresco.service.namespace.NamespaceService; import org.alfresco.util.GUID; import org.alfresco.util.TempFileProvider; import org.apache.commons.collections.map.MultiValueMap; -import org.joda.time.DateTime; import org.json.simple.JSONObject; import org.junit.After; import org.junit.Before; @@ -4434,73 +4432,6 @@ public class NodeApiTest extends AbstractSingleNetworkSiteTest deleteNode(folderId); } - @Test - public void testRequestContentUrl() throws Exception - { - setRequestContext(user1); - - // create folder - Folder folderResp = createFolder(Nodes.PATH_MY, "folder" + RUNID); - String folderId = folderResp.getId(); - - // create doc d1 - String d1Name = "content" + RUNID + "_1l"; - Document d1 = createTextFile(folderId, d1Name, "The quick brown fox jumps over the lazy dog 1."); - String nodeId = d1.getId(); - - // node found but direct access isn't available - requestContentUrl(nodeId, null, 501); - - // unknown alias - requestContentUrl("testSomeUndefinedAlias", null, 404); - - // no content - requestContentUrl(Nodes.PATH_MY, null, 400); - requestContentUrl(Nodes.PATH_SHARED, null, 400); - - { - requestContentUrl(nodeId, toJsonAsStringNonNull(new DirectAccessUrlRequest()), 501); - - { - DirectAccessUrlRequest directAccessUrlRequest = new DirectAccessUrlRequest(); - directAccessUrlRequest.setExpiresAt(DateTime.now().plusSeconds(30).toDate()); - directAccessUrlRequest.setValidFor(60L); - - requestContentUrl(nodeId, toJsonAsStringNonNull(directAccessUrlRequest), 400); - } - - { - DirectAccessUrlRequest directAccessUrlRequest = new DirectAccessUrlRequest(); - directAccessUrlRequest.setValidFor(60L); - - requestContentUrl(nodeId, toJsonAsStringNonNull(directAccessUrlRequest), 501); - } - - { - DirectAccessUrlRequest directAccessUrlRequest = new DirectAccessUrlRequest(); - directAccessUrlRequest.setValidFor(-60L); - - requestContentUrl(nodeId, toJsonAsStringNonNull(directAccessUrlRequest), 400); - } - - { - DirectAccessUrlRequest directAccessUrlRequest = new DirectAccessUrlRequest(); - directAccessUrlRequest.setExpiresAt(DateTime.now().plusSeconds(30).toDate()); - requestContentUrl(nodeId, toJsonAsStringNonNull(directAccessUrlRequest), 501); - } - - { - DirectAccessUrlRequest directAccessUrlRequest = new DirectAccessUrlRequest(); - directAccessUrlRequest.setExpiresAt(DateTime.now().minusSeconds(30).toDate()); - requestContentUrl(nodeId, toJsonAsStringNonNull(directAccessUrlRequest), 400); - } - } - - // node not accessible - setRequestContext(user2); - requestContentUrl(nodeId, null, 403); - } - @Test public void testMoveFileCreatedByDeletedUser() throws Exception { diff --git a/remote-api/src/test/java/org/alfresco/rest/api/tests/NodeVersionRenditionsApiTest.java b/remote-api/src/test/java/org/alfresco/rest/api/tests/NodeVersionRenditionsApiTest.java index ffe54788e5..a01cf9e3b8 100644 --- a/remote-api/src/test/java/org/alfresco/rest/api/tests/NodeVersionRenditionsApiTest.java +++ b/remote-api/src/test/java/org/alfresco/rest/api/tests/NodeVersionRenditionsApiTest.java @@ -31,17 +31,14 @@ import org.alfresco.rest.AbstractSingleNetworkSiteTest; import org.alfresco.rest.api.tests.client.HttpResponse; import org.alfresco.rest.api.tests.client.PublicApiClient.Paging; import org.alfresco.rest.api.tests.client.data.ContentInfo; -import org.alfresco.rest.api.tests.client.data.DirectAccessUrlRequest; import org.alfresco.rest.api.tests.client.data.Document; import org.alfresco.rest.api.tests.client.data.Node; import org.alfresco.rest.api.tests.client.data.Rendition; import org.alfresco.rest.api.tests.util.RestApiUtil; import static org.alfresco.rest.api.tests.util.RestApiUtil.toJsonAsString; -import static org.alfresco.rest.api.tests.util.RestApiUtil.toJsonAsStringNonNull; import static org.junit.Assert.*; -import org.joda.time.DateTime; import org.junit.Test; import java.util.*; @@ -369,112 +366,4 @@ public class NodeVersionRenditionsApiTest extends AbstractSingleNetworkSiteTest { return "public"; } - - @Test - public void testRequestContentUrl() throws Exception - { - setRequestContext(user1); - - String myFolderNodeId = getMyNodeId(); - - // create folder - String f1Id = createFolder(myFolderNodeId, "f1").getId(); - - try - { - int verCnt = 1; - int cnt = 1; - String versionLabel = "1.0"; - - String textContentSuffix = "Amazingly few discotheques provide jukeboxes "; - String contentName = "content-2-" + System.currentTimeMillis(); - String content = textContentSuffix + cnt; - - // request minor version on upload (& no pre-request for renditions for live node) - Boolean majorVersion = true; - Map params = new HashMap<>(); - params.put("majorVersion", majorVersion.toString()); - - // create a new file - Document documentResp = createTextFile(f1Id, contentName, content, "UTF-8", params); - String docId = documentResp.getId(); - assertTrue(documentResp.getAspectNames().contains("cm:versionable")); - assertNotNull(documentResp.getProperties()); - assertEquals(versionLabel, documentResp.getProperties().get("cm:versionLabel")); - - cnt = 2; - versionLabel = updateFileVersions(user1, docId, cnt, textContentSuffix, verCnt, majorVersion, versionLabel); - verCnt = verCnt+cnt; - - assertEquals("3.0", versionLabel); - assertEquals(3, verCnt); - - // check version history count - HttpResponse response = getAll(getNodeVersionsUrl(docId), null, null, 200); - List nodes = RestApiUtil.parseRestApiEntries(response.getJsonResponse(), Node.class); - assertEquals(verCnt, nodes.size()); - - // pause briefly - Thread.sleep(DELAY_IN_MS); - - // found but direct access isn't available - checkCreateAndGetVersionRendition(docId, "1.0", "doclib"); - post(getNodeVersionRenditionsOperationUrl(docId, "1.0", "doclib", "request-content-url"), null, null, 501); - - checkCreateAndGetVersionRendition(docId, "3.0", "doclib"); - post(getNodeVersionRenditionsOperationUrl(docId, "3.0", "doclib", "request-content-url"), null, null, 501); - - checkCreateAndGetVersionRendition(docId, "2.0", "doclib"); - post(getNodeVersionRenditionsOperationUrl(docId, "2.0", "doclib", "request-content-url"), null, null, 501); - - { - post(getNodeVersionRenditionsOperationUrl(docId, "2.0", "doclib", "request-content-url"), toJsonAsStringNonNull(new DirectAccessUrlRequest()), null, null, null, - 501); - - { - DirectAccessUrlRequest directAccessUrlRequest = new DirectAccessUrlRequest(); - directAccessUrlRequest.setExpiresAt(DateTime.now().plusSeconds(30).toDate()); - directAccessUrlRequest.setValidFor(60L); - - post(getNodeVersionRenditionsOperationUrl(docId, "2.0", "doclib", "request-content-url"), toJsonAsStringNonNull(directAccessUrlRequest), null, null, null, 400); - } - - { - DirectAccessUrlRequest directAccessUrlRequest = new DirectAccessUrlRequest(); - directAccessUrlRequest.setValidFor(60L); - - post(getNodeVersionRenditionsOperationUrl(docId, "2.0", "doclib", "request-content-url"), toJsonAsStringNonNull(directAccessUrlRequest), null, null, null, 501); - } - - { - DirectAccessUrlRequest directAccessUrlRequest = new DirectAccessUrlRequest(); - directAccessUrlRequest.setValidFor(-60L); - - post(getNodeVersionRenditionsOperationUrl(docId, "2.0", "doclib", "request-content-url"), toJsonAsStringNonNull(directAccessUrlRequest), null, null, null, 400); - } - - { - DirectAccessUrlRequest directAccessUrlRequest = new DirectAccessUrlRequest(); - directAccessUrlRequest.setExpiresAt(DateTime.now().plusSeconds(30).toDate()); - post(getNodeVersionRenditionsOperationUrl(docId, "2.0", "doclib", "request-content-url"), toJsonAsStringNonNull(directAccessUrlRequest), null, null, null, 501); - } - - { - DirectAccessUrlRequest directAccessUrlRequest = new DirectAccessUrlRequest(); - directAccessUrlRequest.setExpiresAt(DateTime.now().minusSeconds(30).toDate()); - post(getNodeVersionRenditionsOperationUrl(docId, "2.0", "doclib", "request-content-url"), toJsonAsStringNonNull(directAccessUrlRequest), null, null, null, 400); - } - } - - // also live node - checkCreateAndGetVersionRendition(docId, null, "doclib"); - post(getNodeVersionRenditionsOperationUrl(docId, null, "doclib", "request-content-url"), null, null, 501); - } - finally - { - // some cleanup - setRequestContext(user1); - deleteNode(f1Id, true, 204); - } - } } diff --git a/remote-api/src/test/java/org/alfresco/rest/api/tests/NodeVersionsApiTest.java b/remote-api/src/test/java/org/alfresco/rest/api/tests/NodeVersionsApiTest.java index ad12ab3ef1..3091136d2b 100644 --- a/remote-api/src/test/java/org/alfresco/rest/api/tests/NodeVersionsApiTest.java +++ b/remote-api/src/test/java/org/alfresco/rest/api/tests/NodeVersionsApiTest.java @@ -33,13 +33,11 @@ import org.alfresco.rest.api.tests.client.HttpResponse; import org.alfresco.rest.api.tests.client.PublicApiClient; import org.alfresco.rest.api.tests.client.PublicApiClient.Paging; import org.alfresco.rest.api.tests.client.PublicApiHttpClient; -import org.alfresco.rest.api.tests.client.data.DirectAccessUrlRequest; import org.alfresco.rest.api.tests.client.data.Document; import org.alfresco.rest.api.tests.client.data.Node; import org.alfresco.rest.api.tests.util.RestApiUtil; import org.alfresco.util.Pair; import org.alfresco.util.TempFileProvider; -import org.joda.time.DateTime; import org.junit.Test; import java.io.ByteArrayInputStream; @@ -1319,91 +1317,6 @@ public class NodeVersionsApiTest extends AbstractSingleNetworkSiteTest } } - @Test - public void testRequestContentUrl() throws Exception - { - // create folder - setRequestContext(user1); - - String f1Id = null; - - try - { - f1Id = createFolder(Nodes.PATH_MY, "testRequestContentUrl-f1").getId(); - - String textContentSuffix = "Amazingly few discotheques provide jukeboxes "; - String contentName = "content-1"; - - int cnt = 6; - Pair pair = uploadTextFileVersions(user1, f1Id, contentName, cnt, textContentSuffix, 0, null, null); - String versionLabel = pair.getFirst(); - String docId = pair.getSecond(); - - assertEquals("1.5", versionLabel); // 1.0, 1.1, ... 1.5 - - // node and version found but direct access isn't available - requestContentUrl(docId, "1.2", null,501); - requestContentUrl(docId, "1.5", null,501); - - // node not found - requestContentUrl("testSomeUndefinedAlias", "1.6", null,404); - - // version not found - requestContentUrl(docId, "1.6", null,404); - - { - requestContentUrl(docId, "1.5", toJsonAsStringNonNull(new DirectAccessUrlRequest()), 501); - - { - DirectAccessUrlRequest directAccessUrlRequest = new DirectAccessUrlRequest(); - directAccessUrlRequest.setExpiresAt(DateTime.now().plusSeconds(30).toDate()); - directAccessUrlRequest.setValidFor(60L); - - requestContentUrl(docId, "1.5", toJsonAsStringNonNull(directAccessUrlRequest), 400); - } - - { - DirectAccessUrlRequest directAccessUrlRequest = new DirectAccessUrlRequest(); - directAccessUrlRequest.setValidFor(60L); - - requestContentUrl(docId, "1.5", toJsonAsStringNonNull(directAccessUrlRequest), 501); - } - - { - DirectAccessUrlRequest directAccessUrlRequest = new DirectAccessUrlRequest(); - directAccessUrlRequest.setValidFor(-60L); - - requestContentUrl(docId, "1.5", toJsonAsStringNonNull(directAccessUrlRequest), 400); - } - - { - DirectAccessUrlRequest directAccessUrlRequest = new DirectAccessUrlRequest(); - directAccessUrlRequest.setExpiresAt(DateTime.now().plusSeconds(30).toDate()); - requestContentUrl(docId, "1.5", toJsonAsStringNonNull(directAccessUrlRequest), 501); - } - - { - DirectAccessUrlRequest directAccessUrlRequest = new DirectAccessUrlRequest(); - directAccessUrlRequest.setExpiresAt(DateTime.now().minusSeconds(30).toDate()); - requestContentUrl(docId, "1.5", toJsonAsStringNonNull(directAccessUrlRequest), 400); - } - } - - // node not accessible - setRequestContext(user2); - requestContentUrl(docId, "1.5", null,403); - } - finally - { - if (f1Id != null) - { - // some cleanup - setRequestContext(user1); - deleteNode(f1Id, true, 204); - } - } - } - @Override public String getScope() { diff --git a/remote-api/src/test/java/org/alfresco/rest/api/tests/RenditionsTest.java b/remote-api/src/test/java/org/alfresco/rest/api/tests/RenditionsTest.java index a8451666b9..b596a37bc2 100644 --- a/remote-api/src/test/java/org/alfresco/rest/api/tests/RenditionsTest.java +++ b/remote-api/src/test/java/org/alfresco/rest/api/tests/RenditionsTest.java @@ -27,7 +27,6 @@ package org.alfresco.rest.api.tests; import static org.alfresco.rest.api.tests.util.RestApiUtil.toJsonAsString; -import static org.alfresco.rest.api.tests.util.RestApiUtil.toJsonAsStringNonNull; import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertFalse; import static org.junit.Assert.assertNotEquals; @@ -47,7 +46,6 @@ import org.alfresco.rest.api.tests.client.PublicApiClient.ExpectedErrorResponse; import org.alfresco.rest.api.tests.client.PublicApiClient.ExpectedPaging; import org.alfresco.rest.api.tests.client.PublicApiClient.Paging; import org.alfresco.rest.api.tests.client.data.ContentInfo; -import org.alfresco.rest.api.tests.client.data.DirectAccessUrlRequest; import org.alfresco.rest.api.tests.client.data.Document; import org.alfresco.rest.api.tests.client.data.Rendition; import org.alfresco.rest.api.tests.client.data.Rendition.RenditionStatus; @@ -59,7 +57,6 @@ import org.alfresco.service.cmr.repository.ContentService; import org.alfresco.service.cmr.site.SiteVisibility; import org.alfresco.service.cmr.thumbnail.ThumbnailService; import org.alfresco.util.TempFileProvider; -import org.joda.time.DateTime; import org.junit.After; import org.junit.Before; import org.junit.Test; @@ -974,121 +971,4 @@ public class RenditionsTest extends AbstractBaseApiTest return synchronousTransformClient.isSupported(MimetypeMap.MIMETYPE_WORD, -1, null, MimetypeMap.MIMETYPE_PDF, Collections.emptyMap(), null, null); } - - @Test - public void testRequestContentUrl() throws Exception - { - setRequestContext(networkN1.getId(), userOneN1.getId(), null); - - // Create a folder within the site document's library - String folderName = "folder" + System.currentTimeMillis(); - String folder_Id = addToDocumentLibrary(userOneN1Site, folderName, TYPE_CM_FOLDER, userOneN1.getId()); - - // Create multipart request - String fileName = "quick.pdf"; - File file = getResourceFile(fileName); - MultiPartBuilder multiPartBuilder = MultiPartBuilder.create().setFileData(new FileData(fileName, file)); - MultiPartRequest reqBody = multiPartBuilder.build(); - - // Upload quick.pdf file into 'folder' - HttpResponse response = post(getNodeChildrenUrl(folder_Id), reqBody.getBody(), null, reqBody.getContentType(), 201); - Document document = RestApiUtil.parseRestApiEntry(response.getJsonResponse(), Document.class); - String contentNodeId = document.getId(); - - // pause briefly - Thread.sleep(DELAY_IN_MS); - - // Get rendition (not created yet) information for node - response = getSingle(getNodeRenditionsUrl(contentNodeId), "doclib", 200); - Rendition rendition = RestApiUtil.parseRestApiEntry(response.getJsonResponse(), Rendition.class); - assertNotNull(rendition); - assertEquals(RenditionStatus.NOT_CREATED, rendition.getStatus()); - - // the rendition hasn't been created yet - post(getNodeRenditionOperationUrl(contentNodeId, rendition.getId(), "request-content-url"), null, null, 404); - - // Create and get 'doclib' rendition - rendition = createAndGetRendition(contentNodeId, "doclib"); - assertNotNull(rendition); - assertEquals(RenditionStatus.CREATED, rendition.getStatus()); - - // found but direct access isn't available - post(getNodeRenditionOperationUrl(contentNodeId, rendition.getId(), "request-content-url"), null, null, 501); - - { - post(getNodeRenditionOperationUrl(contentNodeId, rendition.getId(), "request-content-url"), toJsonAsStringNonNull(new DirectAccessUrlRequest()), null, null, null, 501); - - { - DirectAccessUrlRequest directAccessUrlRequest = new DirectAccessUrlRequest(); - directAccessUrlRequest.setExpiresAt(DateTime.now().plusSeconds(30).toDate()); - directAccessUrlRequest.setValidFor(60L); - - post(getNodeRenditionOperationUrl(contentNodeId, rendition.getId(), "request-content-url"), toJsonAsStringNonNull(directAccessUrlRequest), null, null, null, 400); - } - - { - DirectAccessUrlRequest directAccessUrlRequest = new DirectAccessUrlRequest(); - directAccessUrlRequest.setValidFor(60L); - - post(getNodeRenditionOperationUrl(contentNodeId, rendition.getId(), "request-content-url"), toJsonAsStringNonNull(directAccessUrlRequest), null, null, null, 501); - } - - { - DirectAccessUrlRequest directAccessUrlRequest = new DirectAccessUrlRequest(); - directAccessUrlRequest.setValidFor(-60L); - - post(getNodeRenditionOperationUrl(contentNodeId, rendition.getId(), "request-content-url"), toJsonAsStringNonNull(directAccessUrlRequest), null, null, null, 400); - } - - { - DirectAccessUrlRequest directAccessUrlRequest = new DirectAccessUrlRequest(); - directAccessUrlRequest.setExpiresAt(DateTime.now().plusSeconds(30).toDate()); - post(getNodeRenditionOperationUrl(contentNodeId, rendition.getId(), "request-content-url"), toJsonAsStringNonNull(directAccessUrlRequest), null, null, null, 501); - } - - { - DirectAccessUrlRequest directAccessUrlRequest = new DirectAccessUrlRequest(); - directAccessUrlRequest.setExpiresAt(DateTime.now().minusSeconds(30).toDate()); - post(getNodeRenditionOperationUrl(contentNodeId, rendition.getId(), "request-content-url"), toJsonAsStringNonNull(directAccessUrlRequest), null, null, null, 400); - } - } - - // nodeId in the path parameter does not represent a file - post(getNodeRenditionOperationUrl(folder_Id, "doclib", "request-content-url"), null, null, 400); - - // nodeId in the path parameter does not exist - post(getNodeRenditionOperationUrl(UUID.randomUUID().toString(), "doclib", "request-content-url"), null, null, 404); - - // renditionId in the path parameter is not registered/available - post(getNodeRenditionOperationUrl(contentNodeId, ("renditionId" + System.currentTimeMillis()), "request-content-url"), null, null, 404); - - // Create a node without any content. Test only if OpenOffice is available - if (isOpenOfficeAvailable()) - { - String emptyContentNodeId = addToDocumentLibrary(userOneN1Site, "emptyDoc.txt", TYPE_CM_CONTENT, userOneN1.getId()); - getSingle(getNodeRenditionsUrl(emptyContentNodeId), "doclib", 200); - } - - // Create multipart request - String jpgFileName = "quick.jpg"; - File jpgFile = getResourceFile(fileName); - reqBody = MultiPartBuilder.create().setFileData(new FileData(jpgFileName, jpgFile)).build(); - - // Upload quick.jpg file into 'folder' - response = post(getNodeChildrenUrl(folder_Id), reqBody.getBody(), null, reqBody.getContentType(), 201); - Document jpgImage = RestApiUtil.parseRestApiEntry(response.getJsonResponse(), Document.class); - String jpgImageNodeId = jpgImage.getId(); - - // List all available renditions (includes those that have been created and - // those that are yet to be created) - response = getAll(getNodeRenditionsUrl(jpgImageNodeId), getPaging(0, 50), 200); - List renditions = RestApiUtil.parseRestApiEntries(response.getJsonResponse(), Rendition.class); - // Check there is no pdf rendition is available for the jpg file - Rendition pdf = getRendition(renditions, "pdf"); - assertNull(pdf); - - // The renditionId (pdf) is registered but it is not applicable for the node's - // mimeType - post(getNodeRenditionOperationUrl(jpgImageNodeId, "pdf", "request-content-url"), null, null, 404); - } } diff --git a/remote-api/src/test/java/org/alfresco/rest/api/tests/client/data/DirectAccessUrl.java b/remote-api/src/test/java/org/alfresco/rest/api/tests/client/data/DirectAccessUrl.java deleted file mode 100644 index 38297d459f..0000000000 --- a/remote-api/src/test/java/org/alfresco/rest/api/tests/client/data/DirectAccessUrl.java +++ /dev/null @@ -1,49 +0,0 @@ -/* - * #%L - * Alfresco Remote API - * %% - * Copyright (C) 2005 - 2016 Alfresco Software Limited - * %% - * This file is part of the Alfresco software. - * If the software was purchased under a paid Alfresco license, the terms of - * the paid license agreement will prevail. Otherwise, the software is - * provided under the following open source license terms: - * - * Alfresco is free software: you can redistribute it and/or modify - * it under the terms of the GNU Lesser General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * Alfresco is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public License - * along with Alfresco. If not, see . - * #L% - */ -package org.alfresco.rest.api.tests.client.data; - -import static org.junit.Assert.assertTrue; - -import java.io.Serializable; - -/** - * Represents a direct access content url. - * - */ -public class DirectAccessUrl extends org.alfresco.service.cmr.repository.DirectAccessUrl implements Serializable, ExpectedComparison -{ - - @Override - public void expected(Object o) - { - assertTrue("o is an instance of " + o.getClass(), o instanceof DirectAccessUrl); - - DirectAccessUrl other = (DirectAccessUrl) o; - - AssertUtil.assertEquals("contentUrl", getContentUrl(), other.getContentUrl()); - } - -} diff --git a/remote-api/src/test/java/org/alfresco/rest/api/tests/client/data/DirectAccessUrlRequest.java b/remote-api/src/test/java/org/alfresco/rest/api/tests/client/data/DirectAccessUrlRequest.java deleted file mode 100644 index 90694c9ae6..0000000000 --- a/remote-api/src/test/java/org/alfresco/rest/api/tests/client/data/DirectAccessUrlRequest.java +++ /dev/null @@ -1,50 +0,0 @@ -/* - * #%L - * Alfresco Remote API - * %% - * Copyright (C) 2005 - 2016 Alfresco Software Limited - * %% - * This file is part of the Alfresco software. - * If the software was purchased under a paid Alfresco license, the terms of - * the paid license agreement will prevail. Otherwise, the software is - * provided under the following open source license terms: - * - * Alfresco is free software: you can redistribute it and/or modify - * it under the terms of the GNU Lesser General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * Alfresco is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public License - * along with Alfresco. If not, see . - * #L% - */ -package org.alfresco.rest.api.tests.client.data; - -import static org.junit.Assert.assertTrue; - -import java.io.Serializable; - -/** - * Represents a direct access url request. - * - */ -public class DirectAccessUrlRequest extends org.alfresco.rest.api.model.DirectAccessUrlRequest implements Serializable, ExpectedComparison -{ - - @Override - public void expected(Object o) - { - assertTrue("o is an instance of " + o.getClass(), o instanceof DirectAccessUrlRequest); - - DirectAccessUrlRequest other = (DirectAccessUrlRequest) o; - - AssertUtil.assertEquals("expiresAt", getExpiresAt(), other.getExpiresAt()); - AssertUtil.assertEquals("validFor", getValidFor(), other.getValidFor()); - } - -} diff --git a/repository/.travis.yml b/repository/.travis.yml deleted file mode 100644 index e69de29bb2..0000000000 diff --git a/repository/README.md b/repository/README.md deleted file mode 100644 index ae8a2cd64a..0000000000 --- a/repository/README.md +++ /dev/null @@ -1,41 +0,0 @@ -### Alfresco Repository -[![Build Status](https://travis-ci.com/Alfresco/alfresco-repository.svg?branch=master)](https://travis-ci.com/Alfresco/alfresco-repository) - -Repository is a library packaged as a jar file which is part of [Alfresco Content Services Repository](https://community.alfresco.com/docs/DOC-6385-project-overview-repository). -The library contains the following: -* DAOs and SQL scripts -* Various Service implementations -* Utility classes - -### Building and testing -The project can be built by running Maven command: -~~~ -mvn clean install -~~~ -The tests are combined in test classes split by test type or Spring application context used in the test, see classes in _src/test/java/org/alfresco_. All of these classes as well as individual tests can be run by specifying the test class name and a set of DB connection properties, for example: -~~~ -mvn clean test -Dtest=SomeRepoTest -Ddb.driver=org.postgresql.Driver -Ddb.name=alfresco -Ddb.url=jdbc:postgresql:alfresco -Ddb.username=alfresco -Ddb.password=alfresco -~~~ - -### Artifacts -The artifacts can be obtained by: -* downloading from [Alfresco repository](https://artifacts.alfresco.com/nexus/content/groups/public) -* getting as Maven dependency by adding the dependency to your pom file: -~~~ - - org.alfresco - alfresco-repository - version - -~~~ -and Alfresco Maven repository: -~~~ - - alfresco-maven-repo - https://artifacts.alfresco.com/nexus/content/groups/public - -~~~ -The SNAPSHOT version of the artifact is **never** published. - -### Contributing guide -Please use [this guide](CONTRIBUTING.md) to make a contribution to the project. diff --git a/repository/pom.xml b/repository/pom.xml index f5cf7419d2..ca07029b0a 100644 --- a/repository/pom.xml +++ b/repository/pom.xml @@ -142,7 +142,7 @@ org.mozilla rhino - 1.7.12 + 1.7.13 org.springframework @@ -276,7 +276,6 @@ org.apache.pdfbox pdfbox - ${dependency.pdfbox.version} org.apache.pdfbox @@ -1068,7 +1067,6 @@ org.alfresco alfresco-pdf-renderer - ${dependency.alfresco-pdf-renderer.version} win64 tgz @@ -1113,7 +1111,6 @@ org.alfresco alfresco-pdf-renderer - ${dependency.alfresco-pdf-renderer.version} linux tgz @@ -1150,54 +1147,6 @@ ${project.build.directory}/test-binaries/alfresco-pdf-renderer/alfresco-pdf-renderer - - osx-alfresco-pdf-renderer-test - - - mac - - - - - org.alfresco - alfresco-pdf-renderer - ${dependency.alfresco-pdf-renderer.version} - osx - tgz - - - - - - false - maven-antrun-plugin - - - extract-alfresco-pdf-renderer-test - generate-test-resources - - run - - - ${skipTests} - - - - - - - - - - - - - - - - ${project.build.directory}/test-binaries/alfresco-pdf-renderer/alfresco-pdf-renderer - - diff --git a/repository/src/main/java/org/alfresco/opencmis/TransactionAwareHolder.java b/repository/src/main/java/org/alfresco/opencmis/TransactionAwareHolder.java index a5c5991583..e35d75d413 100644 --- a/repository/src/main/java/org/alfresco/opencmis/TransactionAwareHolder.java +++ b/repository/src/main/java/org/alfresco/opencmis/TransactionAwareHolder.java @@ -81,22 +81,19 @@ public class TransactionAwareHolder extends Holder { this.internalHolder = internalHolder; this.value = internalHolder.getValue(); - txListener = new TxAwareHolderListener(); } @Override public T getValue() { - if (TransactionSynchronizationManager.isSynchronizationActive()) - { - AlfrescoTransactionSupport.bindListener(txListener); - } + registerTxListenerIfNeeded(); return this.value; } @Override public void setValue(T value) { + registerTxListenerIfNeeded(); this.value = value; } @@ -109,6 +106,17 @@ public class TransactionAwareHolder extends Holder '}'; } + // MNT-21800 CMIS Web Service Check Out returns error + private void registerTxListenerIfNeeded() + { + if (this.txListener == null && TransactionSynchronizationManager.isSynchronizationActive()) + { + TxAwareHolderListener listener = new TxAwareHolderListener(); + AlfrescoTransactionSupport.bindListener(listener); + this.txListener = listener; + } + } + private class TxAwareHolderListener extends TransactionListenerAdapter { @Override diff --git a/repository/src/main/java/org/alfresco/repo/domain/permissions/ADMAccessControlListDAO.java b/repository/src/main/java/org/alfresco/repo/domain/permissions/ADMAccessControlListDAO.java index 774cccc9bd..8d7b54d07d 100644 --- a/repository/src/main/java/org/alfresco/repo/domain/permissions/ADMAccessControlListDAO.java +++ b/repository/src/main/java/org/alfresco/repo/domain/permissions/ADMAccessControlListDAO.java @@ -430,8 +430,8 @@ public class ADMAccessControlListDAO implements AccessControlListDAO // { // setFixedAcls(child.getId(), inheritFrom, mergeFrom, sharedAclToReplace, changes, false); // } - // Already replaced - if(acl.equals(sharedAclToReplace)) + // Still has old shared ACL or already replaced + if(acl.equals(sharedAclToReplace) || acl.equals(mergeFrom)) { propagateOnChildren = setFixAclPending(child.getId(), inheritFrom, mergeFrom, sharedAclToReplace, changes, false, asyncCall, propagateOnChildren); } diff --git a/repository/src/main/java/org/alfresco/repo/security/permissions/impl/SimplePermissionReference.java b/repository/src/main/java/org/alfresco/repo/security/permissions/impl/SimplePermissionReference.java index e34f22b0c8..9ff1ebc3ab 100644 --- a/repository/src/main/java/org/alfresco/repo/security/permissions/impl/SimplePermissionReference.java +++ b/repository/src/main/java/org/alfresco/repo/security/permissions/impl/SimplePermissionReference.java @@ -1,35 +1,35 @@ -/* - * #%L - * Alfresco Repository - * %% - * Copyright (C) 2005 - 2016 Alfresco Software Limited - * %% - * This file is part of the Alfresco software. - * If the software was purchased under a paid Alfresco license, the terms of - * the paid license agreement will prevail. Otherwise, the software is - * provided under the following open source license terms: - * - * Alfresco is free software: you can redistribute it and/or modify - * it under the terms of the GNU Lesser General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * Alfresco is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public License - * along with Alfresco. If not, see . - * #L% - */ +/* + * #%L + * Alfresco Repository + * %% + * Copyright (C) 2005 - 2020 Alfresco Software Limited + * %% + * This file is part of the Alfresco software. + * If the software was purchased under a paid Alfresco license, the terms of + * the paid license agreement will prevail. Otherwise, the software is + * provided under the following open source license terms: + * + * Alfresco is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * Alfresco is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with Alfresco. If not, see . + * #L% + */ package org.alfresco.repo.security.permissions.impl; -import java.util.HashMap; -import java.util.concurrent.locks.ReadWriteLock; -import java.util.concurrent.locks.ReentrantReadWriteLock; +import java.util.concurrent.ConcurrentHashMap; +import java.util.concurrent.ConcurrentMap; import org.alfresco.service.namespace.QName; +import org.alfresco.util.Pair; /** * A simple permission reference. @@ -37,62 +37,30 @@ import org.alfresco.service.namespace.QName; * @author andyh */ public final class SimplePermissionReference extends AbstractPermissionReference -{ +{ private static final long serialVersionUID = 637302438293417818L; - private static ReadWriteLock lock = new ReentrantReadWriteLock(); - - private static HashMap> instances = new HashMap>(); + //Use thread-safe map initiallized with a slightly larger capacity to reduce the posibility of two or more threads attempting to resize at the same time + private static ConcurrentMap, SimplePermissionReference> instances = new ConcurrentHashMap<>(100, 0.9f, 2); /** - * Factory method to create simple permission refrences + * Factory method to create simple permission references * * @return a simple permission reference */ public static SimplePermissionReference getPermissionReference(QName qName, String name) { - lock.readLock().lock(); - try - { - HashMap typed = instances.get(qName); - if(typed != null) + Pair key = new Pair<>(qName, name); + SimplePermissionReference instance = instances.get(key); + if (instance == null) { - SimplePermissionReference instance = typed.get(name); - if(instance != null) - { - return instance; - } - } - } - finally - { - lock.readLock().unlock(); - } - - lock.writeLock().lock(); - try - { - HashMap typed = instances.get(qName); - if(typed == null) - { - typed = new HashMap(); - instances.put(qName, typed); - } - SimplePermissionReference instance = typed.get(name); - if(instance == null) - { - instance = new SimplePermissionReference(qName, name); - typed.put(name, instance); + instance = new SimplePermissionReference(qName, name); + instances.putIfAbsent(key, instance); } + return instance; - } - finally - { - lock.writeLock().unlock(); - } } - /* * The type */ diff --git a/repository/src/main/java/org/alfresco/repo/solr/SOLRTrackingComponentImpl.java b/repository/src/main/java/org/alfresco/repo/solr/SOLRTrackingComponentImpl.java index 1f202b250a..679c584fe8 100644 --- a/repository/src/main/java/org/alfresco/repo/solr/SOLRTrackingComponentImpl.java +++ b/repository/src/main/java/org/alfresco/repo/solr/SOLRTrackingComponentImpl.java @@ -690,20 +690,18 @@ public class SOLRTrackingComponentImpl implements SearchTrackingComponent return new ArrayList(visited); } - + /** Get properties that we want to be indexed. */ protected Map getProperties(Long nodeId) { - Map props = null; - // ALF-10641 - // Residual properties are un-indexed -> break serlialisation + // Residual properties are un-indexed -> break serialisation nodeDAO.setCheckNodeConsistency(); Map sourceProps = nodeDAO.getNodeProperties(nodeId); - props = new HashMap((int)(sourceProps.size() * 1.3)); + Map props = new HashMap<>(sourceProps.size()); for(QName propertyQName : sourceProps.keySet()) { PropertyDefinition propDef = dictionaryService.getProperty(propertyQName); - if(propDef != null) + if(propDef != null && propDef.isIndexed()) { props.put(propertyQName, sourceProps.get(propertyQName)); } diff --git a/repository/src/main/resources/alfresco/dbscripts/create/org.alfresco.repo.domain.dialect.MySQLInnoDBDialect/AlfrescoCreate-RepoTables.sql b/repository/src/main/resources/alfresco/dbscripts/create/org.alfresco.repo.domain.dialect.MySQLInnoDBDialect/AlfrescoCreate-RepoTables.sql index 2599697199..76a1107baa 100644 --- a/repository/src/main/resources/alfresco/dbscripts/create/org.alfresco.repo.domain.dialect.MySQLInnoDBDialect/AlfrescoCreate-RepoTables.sql +++ b/repository/src/main/resources/alfresco/dbscripts/create/org.alfresco.repo.domain.dialect.MySQLInnoDBDialect/AlfrescoCreate-RepoTables.sql @@ -169,7 +169,9 @@ CREATE TABLE alf_transaction change_txn_id VARCHAR(56) NOT NULL, commit_time_ms BIGINT, PRIMARY KEY (id), - KEY idx_alf_txn_ctms (commit_time_ms) + KEY idx_alf_txn_ctms (commit_time_ms, id), + KEY idx_alf_txn_ctms_sc (commit_time_ms), + key idx_alf_txn_id_ctms (id, commit_time_ms) ) ENGINE=InnoDB; CREATE TABLE alf_store @@ -210,6 +212,8 @@ CREATE TABLE alf_node KEY idx_alf_node_crd (audit_created, store_id, type_qname_id), KEY idx_alf_node_mor (audit_modifier, store_id, type_qname_id), KEY idx_alf_node_mod (audit_modified, store_id, type_qname_id), + KEY idx_alf_node_ver (version), + KEY idx_alf_node_txn (transaction_id), CONSTRAINT fk_alf_node_acl FOREIGN KEY (acl_id) REFERENCES alf_access_control_list (id), CONSTRAINT fk_alf_node_store FOREIGN KEY (store_id) REFERENCES alf_store (id), CONSTRAINT fk_alf_node_tqn FOREIGN KEY (type_qname_id) REFERENCES alf_qname (id), diff --git a/repository/src/main/resources/alfresco/dbscripts/create/org.alfresco.repo.domain.dialect.MySQLInnoDBDialect/Schema-Reference-ALF.xml b/repository/src/main/resources/alfresco/dbscripts/create/org.alfresco.repo.domain.dialect.MySQLInnoDBDialect/Schema-Reference-ALF.xml index ae71a60993..7d6c902e4d 100644 --- a/repository/src/main/resources/alfresco/dbscripts/create/org.alfresco.repo.domain.dialect.MySQLInnoDBDialect/Schema-Reference-ALF.xml +++ b/repository/src/main/resources/alfresco/dbscripts/create/org.alfresco.repo.domain.dialect.MySQLInnoDBDialect/Schema-Reference-ALF.xml @@ -1681,6 +1681,16 @@ type_qname_id + + + version + + + + + transaction_id + + @@ -2630,6 +2640,18 @@ commit_time_ms + id + + + + + commit_time_ms + + + + + id + commit_time_ms diff --git a/repository/src/main/resources/alfresco/dbscripts/create/org.alfresco.repo.domain.dialect.PostgreSQLDialect/AlfrescoCreate-RepoTables.sql b/repository/src/main/resources/alfresco/dbscripts/create/org.alfresco.repo.domain.dialect.PostgreSQLDialect/AlfrescoCreate-RepoTables.sql index 393e9accec..364625ea54 100644 --- a/repository/src/main/resources/alfresco/dbscripts/create/org.alfresco.repo.domain.dialect.PostgreSQLDialect/AlfrescoCreate-RepoTables.sql +++ b/repository/src/main/resources/alfresco/dbscripts/create/org.alfresco.repo.domain.dialect.PostgreSQLDialect/AlfrescoCreate-RepoTables.sql @@ -184,6 +184,8 @@ CREATE TABLE alf_transaction PRIMARY KEY (id) ); CREATE INDEX idx_alf_txn_ctms ON alf_transaction (commit_time_ms, id); +CREATE INDEX idx_alf_txn_ctms_sc ON alf_transaction (commit_time_ms); +CREATE INDEX idx_alf_txn_id_ctms ON alf_transaction (id, commit_time_ms); CREATE SEQUENCE alf_store_seq START WITH 1 INCREMENT BY 1; CREATE TABLE alf_store @@ -231,6 +233,8 @@ CREATE INDEX fk_alf_node_acl ON alf_node (acl_id); CREATE INDEX fk_alf_node_store ON alf_node (store_id); CREATE INDEX idx_alf_node_tqn ON alf_node (type_qname_id, store_id, id); CREATE INDEX fk_alf_node_loc ON alf_node (locale_id); +CREATE INDEX idx_alf_node_ver ON alf_node (version); +CREATE INDEX idx_alf_node_txn ON alf_node (transaction_id); CREATE INDEX fk_alf_store_root ON alf_store (root_node_id); ALTER TABLE alf_store ADD CONSTRAINT fk_alf_store_root FOREIGN KEY (root_node_id) REFERENCES alf_node (id); diff --git a/repository/src/main/resources/alfresco/dbscripts/create/org.alfresco.repo.domain.dialect.PostgreSQLDialect/Schema-Reference-ALF.xml b/repository/src/main/resources/alfresco/dbscripts/create/org.alfresco.repo.domain.dialect.PostgreSQLDialect/Schema-Reference-ALF.xml index b4fc5abffd..312c35c6d5 100644 --- a/repository/src/main/resources/alfresco/dbscripts/create/org.alfresco.repo.domain.dialect.PostgreSQLDialect/Schema-Reference-ALF.xml +++ b/repository/src/main/resources/alfresco/dbscripts/create/org.alfresco.repo.domain.dialect.PostgreSQLDialect/Schema-Reference-ALF.xml @@ -1739,6 +1739,16 @@ id + + + version + + + + + transaction_id + +
@@ -2703,6 +2713,17 @@ id + + + commit_time_ms + + + + + id + commit_time_ms + +
diff --git a/repository/src/main/resources/alfresco/dbscripts/db-schema-context.xml b/repository/src/main/resources/alfresco/dbscripts/db-schema-context.xml index c8ef744642..786ff0e5d0 100644 --- a/repository/src/main/resources/alfresco/dbscripts/db-schema-context.xml +++ b/repository/src/main/resources/alfresco/dbscripts/db-schema-context.xml @@ -55,6 +55,7 @@ + diff --git a/repository/src/main/resources/alfresco/dbscripts/upgrade/6.3/org.alfresco.repo.domain.dialect.MySQLInnoDBDialect/add-indexes-node-transaction.sql b/repository/src/main/resources/alfresco/dbscripts/upgrade/6.3/org.alfresco.repo.domain.dialect.MySQLInnoDBDialect/add-indexes-node-transaction.sql new file mode 100644 index 0000000000..9040ee0d81 --- /dev/null +++ b/repository/src/main/resources/alfresco/dbscripts/upgrade/6.3/org.alfresco.repo.domain.dialect.MySQLInnoDBDialect/add-indexes-node-transaction.sql @@ -0,0 +1,35 @@ +-- +-- Title: Update alf_node and alf_transaction indexes for more performance +-- Database: MySQL +-- Since: V6.3 +-- Author: Eva Vasques +-- +-- Please contact support@alfresco.com if you need assistance with the upgrade. +-- + +DROP INDEX idx_alf_node_ver; --(optional) +CREATE INDEX idx_alf_node_ver ON alf_node (version); + +DROP INDEX idx_alf_node_txn; --(optional) +CREATE INDEX idx_alf_node_txn ON alf_node (transaction_id); + +DROP INDEX idx_alf_txn_ctms; --(optional) +CREATE INDEX idx_alf_txn_ctms ON alf_transaction (commit_time_ms, id); + +DROP INDEX idx_alf_txn_ctms_sc; --(optional) +CREATE INDEX idx_alf_txn_ctms_sc ON alf_transaction (commit_time_ms); + +DROP INDEX idx_alf_txn_id_ctms; --(optional) +CREATE INDEX idx_alf_txn_id_ctms ON alf_transaction (id, commit_time_ms); + +-- +-- Record script finish +-- +DELETE FROM alf_applied_patch WHERE id = 'patch.db-V6.3-add-indexes-node-transaction'; +INSERT INTO alf_applied_patch + (id, description, fixes_from_schema, fixes_to_schema, applied_to_schema, target_schema, applied_on_date, applied_to_server, was_executed, succeeded, report) + VALUES + ( + 'patch.db-V6.3-add-indexes-node-transaction', 'Create aditional indexes on alf_node and alf_transaction', + 0, 14001, -1, 14002, null, 'UNKNOWN', ${TRUE}, ${TRUE}, 'Script completed' + ); \ No newline at end of file diff --git a/repository/src/main/resources/alfresco/dbscripts/upgrade/6.3/org.alfresco.repo.domain.dialect.PostgreSQLDialect/add-indexes-node-transaction.sql b/repository/src/main/resources/alfresco/dbscripts/upgrade/6.3/org.alfresco.repo.domain.dialect.PostgreSQLDialect/add-indexes-node-transaction.sql new file mode 100644 index 0000000000..10fba2d958 --- /dev/null +++ b/repository/src/main/resources/alfresco/dbscripts/upgrade/6.3/org.alfresco.repo.domain.dialect.PostgreSQLDialect/add-indexes-node-transaction.sql @@ -0,0 +1,32 @@ +-- +-- Title: Update alf_node and alf_transaction indexes for more performance +-- Database: PostgreSQL +-- Since: V6.3 +-- Author: Eva Vasques +-- +-- Please contact support@alfresco.com if you need assistance with the upgrade. +-- + +DROP INDEX idx_alf_node_ver; --(optional) +CREATE INDEX idx_alf_node_ver ON alf_node (version); + +DROP INDEX idx_alf_node_txn; --(optional) +CREATE INDEX idx_alf_node_txn ON alf_node (transaction_id); + +DROP INDEX idx_alf_txn_ctms_sc; --(optional) +CREATE INDEX idx_alf_txn_ctms_sc ON alf_transaction (commit_time_ms); + +DROP INDEX idx_alf_txn_id_ctms; --(optional) +CREATE INDEX idx_alf_txn_id_ctms ON alf_transaction (id, commit_time_ms); + +-- +-- Record script finish +-- +DELETE FROM alf_applied_patch WHERE id = 'patch.db-V6.3-add-indexes-node-transaction'; +INSERT INTO alf_applied_patch + (id, description, fixes_from_schema, fixes_to_schema, applied_to_schema, target_schema, applied_on_date, applied_to_server, was_executed, succeeded, report) + VALUES + ( + 'patch.db-V6.3-add-indexes-node-transaction', 'Create aditional indexes on alf_node and alf_transaction', + 0, 14001, -1, 14002, null, 'UNKNOWN', ${TRUE}, ${TRUE}, 'Script completed' + ); \ No newline at end of file diff --git a/repository/src/main/resources/alfresco/messages/patch-service.properties b/repository/src/main/resources/alfresco/messages/patch-service.properties index 4b90cc7e23..915718a744 100644 --- a/repository/src/main/resources/alfresco/messages/patch-service.properties +++ b/repository/src/main/resources/alfresco/messages/patch-service.properties @@ -402,4 +402,6 @@ patch.db-V5.2-remove-jbpm-tables-from-db.description=Removes all JBPM related ta patch.db-V6.0-change-set-indexes.description=Add additional indexes to support acl tracking. -patch.db-V6.3-remove-alf_server-table.description=Remove alf_server table. \ No newline at end of file +patch.db-V6.3-remove-alf_server-table.description=Remove alf_server table. + +patch.db-V6.3-add-indexes-node-transaction.description=Create additional indexes on alf_node and alf_transaction \ No newline at end of file diff --git a/repository/src/main/resources/alfresco/patch/patch-services-context.xml b/repository/src/main/resources/alfresco/patch/patch-services-context.xml index 797813a1a4..6bc9a5a6a4 100644 --- a/repository/src/main/resources/alfresco/patch/patch-services-context.xml +++ b/repository/src/main/resources/alfresco/patch/patch-services-context.xml @@ -1408,4 +1408,16 @@ classpath:alfresco/dbscripts/upgrade/6.3/${db.script.dialect}/remove-alf_server-table.sql + + + patch.db-V6.3-add-indexes-node-transaction + patch.db-V6.3-add-indexes-node-transaction.description + 0 + 14001 + 14002 + ${system.new-node-transaction-indexes.ignored} + + classpath:alfresco/dbscripts/upgrade/6.3/${db.script.dialect}/add-indexes-node-transaction.sql + + diff --git a/repository/src/main/resources/alfresco/repository.properties b/repository/src/main/resources/alfresco/repository.properties index 773f76d83b..dc06cb26f6 100644 --- a/repository/src/main/resources/alfresco/repository.properties +++ b/repository/src/main/resources/alfresco/repository.properties @@ -3,7 +3,7 @@ repository.name=Main Repository # Schema number -version.schema=14001 +version.schema=14002 # Directory configuration @@ -1332,4 +1332,7 @@ system.prop_table_cleaner.algorithm=V2 # Configure the expiration time of the direct access url. This is the length of time in seconds that the link is valid for. # Note: It is up to the actual ContentStore implementation if it can fulfil this request or not. -alfresco.content.directAccessUrl.lifetimeInSec=300 \ No newline at end of file +alfresco.content.directAccessUrl.lifetimeInSec=300 + +# Creates additional indexes on alf_node and alf_transaction. Recommended for large repositories. +system.new-node-transaction-indexes.ignored=true \ No newline at end of file diff --git a/repository/src/main/resources/alfresco/subsystems/Search/solr4/solr-search-context.xml b/repository/src/main/resources/alfresco/subsystems/Search/solr4/solr-search-context.xml index c111fb94ac..52f5a18f82 100644 --- a/repository/src/main/resources/alfresco/subsystems/Search/solr4/solr-search-context.xml +++ b/repository/src/main/resources/alfresco/subsystems/Search/solr4/solr-search-context.xml @@ -153,18 +153,15 @@ - - - - - - - + + + + diff --git a/repository/src/test/java/org/alfresco/repo/domain/permissions/FixedAclUpdaterTest.java b/repository/src/test/java/org/alfresco/repo/domain/permissions/FixedAclUpdaterTest.java index 16b787afb2..70566e8d0f 100644 --- a/repository/src/test/java/org/alfresco/repo/domain/permissions/FixedAclUpdaterTest.java +++ b/repository/src/test/java/org/alfresco/repo/domain/permissions/FixedAclUpdaterTest.java @@ -25,7 +25,10 @@ */ package org.alfresco.repo.domain.permissions; +import java.util.ArrayList; +import java.util.Collections; import java.util.HashSet; +import java.util.List; import java.util.Set; import org.alfresco.model.ContentModel; @@ -41,6 +44,7 @@ import org.alfresco.repo.transaction.RetryingTransactionHelper.RetryingTransacti import org.alfresco.repo.transaction.TransactionListenerAdapter; import org.alfresco.service.ServiceRegistry; import org.alfresco.service.cmr.model.FileFolderService; +import org.alfresco.service.cmr.model.FileInfo; import org.alfresco.service.cmr.repository.NodeRef; import org.alfresco.service.cmr.security.PermissionService; import org.alfresco.service.namespace.QName; @@ -69,6 +73,7 @@ public class FixedAclUpdaterTest extends TestCase private Repository repository; private FixedAclUpdater fixedAclUpdater; private NodeRef folderAsyncCallNodeRef; + private NodeRef folderAsyncCallWithCreateNodeRef; private NodeRef folderSyncCallNodeRef; private PermissionsDaoComponent permissionsDaoComponent; private PermissionService permissionService; @@ -100,6 +105,10 @@ public class FixedAclUpdaterTest extends TestCase filesPerLevel); folderSyncCallNodeRef = txnHelper.doInTransaction(cb2); + RetryingTransactionCallback cb3 = createFolderHierchyCallback(home, fileFolderService, + "rootFolderAsyncWithCreateCall", filesPerLevel); + folderAsyncCallWithCreateNodeRef = txnHelper.doInTransaction(cb3); + // change setFixedAclMaxTransactionTime to lower value so setInheritParentPermissions on created folder // hierarchy require async call setFixedAclMaxTransactionTime(permissionsDaoComponent, home, 50); @@ -147,8 +156,10 @@ public class FixedAclUpdaterTest extends TestCase aspect.add(ContentModel.ASPECT_TEMPORARY); nodeDAO.addNodeAspects(nodeDAO.getNodePair(folderAsyncCallNodeRef).getFirst(), aspect); nodeDAO.addNodeAspects(nodeDAO.getNodePair(folderSyncCallNodeRef).getFirst(), aspect); + nodeDAO.addNodeAspects(nodeDAO.getNodePair(folderAsyncCallWithCreateNodeRef).getFirst(), aspect); fileFolderService.delete(folderAsyncCallNodeRef); fileFolderService.delete(folderSyncCallNodeRef); + fileFolderService.delete(folderAsyncCallWithCreateNodeRef); return null; }, false, true); } @@ -189,7 +200,34 @@ public class FixedAclUpdaterTest extends TestCase testWork(folderAsyncCallNodeRef, true); } + @Test + public void testAsyncWithNodeCreation() + { + testWorkWithNodeCreation(folderAsyncCallWithCreateNodeRef, true); + } + private void testWork(NodeRef folderRef, boolean asyncCall) + { + setPermissionsOnTree(folderRef, asyncCall); + triggerFixedACLJob(folderRef); + } + + private void testWorkWithNodeCreation(NodeRef folderRef, boolean asyncCall) + { + setPermissionsOnTree(folderRef, asyncCall); + + // MNT-21847 - Create a new content in folder that has the aspect applied + txnHelper.doInTransaction((RetryingTransactionCallback) () -> { + NodeRef folderWithPendingAcl = getFirstFolderWithAclPending(folderRef); + assertNotNull("No children folders were found with pendingFixACl aspect", folderWithPendingAcl); + createFile(fileFolderService, folderWithPendingAcl, "NewFile", ContentModel.TYPE_CONTENT); + return null; + }, false, true); + + triggerFixedACLJob(folderRef); + } + + private void setPermissionsOnTree(NodeRef folderRef, boolean asyncCall) { // kick it off by setting inherit parent permissions == false txnHelper.doInTransaction((RetryingTransactionCallback) () -> { @@ -203,21 +241,61 @@ public class FixedAclUpdaterTest extends TestCase assertTrue("There are no nodes to process", getNodesCountWithPendingFixedAclAspect() > 0); return null; }, false, true); + } - // run the fixedAclUpdater until there is nothing more to fix (running the updater - // may create more to fix up) + private void triggerFixedACLJob(NodeRef folder) + { + // run the fixedAclUpdater until there is nothing more to fix (running the updater may create more to fix up) or + // the count doesn't change, meaning we have a problem. txnHelper.doInTransaction((RetryingTransactionCallback) () -> { int count = 0; + int previousCount = 0; do { + previousCount = count; count = fixedAclUpdater.execute(); - } while (count > 0); + } while (count > 0 && previousCount != count); return null; }, false, true); // check if nodes with ASPECT_PENDING_FIX_ACL are processed txnHelper.doInTransaction((RetryingTransactionCallback) () -> { assertEquals("Not all nodes were processed", 0, getNodesCountWithPendingFixedAclAspect()); + //Remove the tree that failed so it does not influence the other test results + removeNodesWithPendingAcl(folder); + return null; + }, false, true); + } + + private NodeRef getFirstFolderWithAclPending(NodeRef parentNodeRef) + { + NodeRef folderWithPendingFixedAcl = null; + List primaryChildFolders = fileFolderService.listFolders(parentNodeRef); + for (int i = 0; i < primaryChildFolders.size(); i++) + { + NodeRef thisChildFolder = primaryChildFolders.get(i).getNodeRef(); + Long thisChildNodeId = nodeDAO.getNodePair(thisChildFolder).getFirst(); + if (nodeDAO.hasNodeAspect(thisChildNodeId, ContentModel.ASPECT_PENDING_FIX_ACL)) + { + folderWithPendingFixedAcl = thisChildFolder; + break; + } + + if (folderWithPendingFixedAcl == null) + { + folderWithPendingFixedAcl = getFirstFolderWithAclPending(thisChildFolder); + } + } + return folderWithPendingFixedAcl; + } + + private void removeNodesWithPendingAcl(NodeRef folder) + { + txnHelper.doInTransaction((RetryingTransactionCallback) () -> { + Set aspect = new HashSet<>(); + aspect.add(ContentModel.ASPECT_TEMPORARY); + nodeDAO.addNodeAspects(nodeDAO.getNodePair(folder).getFirst(), aspect); + fileFolderService.delete(folder); return null; }, false, true); } diff --git a/repository/src/test/java/org/alfresco/repo/solr/SOLRTrackingComponentUnitTest.java b/repository/src/test/java/org/alfresco/repo/solr/SOLRTrackingComponentUnitTest.java new file mode 100644 index 0000000000..316f5398f0 --- /dev/null +++ b/repository/src/test/java/org/alfresco/repo/solr/SOLRTrackingComponentUnitTest.java @@ -0,0 +1,115 @@ +/* + * #%L + * Alfresco Repository + * %% + * Copyright (C) 2005 - 2016 Alfresco Software Limited + * %% + * This file is part of the Alfresco software. + * If the software was purchased under a paid Alfresco license, the terms of + * the paid license agreement will prevail. Otherwise, the software is + * provided under the following open source license terms: + * + * Alfresco is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * Alfresco is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with Alfresco. If not, see . + * #L% + */ +package org.alfresco.repo.solr; + +import static java.util.Collections.emptyMap; + +import static org.junit.Assert.assertEquals; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; +import static org.mockito.MockitoAnnotations.initMocks; + +import java.io.Serializable; +import java.util.Map; + +import org.alfresco.repo.domain.node.NodeDAO; +import org.alfresco.service.cmr.dictionary.DictionaryService; +import org.alfresco.service.cmr.dictionary.PropertyDefinition; +import org.alfresco.service.namespace.QName; +import org.junit.Before; +import org.junit.Test; +import org.mockito.InjectMocks; +import org.mockito.Mock; + +/** Unit tests for {@link org.alfresco.repo.solr.SOLRTrackingComponent}. */ +public class SOLRTrackingComponentUnitTest +{ + /** A pair of QNames for use in the tests. */ + private static final QName FIRST_PROPERTY = QName.createQName("the://first/property"); + private static final QName SECOND_PROPERTY = QName.createQName("the://second/property"); + /** A node id for use in the tests. */ + private static final long NODE_ID = 123L; + + /** The class under test. */ + @InjectMocks + private SOLRTrackingComponentImpl solrTrackingComponent; + @Mock + private NodeDAO nodeDAO; + @Mock + private DictionaryService dictionaryService; + + @Before + public void setUp() + { + initMocks(this); + } + + /** Check that properties of different types can be returned. */ + @Test + public void testGetProperties_indexedPropertiesPassedThrough() + { + Map propertiesFromDB = Map.of(FIRST_PROPERTY, "value1", SECOND_PROPERTY, 2); + when(nodeDAO.getNodeProperties(NODE_ID)).thenReturn(propertiesFromDB); + PropertyDefinition firstDefinition = mock(PropertyDefinition.class); + when(firstDefinition.isIndexed()).thenReturn(true); + when(dictionaryService.getProperty(FIRST_PROPERTY)).thenReturn(firstDefinition); + PropertyDefinition secondDefinition = mock(PropertyDefinition.class); + when(secondDefinition.isIndexed()).thenReturn(true); + when(dictionaryService.getProperty(SECOND_PROPERTY)).thenReturn(secondDefinition); + + Map properties = solrTrackingComponent.getProperties(NODE_ID); + + assertEquals("Expected both properties to be returned.", propertiesFromDB, properties); + } + + /** Check that a property is not indexed if it is not registered in the dictionary service. */ + @Test + public void testGetProperties_propertyWithoutModelIsNotIndexed() + { + Map propertiesFromDB = Map.of(FIRST_PROPERTY, "value1"); + when(nodeDAO.getNodeProperties(NODE_ID)).thenReturn(propertiesFromDB); + when(dictionaryService.getProperty(FIRST_PROPERTY)).thenReturn(null); + + Map properties = solrTrackingComponent.getProperties(NODE_ID); + + assertEquals("Expected residual property to be skipped.", emptyMap(), properties); + } + + /** Check that a property is not indexed if the model contains */ + @Test + public void testGetProperties_propertySkippedIfIndexFalseSet() + { + Map propertiesFromDB = Map.of(FIRST_PROPERTY, "value1"); + when(nodeDAO.getNodeProperties(NODE_ID)).thenReturn(propertiesFromDB); + PropertyDefinition firstDefinition = mock(PropertyDefinition.class); + when(firstDefinition.isIndexed()).thenReturn(false); + when(dictionaryService.getProperty(FIRST_PROPERTY)).thenReturn(firstDefinition); + + Map properties = solrTrackingComponent.getProperties(NODE_ID); + + assertEquals("Unexpected property when index enabled set to false.", emptyMap(), properties); + } +}