* Added alternative approach for DBQueryEngine to page through the entire query using "LIMIT" requests instead of result set streaming
* Use the alternative approach for MySql
Having added AGS to the community repo project two of the build jobs were take a long time to run in comparison to others:
AGS Unit & Integration Tests (MySQL) ~ 1hr
AGS Unit & Integration Tests (PostgreSQL) ~ 45mins
Having split the jobs in this commit, the individual job times are now comparable to the other jobs and the total build time is reduced by ~30 mins.
The same number of integration tests (418) are run 196+127+95 across the three jobs for each database.
* Bug Fix: Extended the branch check for tas jobs to release branches in addition to master
* Provides flexibility to enable or disable test jobs that are disabled or enabled based on the branch.
Test the build with the existing [skip db] and [ags] too
(cherry picked from commit f286d233d9)
Removed trigger of downstream project on feature branch to save agents
(cherry picked from commit f6c87dddbe)
# Conflicts:
# scripts/travis/maven_release.sh
- fix messaging context for tests by adding missing mocked beans and properties
- add tests for amqp, jms, activemq protocols
- add travis job to run messaging tests
ACS-1180 MySQL 8
ACS-1183 PostgreSQL 12.4, 13.1
ACS-1184 MariaDB 10.4, 10.5
Does not include Oracle or MS SQL Server which require new images.
Order of DB test changed to to be the same as alfresco-enterprise-repo.
Now that the T-Engines provide parity with in process legacy transforms and metadata extract, the legacy transform code is now being removed. This will allow the documentation to be greatly simplified and a number of libraries to be removed.
Custom legacy transforms need to be moved to T-Engines. ACS 6 still provides a version where both may be run in parallel. New versions of the document transform engine (DTE) and media management (MM) are planned for ACS 7.
- **scripts/travis/init.sh**: execute the docker logins only on branch builds
- **~/.m2/settings.xml**: remove `quay.io` & `docker.io` server definitions
(the docker logins in the init.sh script should be enough)
- **.travis.yml**: use DockerHub images instead of Quay.io images
* ACS-727 - initial commit
- add separate profiles for each binding
- add travis jobs
[tas]
* mark atom / ws unsupported tests
* some formatting to trigger [tas] tests
* exclude tests due to raised bug [tas]
* fix a few rest-api [tas] tests according with some recent changes from tas-utility
* disable some unsupported [tas] cmis webservices tests and adapt a few others
* - use TestGroup instead of hardcoded values
- exclude renditions tests
* trigger [tas] tests
* - disable tests due to raised bugs
- fix some incorrect tests
- add ability to exclude tests separately on each binding
[tas]
* [tas]
* - disable some more [tas] tests due to raised bug
* - fix a few [tas] cmis tests and disable other not supported on ws
* a bit of cleanup
* trigger [tas] tests
Removed all the Extractors that now exist in the T-Engines:
JodConnverterMetadataExtracter
TikaPoweredMetadataExtracter – the abstract base class used by other extractors
-- MailMetadataExtracter
-- PoiMetadataExtracter
-- TikaAutoMetadataExtracter
-- MP3MetadataExtracter
-- TikaSpringConfiguredMetadataExtracter - removed as it required Spring config and would run in process
-- PdfBoxMetadataExtracter
-- OpenDocumentMetadataExtracter
-- OfficeMetadataExtracter
-- DWGMetadataExtracter
HtmlMetadataExtracter
RFC822MetadataExtracter
XmlMetadataExtracter and XPathMetadataExtracter still exist but don't provide any extraction out of the box. The reason they still exist is to support custom transforms (in AMPs) to extract from XML. There are no XML extractors in the T-Engines at the moment, but that is where the custom transformer code really should be moved.
There are new tests to ensure the async transforms take place as expected.
Additionally many of the existing tests still exist (those not related to a specific extractor). Some of these have been modified to reflect that the extract is now async and to no longer check the modified value has not changed (it is now expected to change).
There are also a number of new metadata extract smoke tests that ensure that a selected subset of extracts are supported by the OOTB T-Engines.
Addition of support of async metadata extraction via T-Engines.
Still needs support for RM to control what is extracted in emails.
Still includes OOTB metadata extractors. To be removed.
Still needs removal of legacy transformers and 3rd party libraries they use.
This PR removes legacy Lucene code and dependency to the lucene 2.4.1 library.
The data type analysers were removed with an exception of the definitions in models (datadictionaryModel.xml and cmisModel.xml) to be backwards compatible with existing components (like older versions of Search Services) and to ensure the current old-style models can still work with repository and pass validation.
Does not include chemistry-opencmis 1.1.0 (downgraded to 1.0.0) or jackson-databind 2.11.2 (downgraded to 2.10.1) due to test failures. Original upgrades by dependabot.
* Fix latest T-Engine versions and remove redundantly versions in pom.xml files
* Changes from acs-packaging
* From alfresco-core (included json change after 7.0.0-A7)
Bump commons-io from 2.7 to 2.8.0 (#160)
(cherry picked from commit c5a8f5ffb3988bd87719aa43b7089851607b7308)
MNT-21879 : [Security] Multiple json-java vulnerabilities
- updated org.json.json version
(cherry picked from commit edaffa350b29157bdcd1de3737fa491b88093378)
[revert] MNT-21879 : [Security] Multiple json-java vulnerabilities
- updated org.json.json version
(cherry picked from commit 48e2cffb287adb5c5a1d5092ae7fde895c8d0836)
* from alfresco-data-model
SEARCH-2362: Consider changing properties from mandatory/enforced/pro… (#313)
* SEARCH-2362: Consider changing properties from mandatory/enforced/protected to NON mandatory/enforced/protected as incremental changes
* SEARCH-2362 Update tests to check whole set of returned diffs at once.
Co-authored-by: Tom Page <thomas.page@alfresco.com>
(cherry picked from commit 576601564251038311627f230f286b96ea950383)
Bump dependency.cxf.version from 3.3.7 to 3.4.0 (#309)
(cherry picked from commit dc689a911d83b608d49572cada5c2d16441c1eea)
Bump chemistry-opencmis-commons-impl from 1.0.0 to 1.1.0 (#305)
(cherry picked from commit 98fa7b66851aeada1bb6ea05bd6e747e3c55b342)
Bump commons-compress from 1.19 to 1.20 (#303)
(cherry picked from commit f6a950d4a36f05083536638cc5ff7866436f191d)
Bump pdfbox from 2.0.17 to 2.0.20 (#304)
(cherry picked from commit 435b786817ee18c28b84d093c6fbfbc330434e4c)
* From alfresco-remote-api
MNT-21702 : Kerberos SSO fallback mechanism for WebDAV (#765)
Reinstated 'part' of the reverted code change originally made in MNT-16931 to handle fallback correctly for WebDAV in a kerberos environment.
(cherry picked from commit 4091cb9d4b1cd643f9678a4c5c9654963a72c266)
[MNT-21758] Cannot map an AOS / WebDAV drive with Kerberos SSO enabled (#784)
- Add multi catch for Illegal Argument and Not Found when findWebScript is called.
(cherry picked from commit 59517076d2dfc0477870007a7d136cdf907e322f)
SEARCH-2363: Escape special characters when serializing ChildAssociationRefs and AssociationRefs objects (#788)
(cherry picked from commit f941a6e283517c0949f57fefdb1e7e8d7cfb46c4)
ACS-580: Direct access urls - revert initial api version (#792)
(cherry picked from commit 7f4959fdfaeaf1b7debd78df5dcacd4a5b9e809a)
* From alfresco-repository
SEARCH-2289: Restore the support for SOLR 4 in ACS 6.2+ deployments. (#1063)
This configuration is recommended for upgrading scenarios, where you need to upgrade ACS and to re-index the repository with SOLR 6 while using the system with SOLR 4.
(cherry picked from commit e8bba3ca13880be782c2e4c1d82223e70c370f28)
[MNT-21847] - Aync permissions fail when new nodes are created (#1188)
Fix:
*Changed method setFixedAcls on class ADMAccessControlListDAO to continue to propagate through children to apply the correct acl not only when the current child acl matches the shared acl to replace but also when the current child acl matches the new shared acl
Unit Test:
*Refactored the unit test FixedAclUpdaterTest to be able to add in a new test without repeating code: separating the operations that set the permissions from the one that triggers the job into separate methods
*As it was if one test failed, leaving aspects to be processed, the test would run indefinitely (it was programmed to keep running the job while there where nodes with the aspect). Added a verification to stop triggering the job if the number of nodes with the pendingFixAcl did not change between executions.
*Also, if one test failed, it would leave nodes with pendingFixAcl aspect in the database, and the other tests that ran after would also fail, not completing the goal of processing all nodes with the aspect. If a test fails, the folder structure it ran is now deleted so no nodes with the aspect from that structure are processed by the other tests.
*Added a test to find the first folder in a tree where permissions where set async that has the pendingFixAcl aspect and that creates a new node in it to verify the issue
(cherry picked from commit 443e5e226430a2760492fb82214ad520e7e1cb75)
Bump dependency.pdfbox.version from 2.0.20 to 2.0.21 (#1170)
(cherry picked from commit b93d73dec5f0e94d83f62721842b92e617dfba79)
Bump rhino from 1.7.12 to 1.7.13 (#1185)
(cherry picked from commit a5b5f072c64511999d3d27087a1a82f949371ac8)
Bump spring-security-core from 5.2.1.RELEASE to 5.3.4.RELEASE (#1167)
(cherry picked from commit 54f68d823333254cef74343288e99107494b1e28)
REPO-5339 Improve concurrency of SimplePermissionReference (#1189)
* REPO-5339 Improve concurrency of SimplePermissionReference
* REPO-5339 Change locks to synchronised block
* [REPO-5339] Improve concurrency of SimplePermissionReference
- Move to ConcurrentHashMap
- Capacity set to hold Repo + RM + slight overage for custom permissions
- Getting new SimplePermissionReference will not block while getting existing value.
- If a new value needs to be added to the Map, it will block when adding
- If blocked it will wait, in order, until it can attempt to add
- If it was blocked before it will check first if another operation added before adding as an atomic action
* [REPO-5339] Improve concurrency of SimplePermissionReference
- Simplify getPermissionReference by removing Future
* [REPO-5339] Thread Lock detected for AclReadersGet invocations
- Add load capacity and concurrency level
Co-authored-by: Jared Ottley <jared.ottley@alfresco.com>
(cherry picked from commit 39ded1cc0a364d24f737584eed22bef0918a5359)
[MNT-21766]: Significant degradation of performance as file and site count increase (#1217)
* [MNT-21766]: Significant degradation of performance as file and site
count increase
- Added the following indexes on alf_transaction:
idx_alf_txn_ctms_sc, cols (commit_time_ms)
idx_alf_txn_id_ctms, cols (id, commit_time_ms)
- Added the following indexes on alf_node:
idx_alf_node_ver, cols (version)
idx_alf_node_txn, cols (transaction_id)
- Created patch patch.db-V6.3-add-indexes-node-transaction
- Updated version.schema to 14002
- Added system property system.new-node-transaction-indexes.ignored, set
as true by default to not apply the patch automatically
- Created the MySQL update dbscripts to add the new indexes - and also
drop and recreate the pre-existing index idx_alf_txn_ctms on alf_transaction that was
inconsistent with the other DBMS: was indexing only commit_time_ms when
on all other DBMS index idx_alf_txn_ctms was on (commit_time_ms, id)
- Created the Postgres update dbscripts to add the new indexes
(cherry picked from commit d70746f63a6e6a2154d2e37c577955a17c42256e)
SEARCH-2450 Don't expose properties from solr endpoint if the model says not to index them. (#1228)
(cherry picked from commit 2fea6c9484def402e2bfd7c3717071ff8df69c01)
Fix/MNT-21800 CMIS Web Service Check Out returns error (#1232)
* MNT-21800 : CMIS Web Service Check Out returns error
Integrate path into codebase.
(cherry picked from commit 51a7793668ba1c969c5de37603190244e3839cd0)
* Revert dependency.opencmis.version change 1.1.0 back to 1.0.0 done by dependabot on 08/09/2020 as the upgraded failed in alfresco-remote-api tests which are now part of the same repo.
* Trial build with previous rino version
* Revert cxf upgrade to 3.4.0 from 3.7.7 as dependabot failed to apply it in acs-packaging
Revert org-json to 20160212 from 20090211 as dependabot failed to apply it in alfresco-repository
* Revert "Trial build with previous rino version"
This reverts commit 40f8120d5d.
* Remove READMEs that have been moved to the top level.
* Fix location of keystore
Only copy win64 and linux pdf-renderer to dist zip
Sort out what get included in dist jar and zip
* jackson databind revert
- Simplify dependencies and standardise order
- README updated
- Green builds
- Changes to make Jars in enterprise war match
- Changes to make files in enterprise image match
- Added travis_wait 40 to the initial build as it can take 20 minutes to download artifacts
