* period.getMillis() (joda lib) returns an int instead of a long, so on larger periods (like 30days) we get an error that the value cannot fit in an int
* Changed the validation to only calculate the remaining milliseconds in the edge case of the remaining days being zero (as both 23h before and 23h after correspond to 0 remaining days)
* Added a unit test that validates the usage 60 days up from the licence expiring to validate the problem stops occurring
* Added a unit test that validates the usage 30 days after the licence expiring to cover all use cases
(cherry picked from commit 6e1d25a8f0)
* Remove pdfbox jars as they should no longer be needed.
* Reintroduce tests that use Tika to 'guess' mimetypes as it was the tika parse that was pulling in the pdfbox libraries.
Classes that use Tika:
* HTMLRenderingEngine - removed as it is no longer used
* RemoteConnectorResponseImpl - called tika utility toByteArray so not using pdfbox
* TikaCharsetFinder - called to identify the charset not mimetype so not using pdfbox
* MimetypeMap - main use of Tika. Used to detect mimetypes. Might have been using pdfbox.
Cherry pick from master (7.1.0)
* MNT-21837 - License expires 23h earlier
* Original problem is that the remaining time to license expires is calculated in days (long, not double) and anything less then 24h computes to 0 days. The validation for license being valid is that the remaining days is more than zero.
* Added method calculateMs in DateUtil class to return the interval in ms considering 2 given dates and respective timezones
* Changed the validation to use the remaining milliseconds value instead of remaining days
* Added unit test where license expires 6 hours from now - repo is not supposed to enter read-only mode
* Added unit test where license expired 1 minute ago - repo is supposed to enter read-only mode
* Removed unused import
(cherry picked from commit 630cd99067)
* Changes made to correct the value of totalItems when performing a TMDQ
* Fixes after review
- Slight change was made to NodePermissionAssessor to log when permission
limits are exceeded
* Now pre-computing maxPermissionChecks value as per review suggestion
(cherry picked from commit cb636d1140)
Originally fixed on 6.2.N. A couple of the Legacy classes no longer exist so changes are not on master. Changes to the renditions, default page limit and test still apply.
(cherry picked from commit 9d276aed7081e0078ed3f7f01dabdec20c599bfe)
* SEARCH-2802: Filter HTTP requests (now "none" and "secret" communication methods are available) from X509 Web Filter.
* SEARCH-2802: HttpClientFactory (for Repository and Search Services clients) support for Shared Secret communication.
* SEARCH-2802: Fix HttpClientFactory base unit tests.
(cherry picked from commit 20dd0efc6f)
* On move node, verify if parent has pending acl aspect applied and consider the pending shared ACL to update inheritance to avoid ending up with mixed permissions as children of pending acl nodes do not have the correct acls and when moved, keep their wrong acl.
* Add public method setInheritanceForChildren that receives an additional param: forceSharedACL. If an unexpected ACL occurs in a child, it can be overridden by setting it.
* Implement method setInheritanceForChildren that receives an additional parameter: forceSharedACL
* Add method setFixedAcls that receives an additional parameter: forceSharedACL - When a child node has an unexpected ACL, setting this parameter to true will force it to assume the new shared ACL instead of throwing a concurrency exception. When the shared ACL is forces, a warning is thrown in the log informing on what node exactly are we forcing the ACL. This is only possible when the child ACL is type SHARED and when it has an unexpected ACL
* All methods that called setFixedAcls without the new parameter will continue to operate as normal, as having forceSharedACL=false
* Added property forceSharedACL to the FixedACLUpdaterJob. If set to true it will force shared ACL to propagate through children even if there is an unexpected ACL
* When there is a exception detected when doing setInheritanceForChildren on the job, catch and log the error, but do not rollback the entire batch
* On copy/move unit tests I changed the ACL of the target folders on copy and move tests so that the old shared ACL accessed was never the same for origin and target folders as happens when performing these operations between sites
* Added unit test to verify fix for MNT-21898 - testAsyncWithNodeMoveChildToChildPendingFolder
* Added unit test to verify system property for the job: forceSharedACL - testAsyncWithErrorsForceSharedACL
(cherry picked from commit ace87c9c3b)
[skip ci]
The TransformationOptionsConverter class did not convert the newer transform option format a Map<String, String> to the legacy ImageTransformationOptions class that contains the commandOption (OPT_COMMAND_OPTIONS) property. As a result no legacy transformer is asked if it can do the transform.
There are no Legacy transformers in ACS 7, so this fix cannot be applied there as the class has been removed. Fixing on 6.2.N.
(cherry picked from commit 3a8cb74f26)
[skip ci]
Original issue was ACS-1291, however the original commit for this on master was reverted and then later included in REPO-5610
REPO-5610 events are not actually sent to activemq (#360)
* Add events tests
* Polished put test: connects to JMS via TCP and validate that the event sent is also received back
* Now the tests provides a simple main() that listens on the topic, useful for quick debug sessions
* Now the user name is collected in the calling thread, so that the sendEvent does not silently fails
* Apply changes following review
* Now using queue system to guarantee events order
* Add license
* Updated logs and corrected comments
* Remove empty methods
* Now catering for spurious events at startup when database is bootstrapped
* Now preserving the txn-id in all events
* Moved up definitions in events2.xml after PR feedback
Co-authored-by: Bruno Bossola bruno@meterian.com
(cherry picked from commit 046116d)
Backport to alfresco-enterprise-repo will also include fixes for MNT-22301 Query Accelerator - datetime & stores
* REPO-5376 Remove all temporary code
* Remove DBStats, SingleTaskRestartableWatch
* Remove propertiesCache and aspectsCache from DBQueryEngine as they were marked as temporary
* Remove further temporary code
Co-authored-by: Nana Insaidoo <insaidoo.nana@yahoo.it>
* [SEARCH-2677] Extract SearchEngineResultSet and SearchEngineResultMetadata interfaces
* [SEARCH-2677] Scan the ResultSet decorator chain for a maximum of 3 nested levels
Re enabling changes disabled in ACS-936 we can see deadlock in asynchronouslyRefreshedCacheThreadPool (AbstractAsynchronouslyRefreshedCache).
There should be no reason to be calling the dictionary destroy method before the doCall() finishes...and it is the use of the destroy method that carries the risk of deadlock.
Proposed fix: the liveLock is used for the doCall() method, this will stop deadlock from external calls to dictionary destroy() while doCall is in progress.
Removed invalidating cache fix (e.g. fix the issue where cluster nodes could have null values and other nodes had default value… so no properly invalidated on node startup). This fix was moved in ClusteringBootstrap as the initial one was causing issues.
Created unit test testIfGroupIsAdminAuthority to check if public method isAdminAuthority correctly validates groups:
* Verify it can identify GROUP_ALFRESCO_ADMINISTRATORS as an admin group
* Verify created groups (non administrators) and users are not considered administrators
* Add a subgroup to the parent group and verify if its still identified as non admin
* Add the group to the administrators group and verify if both group and subgroup are identified correctly as admins
* Add the user to the subgroup and verify if he's an admin
* Create a group with the same name as an admin user, group should not be identified as admin
* REPO-5549 Fix: Local transformer names must exist and be unique doclib Read from ... when overriding a Local transform.
* Problem was that there were two conflicting pieces of code in play in the LocalTransformRegistry.register method. One that checked for duplicate T-Engine names and the other that allowed transforms to be overridden if they had the same name.
* The code checking for duplicate names needed an extra clause to only look at T-Engines (they have a T-Engine url associated with them).
* The code that overrode transforms then worked, but still had issues as the supported source to target mimetypes, priorities and max sizes were not cleared.
* It turned out to be simpler to split the original LocalTransformRegistry.register method into two. Extracting a new method into CombinedConfig.removeOverriddenOrInvalidTransformers that discarded invalid or overridden config before the list of supported source to target mimetypes was created rather than try to fix them up later. This is why there appear to be quite a few changes.
* More extensive unit tests were also added.
Provides hook points for the query accelerator in alfresco-enterprise-repo.
In addition to these, it also includes temporary code to add timing headers to REST API calls. These will be removed as part of REPO-5376.
Commits mainly by Bruno and Nana. Merging from old projects and changes from master by Alan.
Bug found while reviewing documents on how to create a custom metadata extractor. The original refactor had left the repo doing the mapping. It should have been passing the fully qualified repo properties to the T-Engine to do the mapping.
Linked to:
Alfresco/acs-packaging#1826Alfresco/alfresco-transform-core#316
Now that the T-Engines provide parity with in process legacy transforms and metadata extract, the legacy transform code is now being removed. This will allow the documentation to be greatly simplified and a number of libraries to be removed.
Custom legacy transforms need to be moved to T-Engines. ACS 6 still provides a version where both may be run in parallel. New versions of the document transform engine (DTE) and media management (MM) are planned for ACS 7.
* MNT-22040 - Copying nodes that contain sys:pendingFixAcl aspect
* Increased coverage of unit tests
* Added public method removePendingAclAspect
* Changed the job to use the removePendingAclAspect method instead of
repeating the same set of operations twice
* Changed method setFixedAcls to use sharedAclToReplace property of node
if it has the pendingFixAcl aspect applied instead of using the current
shared ACL to be replaced
* Added more tests and validations
* Fixed cuncurrency issues when updating parent and child at the same time, permissions triggered by a move on a node with a pending acl and permissions needeing to be reapplied on nodes that already have the pending acl aspect
* Increase the tree strcuture as sometimes we don't reach the timeout in tests
* code cleanup and formatting
* ACS-976 Metadata extraction NPE thrown in multi-node cluster using ATS
In a multi repo node cluster, it is possible one node requests a metadata extract via the ATS and
that another responds to it. In this case the responder needs to recreate the TransformRequest. In
this case the target mimetype was wrong, resulting in the TransformRequest being sent to the wrong
code which then through a NPE because there was no replyQueue specified on the TransformRequest.
* ACS-963 Investigate testSharedLinkCreateGetDelete failures
Modified date changes. It can change now as metadata extract is async
* ACS-953 Unable to upgrade from ACS-7.0-M1 or ACS-7.0-M2 to ACS-7.0-A12
Now that we create internal releases of the form M.m.r-M9 (milestone), M.m.r-A9 (alpha) and M.m.r-RC9 (release candidate) during development and testing, we need to ensure we can upgrade from any of these to any other, as they may occur in any order and also to the final external release M.m.r. We also will allow a change from the final release back to an internal one for regression testing. The code within ModuleComponentHelper which calls this method, checks if it is the same version (0) and then if it is downgrading (> 0), so if they share the same M.m.r part matches AND is one of these formats, we return <0.
Added an optional extractMapping transform option to all metadata extractors to override the default one in the T-Engine.
In the case of the AGS AMP it extends the RFC822MetadataExtracter with its own class to specify a different set of document to system mappings. The class in the repo no longer does extractions, but is now used by the AsynchronousExtractor, which offloads extractions to T-Engines to obtain the mappings if it has been extended that are then passed to the T-Engine.
Removed all the Extractors that now exist in the T-Engines:
JodConnverterMetadataExtracter
TikaPoweredMetadataExtracter – the abstract base class used by other extractors
-- MailMetadataExtracter
-- PoiMetadataExtracter
-- TikaAutoMetadataExtracter
-- MP3MetadataExtracter
-- TikaSpringConfiguredMetadataExtracter - removed as it required Spring config and would run in process
-- PdfBoxMetadataExtracter
-- OpenDocumentMetadataExtracter
-- OfficeMetadataExtracter
-- DWGMetadataExtracter
HtmlMetadataExtracter
RFC822MetadataExtracter
XmlMetadataExtracter and XPathMetadataExtracter still exist but don't provide any extraction out of the box. The reason they still exist is to support custom transforms (in AMPs) to extract from XML. There are no XML extractors in the T-Engines at the moment, but that is where the custom transformer code really should be moved.
There are new tests to ensure the async transforms take place as expected.
Additionally many of the existing tests still exist (those not related to a specific extractor). Some of these have been modified to reflect that the extract is now async and to no longer check the modified value has not changed (it is now expected to change).
There are also a number of new metadata extract smoke tests that ensure that a selected subset of extracts are supported by the OOTB T-Engines.
* REPO-5208 Addition of extra async metadata extract tests for overriding policy, tag extractio and carryAspectProperties
Main author: Adina Ababei <adina.ababei@ness.com>
Testing of tagging modified by Alan Davis to pass when Solr is not running.
