57 Commits

Author SHA1 Message Date
Lucian Tuca
fbe87a6460
ATS-440 (#149)
ATS-440: ATS: T-Engines - release 2.1.0
   - preparing next release
2019-10-28 11:20:52 +02:00
CezarLeahu
c650bf292c
ATS-545: Code formatting and small improvements (#113) 2019-09-12 16:46:48 +03:00
alandavis
d6777b58eb REPO-4639 Content conversion failed using Tika (#108)
* REPO-4639: Split tika engine_config.json into separate transformers.

* WIP: REPO-4639 Content conversion failed using Tika

The Tika T-Engine "transform" option does not exist when called via the Transform Service or Local transforms, which resulted in no transforms taking place. However this value is really not be needed as the T-Engine should be able to read its own engine_config.xml to work out which sub transform to use. Transforms only worked via Legacy transforms, which used a T-Engine.

This code is based on tried and tested ACS repository code. It has been further simplified.

TODO:
- replace the ConfigFileFinder class just added with something that uses Spring to read the JSON. i.e. simplify it.
- replace the CombinedConfig class just added with something that does not need the InLineTransformer. i.e. simplify it.
- create tests based on the repo tests
- remove the source and target mimetype checks in Tika as a check against engine_config.xml is cleaner.
- repeat the process for the Misc T-Engine as it has similar code checking source and target mimetypes.
- remove the transform option passed by the legacy transforms.

* Removed CombindConfig and ConfigFileFnder classes.

* Extracted AbstractTransformRegistry so that it may be used in the ACS repository too.

TODO AbstractTransformRegistry and AbstractTransformRegistry need to be moved to the alfresco-transform-model pakage

* tidy up only

* REPO-4639: Add priority to duplicate transforms.

* REPO-4639: Refactor TikaTransformationIT to use the new Tika /transform specifications
Changes AbstractTransformerControllerTest as the engine_config is now loaded in TransformRegistryImpl instead of AbstractTransformerController

* Rename to TransformServiceRegistry, so we don't have to change the repo code.

* Added the baseUrl parameter to the register method and fixed the missed rename in the last commit.

* Javadoc change only

* Moved common classes (with repo) AbbstractTransformRegistry and TransformServiceRegistry to alfresco-transform-model

* Replace (simplify) all the isTransformable calls with a check against the JSON.
- Tests now only pass targetEncoding to the 'string' transformer.

* Fix failing tests.

* Revert port change

* REPO-4639 : Add priorities to misc engine_config

* REPO-4639 : Add priorities to pdf-renderer and  imagemagick engine_config

* Remove test that is @Ignored

* Pick up alfresco-transformer-model 1.0.2.7-REPO-4639-1

* REPO-4639 : Add priorities to libreoffice engine_config

* REPO-4639 : Add priorities to tika engine_config

* REPO-4639 : Remove all priorities with value equal to 50 (default) from engine_config

* Switch over to using TransformServiceRegistry in org.alfresco.transform.client.registry
Reintroduce the noExtensionSourceFilenameTest having removed @Ignore.

* New whitesource issue on commons-compress 1.18. Upgrading to 1.19.

* Removed the text/javascript -> text/plain test as this is not supported

* Modifications as a result of changes to method names in alfresco-transform-model

* Pick up alfresco-transform-model 1.0.2.7-ATS545-2

* Remove unused imports
2019-09-12 15:34:42 +03:00
Alexandru-Eusebiu Epure
bcd6fefe4d
REPO-4617 Add missing source types for T-engines (#95)
Update missing sourceMediaType -> targetMediaType for alfresco-pdf-renderer
   Update missing sourceMediaType -> targetMediaType for imagemagick
   Update missing sourceMediaType -> targetMediaType for tika libreoffice
   Update missing sourceMediaType -> targetMediaType for tika
   Update missing sourceMediaType -> targetMediaType for misc transformer
   Add T-engine Integration Tests
   Fix JavaDoc warnings
   Add sample files for tested mimetypes
2019-09-03 13:34:07 +03:00
CezarLeahu
d9747f015d
ATS-466/ATS-538/ATS-539: Incorporate Misc T-Engine in ATS (#98)
- fix multiple Misc Transformer bugs related to file mimeTypes
- remove usage of 'source/targetMimetype' as transform options/parameters
- add 'source/targetMimetype' arguments to the 'processTransform' method
- remove unnecessary code (e.g. useless overridden methods)
- add quick* test resource files
- add integration test for 'Local Transformations' on the Misc engine
- set up Integration Tests POM configuration for all T-Engine modules
2019-08-26 13:59:38 +03:00
CezarLeahu
bb187dc00f
ATS-488 : Remove alfresco-core dependency (#90)
- remove *alfresco-core* dependency
- remove *alfresco-data-model* dependency
- replace _TempFileProvider_ with local implementation
- duplicate _RuntimeExec_ and _ExecParameterTokenizer_ from alfresco-core
- partially duplicate _MimetypeMap_ from alfresco-data-model
2019-08-20 10:05:39 +03:00
Cezar.Leahu
1f47cd47b4 ATS-532 : Restore Java language level to 1.8 2019-08-19 19:43:23 +03:00
CezarLeahu
22de0ce5df
ATS-532 : Code improvements (#89)
- move startup message from controllers to the Application classes (SpringBoot configuration beans)
- added static imports for most static variables and static methods
- simplified a few nested *if*s
- replaced Arrays.asList() with explicit immutable collections
- fixed a few IntelliJ code inspection warnings
2019-08-18 18:45:14 +03:00
Cezar.Leahu
485347729b ATS-531 : Reformat code 2019-08-14 22:46:36 +03:00
CezarLeahu
3e4f6af0e4
ATS-467 : JMS config startup message (#72) 2019-07-10 13:42:28 +03:00
Lucian Tuca
1ae2dd745a Merge branch 'master' into ATS-460 2019-07-09 12:11:05 +03:00
Cezar.Leahu
8b3e451b44 ATS-467 : Disable JMS Config beans on missing "activemq.url" property 2019-07-09 10:17:19 +03:00
Lucian Tuca
f1edbb71e9 ATS-460 : ATS: T-Engines - Update license information to *not* refer to Enterprise edition
- updated controller line
2019-07-08 11:40:29 +03:00
CezarLeahu
be48c8e7a9
ATS-467 : T-Engine logs fill up with ActiveMQ errors when used in ACS Community edition
- instantiate JMS beans only when 'activemq.url' property is set
- fix integration tests
2019-07-05 11:39:39 +03:00
eknizat
ff0f659ded
REPO-4331: Add remaining core transformers (#45)
* REPO-4331: Add remaining core transformers
* HtmlParserContentTransformer
* AppleIWorksContentTransformer
* StringExtractingContentTransformer
* TextToPdfContentTransformer
* OOXMLThumbnailContentTransformer
2019-06-20 12:31:38 +01:00
DenisGabriela
8ba4ae9c11 ATS-250 : ATS: T-Engines: Improve the convert message error handling (T-Base) (#51)
- code refactoring:
    if successful -> return the object ( Optional.ofNullable(request) )
    if not successful -> throw an exception (instead of sending an error message on the queue)
- when calling 'convert()' method, catch this exception and send a T-Reply with error message on the queue
- add correlationID to error logs
2019-06-13 16:52:17 +03:00
Cezar.Leahu
8419cc4b6d ATS-448 : Small T-Engine configuration improvements
- ignore empty fields during serialization
 - changed application.properties files to application-default.yaml
2019-06-05 13:21:15 +03:00
DenisGabriela
ee6227c8b7 ATS-434 : Implement /transform/config endpoint in T-Engines (#46)
- fix config file loading
- rename `/info` endpoint to '/transform/config'
2019-06-04 13:43:58 +03:00
DenisGabriela
d2292f94a0 ATS-434 : Implement /info endpoint in T-Engines (#44)
- implement '/info' endpoint
   - add engine_config files
   - use SNAPSHOT transform-model with new Transform Config models (TODO update after transform-model release)
   - remove 'tests' from travis stages
   - add new junits
   - add test resources
2019-05-31 14:14:03 +03:00
CezarLeahu
70652dcb31
ATS-400 : Update Copyright (#23)
* ATS-400 : Update Copyright

* ATS-400 : Update maven license plugin configuration

* ATS-400 : Update Copyright with the license-maven-plugin

* ATS-400 : Update licence-maven-plugin default configuration
2019-05-20 14:06:25 +03:00
Lucian Tuca
279efd3071 ATS-329
- updated exception package
2019-04-11 16:41:00 +03:00
Lucian Tuca
c5f528c434 ATS-371 : Error in AWS_AI Engine when upload .jpeg files in ACS
- updated behaviour for empty source files to throw bad request exception and replies
   - updated docker-transformer for ai engine
2019-04-05 12:35:39 +03:00
Denis Ungureanu
07386b1a1d ATS-237 : Unit tests for the queue logic (router & engine) 2019-02-05 13:56:59 +00:00
Lucian Tuca
7c466b7c82 ATS-238 2019-02-05 13:55:43 +00:00
Cezar.Leahu
89786df316 ATS-234 : Increase default JMS connection number from 10 to 20 2019-01-11 17:27:01 +02:00
Lucian Tuca
86bb0210e0 ATS-227 : PoC: Improve scaling/performance of transforms via T-Engine queues ? 2019-01-11 08:49:10 +00:00
Cezar Leahu
54a7b07e55 ATS-213 : Transformers should have unique option name 2018-12-11 16:02:04 +00:00
Jan Vonka
e20f7a1bd2 ATS-224 : (shadow of ACE-5923) T-Engines: refine logging for probes (liveness & readiness)
- change log level from info to trace (and use parameterised formatting)
2018-12-07 10:00:52 +00:00
DenisGabriela
7c6d2ed024 ATS-198 : ACE-5893: T-Engines - Files bigger than 1MB can not be uploaded via multipart API
- renamed multipart properties
2018-11-15 12:42:01 +02:00
Cezar Leahu
24611ea42e ATS-176 : Log improvements with Slf4j 2018-11-13 15:22:38 +00:00
Cezar.Leahu
fb25362e08 ATS-191 : Cleanup source temp files from T-Engines after SFS upload 2018-11-12 12:23:49 +02:00
Cezar.Leahu
39cfc09ab6 ATS-191 : Delete temp target files after SFS upload 2018-11-05 19:14:38 +02:00
Cezar.Leahu
f007b674e8 ATS-175 : JavaDoc (2) 2018-10-31 11:25:36 +02:00
Cezar.Leahu
ad4ea1574e ATS-175 : Replace HTTP numeric codes with constants 2018-10-30 16:11:10 +02:00
Cezar.Leahu
d85c03d362 ATS-175 : T-Engine code cleanup 2018-10-26 16:38:09 +03:00
DenisGabriela
8e7e775eef ATS-126 : Expose JVM metrics - eg. CPU & Memory (within jvm process / container)
- addressed review comments
2018-10-15 16:08:47 +03:00
DenisGabriela
6a7bbb3c4e ATS-126 : Expose JVM metrics - eg. CPU & Memory (within jvm process / container)
- added pod name tag tag on metrics
2018-10-15 12:41:42 +03:00
DenisGabriela
494a16c00d ATS-126 : Expose JVM metrics - eg. CPU & Memory (within jvm process / container)
- removed disabling of non-t-engines' metrics
2018-10-10 16:42:34 +03:00
DenisGabriela
2d578997b9 ATS-126 : Expose JVM metrics - eg. CPU & Memory (within jvm process / container)
- removed disabling of non-t-engines' metrics
2018-10-10 16:36:19 +03:00
DenisGabriela
7c2cb2f768 ATS-126 : Expose JVM metrics - eg. CPU & Memory (within jvm process / container)
- added actuator & micrometer
   - enabled /health, /metrics & /prometheus
   - disabled some of the default metrics (subject to be discussed)
2018-10-10 11:41:57 +03:00
DenisGabriela
24d44398ab ATS-35 : T-Engines: update Docker Transformers to Spring Boot 2.0.x GA
- upgraded to Spring boot 2.0.5
   - added dependency to spring-boot-starter-web (before it was brought in by thymeleaf 1.5.15)
   - removed try-catch since UnsupportedEncodingException is not thrown anymore by HierarchicalUriComponents.encodeUriComponent(..)
   - updated WebApplicationConfig to implement WebMvcConfigurer (WebMvcConfigurerAdapter is deprecated)
   - updated import of LocalServerPort
   - updated tests to use new method from InvocationOnMock
2018-10-08 14:38:27 +03:00
Cezar Leahu
34bfc8b60f ATS-116 : Disable k8s Liveness probes with Transformations 2018-09-25 14:45:24 +01:00
Denis Ungureanu
dbedbce8c6 ATS-68 : ATS-16: Fix error status code mapping for expected invalid requests
- fixed code after updating the transform-data-model version (ATS-70)
2018-08-21 15:11:22 +03:00
Denis Ungureanu
9fdafcb60f ATS-68 : ATS-16: Fix error status code mapping for expected invalid requests
- updated tests
   - added negative test for 400 reply
2018-08-21 14:05:31 +03:00
Denis Ungureanu
18f055bf46 ATS-68 : ATS-16: Fix error status code mapping for expected invalid requests
- validated the transformRequest on '/transform' endpoint
2018-08-20 15:41:34 +03:00
Lucian Tuca
dda632c7a5 feature/ATS-16 2018-08-17 09:32:25 +01:00
Alan Davis
82c5e3e96a REPO-3425 Transformers: Tika based transformers 2018-06-28 13:25:01 +01:00
Alan Davis
7de82a3541 Feature/repo 3505 probes 2018-05-23 15:36:56 +01:00
Alan Davis
93ab9ad92f REPO-3449 Pickup ch.qos.logback 1.2.3 and enhance logging 2018-04-20 18:12:19 +01:00
Alan Davis
73af441ab3 REPO-3424 More unit tests and pick up latest alfresco-core (7.2) 2018-04-16 23:22:18 +01:00