inteligr8/alfresco-transform-core
1
0
Fork 0
mirror of https://github.com/Alfresco/alfresco-transform-core.git synced 2025-08-14 17:58:27 +00:00
Code Issues Packages Projects Releases Wiki Activity

Save point: [skip ci]

Browse Source 
* Beginnings of new t-base (using TransformEngine and CustomeTransformer, no need for a controller of Application in t-engine modules)
* Using org.alfresco.transform.<module> package
* Beginnings of new Tika t-engine
This commit is contained in:
alandavis
2022-06-30 13:39:24 +01:00
parent 78c82c9a01
commit b619f27207
157 changed files with 8740 additions and 1543 deletions
Download Patch File Download Diff File Expand all files Collapse all files

233
t-engine-base/README.md Normal file
View File

@@ -0,0 +1,233 @@
# Common code for Transform Engines
This project contains code that is common between all the ACS T-Engine transformers that run as Spring Boot process (optionally within their own
Docker containers). It performs common actions such as logging, throttling requests and handling the streaming of content to and from the container.
For more details on build a custom T-Engine, please refer to the current docs in ACS Packaging, including:
* [ATS Configuration](https://github.com/Alfresco/acs-packaging/blob/master/docs/custom-transforms-and-renditions.md#ats-configuration)
* [Creating a T-Engine](https://github.com/Alfresco/acs-packaging/blob/master/docs/creating-a-t-engine.md)
## Overview
A transformer project is expected to provide the following files:
~~~
src/main/resources/templates/transformForm.html
src/main/java/org/alfresco/transformer/<TransformerName>Controller.java
src/main/java/org/alfresco/transformer/Application.java
~~~
* transformForm.html - A simple test page using [thymeleaf](http://www.thymeleaf.org) that gathers request
parameters so they may be used to test the transformer.
~~~
<html xmlns:th="http://www.thymeleaf.org">
<body>
<div>
<h2>Test Transformation</h2>
<form method="POST" enctype="multipart/form-data" action="/transform">
<table>
<tr><td><div style="text-align:right">file *</div></td><td><input type="file" name="file" /></td></tr>
<tr><td><div style="text-align:right">file *</div></td><td><input type="file" name="file" /></td></tr>
<tr><td><div style="text-align:right">sourceExtension *</div></td><td><input type="text" name="sourceExtension" value="" /></td></tr>
<tr><td><div style="text-align:right">targetExtension *</div></td><td><input type="text" name="targetExtension" value="" /></td></tr>
<tr><td><div style="text-align:right">sourceMimetype *</div></td><td><input type="text" name="sourceMimetype" value="" /></td></tr>
<tr><td><div style="text-align:right">targetMimetype *</div></td><td><input type="text" name="targetMimetype" value="" /></td></tr>
<tr><td><div style="text-align:right">abc:width</div></td><td><input type="text" name="width" value="" /></td></tr>
<tr><td><div style="text-align:right">abc:height</div></td><td><input type="text" name="height" value="" /></td></tr>
<tr><td><div style="text-align:right">timeout</div></td><td><input type="text" name="timeout" value="" /></td></tr>
<tr><td></td><td><input type="submit" value="Transform" /></td></tr>
</table>
</form>
</div>
<div>
<a href="/log">Log entries</a>
</div>
</body>
</html>
~~~
* *TransformerName*Controller.java - A [Spring Boot](https://projects.spring.io/spring-boot/) Controller that
extends AbstractTransformerController to handel requests. It implements a few methods including *transformImpl*
which is intended to perform the actual transform. Generally the transform is done in a sub class of
*JavaExecutor*, when a Java library is being used or *AbstractCommandExecutor*, when an external process is used.
Both are sub interfaces of *Transformer*.
~~~
...
@Controller
public class TransformerNameController extends AbstractTransformerController
{
private static final Logger logger = LoggerFactory.getLogger(TransformerNameController.class);
TransformerNameExecutor executor;
@PostConstruct
private void init()
{
executor = new TransformerNameExecutor();
}
@Override
public String getTransformerName()
{
return "Transformer Name";
}
@Override
public String version()
{
return commandExecutor.version();
}
@Override
public ProbeTestTransform getProbeTestTransform()
{
// See the Javadoc on this method and Probes.md for the choice of these values.
return new ProbeTestTransform(this, "quick.pdf", "quick.png",
7455, 1024, 150, 10240, 60 * 20 + 1, 60 * 15 - 15)
{
@Override
protected void executeTransformCommand(File sourceFile, File targetFile)
{
transformImpl(null, null, null, Collections.emptyMap(), sourceFile, targetFile);
}
};
}
@Override
public void transformImpl(String transformName, String sourceMimetype, String targetMimetype,
Map<String, String> transformOptions, File sourceFile, File targetFile)
{
executor.transform(sourceMimetype, targetMimetype, transformOptions, sourceFile, targetFile);
}
}
~~~
* *TransformerName*Executer.java - *JavaExecutor* and *CommandExecutor* sub classes need to extract values from
*transformOptions* and use them in a call to an external process or as parameters to a library call.
~~~
...
public class TransformerNameExecutor extends AbstractCommandExecutor
{
...
@Override
public void transform(String transformName, String sourceMimetype, String targetMimetype,
Map<String, String> transformOptions,
File sourceFile, File targetFile) throws TransformException
{
final String options = TransformerNameOptionsBuilder
.builder()
.withWidth(transformOptions.get(WIDTH_REQUEST_PARAM))
.withHeight(transformOptions.get(HEIGHT_REQUEST_PARAM))
.build();
Long timeout = stringToLong(transformOptions.get(TIMEOUT));
run(options, sourceFile, targetFile, timeout);
}
}
~~~
* Application.java - [Spring Boot](https://projects.spring.io/spring-boot/) expects to find an Application in
a project's source files. The following may be used:
~~~
package org.alfresco.transformer;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
@SpringBootApplication
public class Application
{
public static void main(String[] args)
{
SpringApplication.run(Application.class, args);
}
}
~~~
Transform requests are handled by the *AbstractTransformerController*, but are either:
* POST requests (a direct http request from a client) where the transform options are passed as parameters, the source is supplied as a multipart file and
the response is a file download.
* POST request (a request via a message queue) where the transform options are supplied as JSON and the response is also JSON.
The source and target content is read from a location accessible to both the client and the transfomer.
**Example JSON request body**
```javascript
var transformRequest = {
"requestId": "1",
"sourceReference": "2f9ed237-c734-4366-8c8b-6001819169a4",
"sourceMediaType": "application/pdf",
"sourceSize": 123456,
"sourceExtension": "pdf",
"targetMediaType": "text/plain",
"targetExtension": "txt",
"clientType": "ACS",
"clientData": "Yo No Soy Marinero, Soy Capitan, Soy Capitan!",
"schema": 1,
"transformRequestOptions": {
"targetMimetype": "text/plain",
"targetEncoding": "UTF-8",
"abc:width": "120",
"abc:height": "200"
}
}
```
**Example JSON response body**
```javascript
var transformReply = {
"requestId": "1",
"status": 201,
"errorDetails": null,
"sourceReference": "2f9ed237-c734-4366-8c8b-6001819169a4",
"targetReference": "34d69ff0-7eaa-4741-8a9f-e1915e6995bf",
"clientType": "ACS",
"clientData": "Yo No Soy Marinero, Soy Capitan, Soy Capitan!",
"schema": 1
}
```
## Building and testing
The project can be built by running the Maven command:
~~~
mvn clean install
~~~
## Artifacts
The artifacts can be obtained by:
* downloading from the [Alfresco repository](https://artifacts.alfresco.com/nexus/content/groups/public/)
* Adding a Maven dependency to your pom file.
~~~
<dependency>
<groupId>org.alfresco</groupId>
<artifactId>alfresco-t-engine-base</artifactId>
<version>1.0</version>
</dependency>
~~~
and the Alfresco Maven repository:
~~~
<repository>
<id>alfresco-maven-repo</id>
<url>https://artifacts.alfresco.com/nexus/content/groups/public</url>
</repository>
~~~
The build plan is available in [TravisCI](https://travis-ci.com/Alfresco/alfresco-transform-core).
## Contributing guide
Please use [this guide](https://github.com/Alfresco/alfresco-repository/blob/master/CONTRIBUTING.md)
to make a contribution to the project.

101
t-engine-base/pom.xml Normal file
View File

@@ -0,0 +1,101 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.alfresco</groupId>
<artifactId>alfresco-transform-core</artifactId>
<version>2.6.1-SNAPSHOT</version>
</parent>
<artifactId>alfresco-t-engine-base</artifactId>
<properties>
<transformer.base.deploy.skip>false</transformer.base.deploy.skip>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-thymeleaf</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-prometheus</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>com.vaadin.external.google</groupId>
<artifactId>android-json</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.dom4j</groupId>
<artifactId>dom4j</artifactId>
</dependency>
<dependency>
<groupId>org.alfresco</groupId>
<artifactId>alfresco-transform-model</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-activemq</artifactId>
</dependency>
<dependency>
<groupId>org.apache.activemq</groupId>
<artifactId>activemq-client</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-annotations</artifactId>
</dependency>
<dependency>
<groupId>org.messaginghub</groupId>
<artifactId>pooled-jms</artifactId>
</dependency>
<dependency>
<groupId>com.google.collections</groupId>
<artifactId>google-collections</artifactId>
<version>1.0</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>test-jar</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-deploy-plugin</artifactId>
<configuration>
<skip>${transformer.base.deploy.skip}</skip>
</configuration>
</plugin>
</plugins>
</build>
</project>

85
t-engine-base/src/main/java/org/alfresco/transform/base/Application.java Normal file
View File

@@ -0,0 +1,85 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.base;
import io.micrometer.core.instrument.MeterRegistry;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.actuate.autoconfigure.metrics.MeterRegistryCustomizer;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration;
import org.springframework.boot.context.event.ApplicationReadyEvent;
import org.springframework.context.annotation.Bean;
import org.springframework.context.event.EventListener;
import java.util.Arrays;
import java.util.List;
import static org.alfresco.transform.base.logging.StandardMessages.LICENCE;
@SpringBootApplication
@EnableAutoConfiguration(exclude = { DataSourceAutoConfiguration.class})
public class Application
{
private static final Logger logger = LoggerFactory.getLogger(Application.class);
@Autowired(required = false)
private List<TransformEngine> transformEngines;
@Value("${container.name}")
private String containerName;
@Bean
MeterRegistryCustomizer<MeterRegistry> metricsCommonTags()
{
return registry -> registry.config().commonTags("containerName", containerName);
}
public static void main(String[] args)
{
SpringApplication.run(Application.class, args);
}
@EventListener(ApplicationReadyEvent.class)
public void startup()
{
logger.info("--------------------------------------------------------------------------------------------------------------------------------------------------------------");
Arrays.stream(LICENCE.split("\\n")).forEach(logger::info);
if (transformEngines != null) {
transformEngines.stream()
.map(transformEngine -> transformEngine.getStartupMessage())
.forEach(message -> Arrays.stream(message.split("\\n")).forEach(logger::info));
}
logger.info("--------------------------------------------------------------------------------------------------------------------------------------------------------------");
logger.info("Starting application components... Done");
}
}

40
t-engine-base/src/main/java/org/alfresco/transform/base/CustomTransformer.java Normal file
View File

@@ -0,0 +1,40 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.base;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.Map;
public interface CustomTransformer
{
String getTransformerName();
void transform(String sourceMimetype, String sourceEncoding, InputStream inputStream,
String targetMimetype, String targetEncoding, OutputStream outputStream,
Map<String, String> transformOptions) throws Exception;
}

208
t-engine-base/src/main/java/org/alfresco/transform/base/QueueTransformService.java Normal file
View File

@@ -0,0 +1,208 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.base;
import static org.springframework.http.HttpStatus.BAD_REQUEST;
import static org.springframework.http.HttpStatus.INTERNAL_SERVER_ERROR;
import java.util.Optional;
import javax.jms.Destination;
import javax.jms.JMSException;
import javax.jms.Message;
import org.alfresco.transform.client.model.TransformReply;
import org.alfresco.transform.client.model.TransformRequest;
import org.alfresco.transform.common.TransformException;
import org.alfresco.transform.base.messaging.TransformMessageConverter;
import org.alfresco.transform.base.messaging.TransformReplySender;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.http.HttpStatus;
import org.springframework.jms.annotation.JmsListener;
import org.springframework.jms.support.converter.MessageConversionException;
import org.springframework.stereotype.Component;
/**
* Queue Transformer service.
* This service reads all the requests for the particular engine, forwards them to the worker
* component (at this time the injected controller - to be refactored) and sends back the reply
* to the {@link Message#getJMSReplyTo()} value. If this value is missing we've got to a dead end.
*
* @author Lucian Tuca
* created on 18/12/2018
*/
@Component
@ConditionalOnProperty(name = "activemq.url")
public class QueueTransformService
{
private static final Logger logger = LoggerFactory.getLogger(QueueTransformService.class);
// TODO: I know this is not smart but all the the transformation logic is in the Controller.
// The controller also manages the probes. There's tons of refactoring needed there, hence this. Sorry.
@Autowired
private TransformController transformController;
@Autowired
private TransformMessageConverter transformMessageConverter;
@Autowired
private TransformReplySender transformReplySender;
@JmsListener(destination = "${queue.engineRequestQueue}", concurrency = "${jms-listener.concurrency}")
public void receive(final Message msg)
{
if (msg == null)
{
logger.error("Received null message!");
return;
}
final String correlationId = tryRetrieveCorrelationId(msg);
Destination replyToDestinationQueue;
try
{
replyToDestinationQueue = msg.getJMSReplyTo();
if (replyToDestinationQueue == null)
{
logger.error(
"Cannot find 'replyTo' destination queue for message with correlationID {}. Stopping. ",
correlationId);
return;
}
}
catch (JMSException e)
{
logger.error(
"Cannot find 'replyTo' destination queue for message with correlationID {}. Stopping. ",
correlationId);
return;
}
logger.trace("New T-Request from queue with correlationId: {}", correlationId);
Optional<TransformRequest> transformRequest;
try
{
transformRequest = convert(msg, correlationId);
}
catch (TransformException e)
{
logger.error(e.getMessage(), e);
replyWithError(replyToDestinationQueue, HttpStatus.valueOf(e.getStatusCode()),
e.getMessage(), correlationId);
return;
}
if (!transformRequest.isPresent())
{
logger.error("T-Request from message with correlationID {} is null!", correlationId);
replyWithInternalSvErr(replyToDestinationQueue,
"JMS exception during T-Request deserialization: ", correlationId);
return;
}
TransformReply reply = transformController.transform(transformRequest.get(), null)
.getBody();
transformReplySender.send(replyToDestinationQueue, reply);
}
/**
* Tries to convert the JMS {@link Message} to a {@link TransformRequest}
* If any error occurs, a {@link TransformException} is thrown
*
* @param msg Message to be deserialized
* @return The converted {@link TransformRequest} instance
*/
private Optional<TransformRequest> convert(final Message msg, String correlationId)
{
try
{
TransformRequest request = (TransformRequest) transformMessageConverter
.fromMessage(msg);
return Optional.ofNullable(request);
}
catch (MessageConversionException e)
{
String message =
"MessageConversionException during T-Request deserialization of message with correlationID "
+ correlationId + ": ";
throw new TransformException(BAD_REQUEST.value(), message + e.getMessage());
}
catch (JMSException e)
{
String message =
"JMSException during T-Request deserialization of message with correlationID "
+ correlationId + ": ";
throw new TransformException(INTERNAL_SERVER_ERROR.value(),
message + e.getMessage());
}
catch (Exception e)
{
String message =
"Exception during T-Request deserialization of message with correlationID "
+ correlationId + ": ";
throw new TransformException(INTERNAL_SERVER_ERROR.value(),
message + e.getMessage());
}
}
private void replyWithInternalSvErr(final Destination destination, final String msg,
final String correlationId)
{
replyWithError(destination, INTERNAL_SERVER_ERROR, msg, correlationId);
}
private void replyWithError(final Destination destination, final HttpStatus status,
final String msg,
final String correlationId)
{
final TransformReply reply = TransformReply
.builder()
.withStatus(status.value())
.withErrorDetails(msg)
.build();
transformReplySender.send(destination, reply, correlationId);
}
private static String tryRetrieveCorrelationId(final Message msg)
{
try
{
return msg.getJMSCorrelationID();
}
catch (Exception ignore)
{
return null;
}
}
}

662
t-engine-base/src/main/java/org/alfresco/transform/base/TransformController.java Normal file
View File

@@ -0,0 +1,662 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.base;
import org.alfresco.transform.base.probes.ProbeTestTransform;
import org.alfresco.transform.common.TransformerDebug;
import org.alfresco.transform.client.model.InternalContext;
import org.alfresco.transform.client.model.TransformReply;
import org.alfresco.transform.client.model.TransformRequest;
import org.alfresco.transform.messages.TransformRequestValidator;
import org.alfresco.transform.config.TransformConfig;
import org.alfresco.transform.registry.TransformServiceRegistry;
import org.alfresco.transform.common.TransformException;
import org.alfresco.transform.base.clients.AlfrescoSharedFileStoreClient;
import org.alfresco.transform.base.logging.LogEntry;
import org.alfresco.transform.base.model.FileRefResponse;
import org.codehaus.plexus.util.FileUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.TypeMismatchException;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.core.io.Resource;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.stereotype.Controller;
import org.springframework.ui.Model;
import org.springframework.validation.DirectFieldBindingResult;
import org.springframework.validation.Errors;
import org.springframework.web.bind.MissingServletRequestParameterException;
import org.springframework.web.bind.annotation.ExceptionHandler;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.ResponseBody;
import org.springframework.web.client.HttpClientErrorException;
import org.springframework.web.multipart.MultipartFile;
import javax.annotation.PostConstruct;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.File;
import java.io.IOException;
import java.net.URL;
import java.util.Arrays;
import java.util.Collection;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.concurrent.atomic.AtomicInteger;
import static java.text.MessageFormat.format;
import static java.util.stream.Collectors.joining;
import static org.alfresco.transform.config.CoreVersionDecorator.setOrClearCoreVersion;
import static org.alfresco.transform.common.RequestParamMap.DIRECT_ACCESS_URL;
import static org.alfresco.transform.common.RequestParamMap.CONFIG_VERSION;
import static org.alfresco.transform.common.RequestParamMap.CONFIG_VERSION_DEFAULT;
import static org.alfresco.transform.common.RequestParamMap.ENDPOINT_TRANSFORM;
import static org.alfresco.transform.common.RequestParamMap.ENDPOINT_TRANSFORM_CONFIG;
import static org.alfresco.transform.base.fs.FileManager.TempFileProvider.createTempFile;
import static org.alfresco.transform.base.fs.FileManager.buildFile;
import static org.alfresco.transform.base.fs.FileManager.createAttachment;
import static org.alfresco.transform.base.fs.FileManager.createSourceFile;
import static org.alfresco.transform.base.fs.FileManager.createTargetFile;
import static org.alfresco.transform.base.fs.FileManager.createTargetFileName;
import static org.alfresco.transform.base.fs.FileManager.deleteFile;
import static org.alfresco.transform.base.fs.FileManager.getFilenameFromContentDisposition;
import static org.alfresco.transform.base.fs.FileManager.save;
import static org.alfresco.transform.base.util.RequestParamMap.FILE;
import static org.alfresco.transform.base.util.RequestParamMap.SOURCE_ENCODING;
import static org.alfresco.transform.base.util.RequestParamMap.SOURCE_EXTENSION;
import static org.alfresco.transform.base.util.RequestParamMap.SOURCE_MIMETYPE;
import static org.alfresco.transform.base.util.RequestParamMap.TARGET_MIMETYPE;
import static org.springframework.http.HttpStatus.BAD_REQUEST;
import static org.springframework.http.HttpStatus.CREATED;
import static org.springframework.http.HttpStatus.INTERNAL_SERVER_ERROR;
import static org.springframework.http.HttpStatus.OK;
import static org.springframework.http.MediaType.APPLICATION_JSON_VALUE;
import static org.springframework.http.MediaType.MULTIPART_FORM_DATA_VALUE;
import static org.springframework.util.StringUtils.getFilenameExtension;
/**
* Provides the main endpoints into the t-engine.
*/
@Controller
public class TransformController
{
private static final Logger logger = LoggerFactory.getLogger(TransformController.class);
private static final List<String> NON_TRANSFORM_OPTION_REQUEST_PARAMETERS = Arrays.asList(SOURCE_EXTENSION,
TARGET_MIMETYPE, SOURCE_MIMETYPE, DIRECT_ACCESS_URL);
@Autowired(required = false)
private List<TransformEngine> transformEngines;
@Autowired(required = false)
private List<CustomTransformer> customTransformers;
@Autowired
private AlfrescoSharedFileStoreClient alfrescoSharedFileStoreClient;
@Autowired
private TransformRequestValidator transformRequestValidator;
@Autowired
private TransformServiceRegistry transformRegistry;
@Autowired
private TransformerDebug transformerDebug;
@Value("${transform.core.version}")
private String coreVersion;
private TransformEngine transformEngine;
ProbeTestTransform probeTestTransform;
private Map<String, CustomTransformer> customTransformersByName = new HashMap<>();
private AtomicInteger httpRequestCount = new AtomicInteger(1);
@PostConstruct
public void init()
{
initTransformEngine();
initProbeTestTransform();
initCustomTransformersByName();
}
private void initTransformEngine()
{
if (transformEngines != null)
{
// Normally there is just one TransformEngine per t-engine, but we also want to be able to amalgamate the
// CustomTransform code from many t-engines into a single t-engine. In this case, there should be a wrapper
// TransformEngine (it has no TransformConfig of its own).
transformEngine = transformEngines.stream()
.filter(transformEngine -> transformEngine.getTransformConfig() == null)
.findFirst()
.orElse(transformEngines.get(0));
}
}
private void initProbeTestTransform()
{
if (transformEngine != null)
{
probeTestTransform = transformEngine.getLivenessAndReadinessProbeTestTransform();
}
}
private void initCustomTransformersByName()
{
if (customTransformers != null)
{
customTransformers.forEach(customTransformer -> customTransformersByName.put(customTransformer.getTransformerName(),
customTransformer));
}
}
/**
* @return a string that may be used in client debug.
*/
@RequestMapping("/version")
@ResponseBody
public String version()
{
return transformEngine.getTransformEngineName() + ' ' + coreVersion + " available";
}
/**
* Test UI page to perform a transform.
*/
@GetMapping("/")
public String transformForm(Model model)
{
return "transformForm";
}
/**
* Test UI error page.
*/
@GetMapping("/error")
public String error()
{
return "error"; // the name of the template
}
/**
* Test UI log page.
*/
@GetMapping("/log")
String log(Model model)
{
model.addAttribute("title", transformEngine.getTransformEngineName() + " Log Entries");
Collection<LogEntry> log = LogEntry.getLog();
if (!log.isEmpty())
{
model.addAttribute("log", log);
}
return "log"; // the name of the template
}
/**
* Kubernetes readiness probe.
*/
@GetMapping("/ready")
@ResponseBody
public String ready(HttpServletRequest request)
{
return probeTestTransform.doTransformOrNothing(request, false, this);
}
/**
* Kubernetes liveness probe.
*/
@GetMapping("/live")
@ResponseBody
public String live(HttpServletRequest request)
{
return probeTestTransform.doTransformOrNothing(request, true, this);
}
@GetMapping(value = ENDPOINT_TRANSFORM_CONFIG)
public ResponseEntity<TransformConfig> transformConfig(
@RequestParam(value = CONFIG_VERSION, defaultValue = CONFIG_VERSION_DEFAULT) int configVersion)
{
logger.info("GET Transform Config version: " + configVersion);
TransformConfig transformConfig = ((TransformRegistryImpl) transformRegistry).getTransformConfig();
transformConfig = setOrClearCoreVersion(transformConfig, configVersion);
return new ResponseEntity<>(transformConfig, OK);
}
@PostMapping(value = ENDPOINT_TRANSFORM, consumes = MULTIPART_FORM_DATA_VALUE)
public ResponseEntity<Resource> transform(HttpServletRequest request,
@RequestParam(value = FILE, required = false) MultipartFile sourceMultipartFile,
@RequestParam(value = SOURCE_MIMETYPE, required = false) String sourceMimetype,
@RequestParam(value = TARGET_MIMETYPE, required = false) String targetMimetype,
@RequestParam Map<String, String> requestParameters)
{
if (logger.isDebugEnabled())
{
logger.debug("Processing request via HTTP endpoint. Params: sourceMimetype: '{}', targetMimetype: '{}', "
+ "requestParameters: {}", sourceMimetype, targetMimetype, requestParameters);
}
final String directUrl = requestParameters.getOrDefault(DIRECT_ACCESS_URL, "");
File sourceFile;
String sourceFilename;
if (directUrl.isBlank())
{
if (sourceMultipartFile == null)
{
throw new TransformException(BAD_REQUEST.value(), "Required request part 'file' is not present");
}
sourceFile = createSourceFile(request, sourceMultipartFile);
sourceFilename = sourceMultipartFile.getOriginalFilename();
}
else
{
sourceFile = getSourceFileFromDirectUrl(directUrl);
sourceFilename = sourceFile.getName();
}
final String targetFilename = createTargetFileName(sourceFilename, sourceMimetype, targetMimetype);
probeTestTransform.incrementTransformerCount();
final File targetFile = createTargetFile(request, targetFilename);
Map<String, String> transformOptions = getTransformOptions(requestParameters);
String transformName = getTransformerName(sourceFile, sourceMimetype, targetMimetype, transformOptions);
String reference = "e"+httpRequestCount.getAndIncrement();
transformerDebug.pushTransform(reference, sourceMimetype, targetMimetype, sourceFile, transformName);
transformerDebug.logOptions(reference, requestParameters);
try
{
transformImpl(transformName, sourceMimetype, targetMimetype, transformOptions, sourceFile, targetFile);
final ResponseEntity<Resource> body = createAttachment(targetFilename, targetFile);
LogEntry.setTargetSize(targetFile.length());
long time = LogEntry.setStatusCodeAndMessage(OK.value(), "Success");
probeTestTransform.recordTransformTime(time);
transformerDebug.popTransform(reference, time);
return body;
}
catch (Throwable t)
{
transformerDebug.logFailure(reference, t.getMessage());
throw t;
}
}
/**
* '/transform' endpoint which consumes and produces 'application/json'
*
* This is the way to tell Spring to redirect the request to this endpoint
* instead of the one which produces 'html'
*
* @param request The transformation request
* @param timeout Transformation timeout
* @return A transformation reply
*/
@PostMapping(value = ENDPOINT_TRANSFORM, produces = APPLICATION_JSON_VALUE)
@ResponseBody
public ResponseEntity<TransformReply> transform(@RequestBody TransformRequest request,
@RequestParam(value = "timeout", required = false) Long timeout)
{
logger.trace("Received {}, timeout {} ms", request, timeout);
final TransformReply reply = new TransformReply();
reply.setRequestId(request.getRequestId());
reply.setSourceReference(request.getSourceReference());
reply.setSchema(request.getSchema());
reply.setClientData(request.getClientData());
final Errors errors = validateTransformRequest(request);
validateInternalContext(request, errors);
initialiseContext(request);
reply.setInternalContext(request.getInternalContext());
if (!errors.getAllErrors().isEmpty())
{
reply.setStatus(BAD_REQUEST.value());
reply.setErrorDetails(errors
.getAllErrors()
.stream()
.map(Object::toString)
.collect(joining(", ")));
transformerDebug.logFailure(reply);
logger.trace("Invalid request, sending {}", reply);
return new ResponseEntity<>(reply, HttpStatus.valueOf(reply.getStatus()));
}
transformerDebug.pushTransform(request);
// Load the source file
File sourceFile;
try
{
final String directUrl = request.getTransformRequestOptions().getOrDefault(DIRECT_ACCESS_URL, "");
if (directUrl.isBlank())
{
sourceFile = loadSourceFile(request.getSourceReference(), request.getSourceExtension());
}
else
{
sourceFile = getSourceFileFromDirectUrl(directUrl);
}
}
catch (TransformException e)
{
reply.setStatus(e.getStatusCode());
reply.setErrorDetails(messageWithCause("Failed at reading the source file", e));
transformerDebug.logFailure(reply);
logger.trace("Failed to load source file (TransformException), sending " + reply);
return new ResponseEntity<>(reply, HttpStatus.valueOf(reply.getStatus()));
}
catch (HttpClientErrorException e)
{
reply.setStatus(e.getStatusCode().value());
reply.setErrorDetails(messageWithCause("Failed at reading the source file", e));
transformerDebug.logFailure(reply);
logger.trace("Failed to load source file (HttpClientErrorException), sending " + reply, e);
return new ResponseEntity<>(reply, HttpStatus.valueOf(reply.getStatus()));
}
catch (Exception e)
{
reply.setStatus(INTERNAL_SERVER_ERROR.value());
reply.setErrorDetails(messageWithCause("Failed at reading the source file", e));
transformerDebug.logFailure(reply);
logger.trace("Failed to load source file (Exception), sending " + reply, e);
return new ResponseEntity<>(reply, HttpStatus.valueOf(reply.getStatus()));
}
// Create local temp target file in order to run the transformation
final String targetFilename = createTargetFileName(sourceFile.getName(), request.getTargetMediaType(), request.getSourceMediaType());
final File targetFile = buildFile(targetFilename);
// Run the transformation
try
{
String targetMimetype = request.getTargetMediaType();
String sourceMimetype = request.getSourceMediaType();
Map<String, String> transformOptions = getTransformOptions(request.getTransformRequestOptions());
transformerDebug.logOptions(request);
String transformName = getTransformerName(sourceFile, sourceMimetype, targetMimetype, transformOptions);
transformImpl(transformName, sourceMimetype, targetMimetype, transformOptions, sourceFile, targetFile);
reply.getInternalContext().setCurrentSourceSize(targetFile.length());
}
catch (TransformException e)
{
reply.setStatus(e.getStatusCode());
reply.setErrorDetails(messageWithCause("Failed at processing transformation", e));
transformerDebug.logFailure(reply);
logger.trace("Failed to perform transform (TransformException), sending " + reply, e);
return new ResponseEntity<>(reply, HttpStatus.valueOf(reply.getStatus()));
}
catch (Exception e)
{
reply.setStatus(INTERNAL_SERVER_ERROR.value());
reply.setErrorDetails(messageWithCause("Failed at processing transformation", e));
transformerDebug.logFailure(reply);
logger.trace("Failed to perform transform (Exception), sending " + reply, e);
return new ResponseEntity<>(reply, HttpStatus.valueOf(reply.getStatus()));
}
// Write the target file
FileRefResponse targetRef;
try
{
targetRef = alfrescoSharedFileStoreClient.saveFile(targetFile);
}
catch (TransformException e)
{
reply.setStatus(e.getStatusCode());
reply.setErrorDetails(messageWithCause("Failed at writing the transformed file", e));
transformerDebug.logFailure(reply);
logger.trace("Failed to save target file (TransformException), sending " + reply, e);
return new ResponseEntity<>(reply, HttpStatus.valueOf(reply.getStatus()));
}
catch (HttpClientErrorException e)
{
reply.setStatus(e.getStatusCode().value());
reply.setErrorDetails(messageWithCause("Failed at writing the transformed file. ", e));
transformerDebug.logFailure(reply);
logger.trace("Failed to save target file (HttpClientErrorException), sending " + reply, e);
return new ResponseEntity<>(reply, HttpStatus.valueOf(reply.getStatus()));
}
catch (Exception e)
{
reply.setStatus(INTERNAL_SERVER_ERROR.value());
reply.setErrorDetails(messageWithCause("Failed at writing the transformed file. ", e));
transformerDebug.logFailure(reply);
logger.trace("Failed to save target file (Exception), sending " + reply, e);
return new ResponseEntity<>(reply, HttpStatus.valueOf(reply.getStatus()));
}
try
{
deleteFile(targetFile);
}
catch (Exception e)
{
logger.error("Failed to delete local temp target file '{}'. Error will be ignored ",
targetFile, e);
}
try
{
deleteFile(sourceFile);
}
catch (Exception e)
{
logger.error("Failed to delete source local temp file " + sourceFile, e);
}
reply.setTargetReference(targetRef.getEntry().getFileRef());
reply.setStatus(CREATED.value());
transformerDebug.popTransform(reply);
logger.trace("Sending successful {}, timeout {} ms", reply, timeout);
return new ResponseEntity<>(reply, HttpStatus.valueOf(reply.getStatus()));
}
private Errors validateTransformRequest(final TransformRequest transformRequest)
{
DirectFieldBindingResult errors = new DirectFieldBindingResult(transformRequest, "request");
transformRequestValidator.validate(transformRequest, errors);
return errors;
}
private void validateInternalContext(TransformRequest request, Errors errors)
{
String errorMessage = InternalContext.checkForBasicErrors(request.getInternalContext(), "T-Request");
if (errorMessage != null)
{
errors.rejectValue("internalContext", null, errorMessage);
}
}
private void initialiseContext(TransformRequest request)
{
// If needed, initialise the context enough to allow logging to take place without NPE checks
request.setInternalContext(InternalContext.initialise(request.getInternalContext()));
}
private File getSourceFileFromDirectUrl(String directUrl)
{
File sourceFile = createTempFile("tmp", ".tmp");
try
{
FileUtils.copyURLToFile(new URL(directUrl), sourceFile);
}
catch (IllegalArgumentException e)
{
throw new TransformException(BAD_REQUEST.value(), "Direct Access Url is invalid.", e);
}
catch (IOException e)
{
throw new TransformException(BAD_REQUEST.value(), "Direct Access Url not found.", e);
}
return sourceFile;
}
protected Map<String, String> getTransformOptions(Map<String, String> requestParameters)
{
Map<String, String> transformOptions = new HashMap<>(requestParameters);
transformOptions.keySet().removeAll(NON_TRANSFORM_OPTION_REQUEST_PARAMETERS);
transformOptions.values().removeIf(v -> v.isEmpty());
return transformOptions;
}
/**
* Loads the file with the specified sourceReference from Alfresco Shared File Store
*
* @param sourceReference reference to the file in Alfresco Shared File Store
* @param sourceExtension default extension if the file in Alfresco Shared File Store has none
* @return the file containing the source content for the transformation
*/
private File loadSourceFile(final String sourceReference, final String sourceExtension)
{
ResponseEntity<Resource> responseEntity = alfrescoSharedFileStoreClient
.retrieveFile(sourceReference);
probeTestTransform.incrementTransformerCount();
HttpHeaders headers = responseEntity.getHeaders();
String filename = getFilenameFromContentDisposition(headers);
String extension = getFilenameExtension(filename) != null ? getFilenameExtension(filename) : sourceExtension;
MediaType contentType = headers.getContentType();
long size = headers.getContentLength();
final Resource body = responseEntity.getBody();
if (body == null)
{
String message = "Source file with reference: " + sourceReference + " is null or empty. "
+ "Transformation will fail and stop now as there is no content to be transformed.";
logger.warn(message);
throw new TransformException(BAD_REQUEST.value(), message);
}
final File file = createTempFile("source_", "." + extension);
logger.debug("Read source content {} length={} contentType={}",
sourceReference, size, contentType);
save(body, file);
LogEntry.setSource(filename, size);
return file;
}
private static String messageWithCause(final String prefix, Throwable e)
{
final StringBuilder sb = new StringBuilder();
sb.append(prefix).append(" - ")
.append(e.getClass().getSimpleName()).append(": ")
.append(e.getMessage());
while (e.getCause() != null)
{
e = e.getCause();
sb.append(", cause ")
.append(e.getClass().getSimpleName()).append(": ")
.append(e.getMessage());
}
return sb.toString();
}
private String getTransformerName(final File sourceFile, final String sourceMimetype,
final String targetMimetype, final Map<String, String> transformOptions)
{
// The transformOptions always contains sourceEncoding when sent to a T-Engine, even though it should not be
// used to select a transformer. Similar to source and target mimetypes and extensions, but these are not
// passed in transformOptions.
String sourceEncoding = transformOptions.remove(SOURCE_ENCODING);
try
{
final long sourceSizeInBytes = sourceFile.length();
final String transformerName = transformRegistry.findTransformerName(sourceMimetype,
sourceSizeInBytes, targetMimetype, transformOptions, null);
if (transformerName == null)
{
throw new TransformException(BAD_REQUEST.value(), "No transforms were able to handle the request");
}
return transformerName;
}
finally
{
if (sourceEncoding != null)
{
transformOptions.put(SOURCE_ENCODING, sourceEncoding);
}
}
}
public void transformImpl(String transformName, String sourceMimetype, String targetMimetype,
Map<String, String> transformOptions, File sourceFile, File targetFile)
{
//javaExecutor.transformExtractOrEmbed(transformName, sourceMimetype, targetMimetype, transformOptions, sourceFile, targetFile);
}
@ExceptionHandler(TypeMismatchException.class)
public void handleParamsTypeMismatch(HttpServletResponse response, MissingServletRequestParameterException e) throws IOException
{
final String message = format("Request parameter ''{0}'' is of the wrong type", e.getParameterName());
final int statusCode = BAD_REQUEST.value();
logger.error(message, e);
LogEntry.setStatusCodeAndMessage(statusCode, message);
response.sendError(statusCode, transformEngine.getTransformEngineName() + " - " + message);
}
@ExceptionHandler(MissingServletRequestParameterException.class)
public void handleMissingParams(HttpServletResponse response, MissingServletRequestParameterException e) throws IOException
{
final String message = format("Request parameter ''{0}'' is missing", e.getParameterName());
final int statusCode = BAD_REQUEST.value();
logger.error(message, e);
LogEntry.setStatusCodeAndMessage(statusCode, message);
response.sendError(statusCode, transformEngine.getTransformEngineName() + " - " + message);
}
@ExceptionHandler(TransformException.class)
public void transformExceptionWithMessage(HttpServletResponse response, TransformException e) throws IOException
{
final String message = e.getMessage();
final int statusCode = e.getStatusCode();
logger.error(message, e);
long time = LogEntry.setStatusCodeAndMessage(statusCode, message);
probeTestTransform.recordTransformTime(time);
response.sendError(statusCode, transformEngine.getTransformEngineName() + " - " + message);
}
}

56
t-engine-base/src/main/java/org/alfresco/transform/base/TransformEngine.java Normal file
View File

@@ -0,0 +1,56 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.base;
import org.alfresco.transform.config.TransformConfig;
import org.alfresco.transform.base.probes.ProbeTestTransform;
/**
* The interface to the custom transform code applied on top of a base t-engine.
*/
public interface TransformEngine
{
/**
* @return the name of the t-engine. The t-router reads config from t-engines in name order.
*/
String getTransformEngineName();
/**
* @return messages to be logged on start up (license & settings). Use \n to split onto multiple lines.
*/
String getStartupMessage();
/**
* @return a definition of what the t-engine supports. Normally read from a json Resource on the classpath.
*/
TransformConfig getTransformConfig();
/**
* @return a ProbeTestTransform (will do a quick transform) for k8 liveness and readiness probes.
*/
ProbeTestTransform getLivenessAndReadinessProbeTestTransform();
}

64
t-engine-base/src/main/java/org/alfresco/transform/base/TransformInterceptor.java Normal file
View File

@@ -0,0 +1,64 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.base;
import static org.alfresco.transform.base.fs.FileManager.SOURCE_FILE;
import static org.alfresco.transform.base.fs.FileManager.TARGET_FILE;
import static org.alfresco.transform.base.fs.FileManager.deleteFile;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import org.alfresco.transform.base.logging.LogEntry;
import org.springframework.web.servlet.handler.HandlerInterceptorAdapter;
/**
* TransformInterceptor
* <br/>
* Handles ThreadLocal Log entries for each request.
*/
public class TransformInterceptor extends HandlerInterceptorAdapter
{
@Override
public boolean preHandle(HttpServletRequest request,
HttpServletResponse response, Object handler)
{
LogEntry.start();
return true;
}
@Override
public void afterCompletion(HttpServletRequest request,
HttpServletResponse response, Object handler, Exception ex)
{
// TargetFile cannot be deleted until completion, otherwise 0 bytes are sent.
deleteFile(request, SOURCE_FILE);
deleteFile(request, TARGET_FILE);
LogEntry.complete();
}
}

106
t-engine-base/src/main/java/org/alfresco/transform/base/TransformRegistryImpl.java Normal file
View File

@@ -0,0 +1,106 @@
/*
* #%L
* Alfresco Repository
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
*
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.base;
import org.alfresco.transform.config.TransformConfig;
import org.alfresco.transform.registry.AbstractTransformRegistry;
import org.alfresco.transform.registry.CombinedTransformConfig;
import org.alfresco.transform.registry.TransformCache;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import javax.annotation.PostConstruct;
import java.util.Comparator;
import java.util.List;
import static org.alfresco.transform.config.CoreVersionDecorator.setCoreVersionOnSingleStepTransformers;
/**
* Used by clients to work out if a transformation is supported based on the engine_config.json.
*/
public class TransformRegistryImpl extends AbstractTransformRegistry
{
private static final Logger log = LoggerFactory.getLogger(TransformRegistryImpl.class);
@Autowired(required = false)
private List<TransformEngine> transformEngines;
@Value("${transform.core.version}")
private String coreVersion;
private TransformConfig transformConfigBeforeIncompleteTransformsAreRemoved;
@PostConstruct
public void init()
{
CombinedTransformConfig combinedTransformConfig = new CombinedTransformConfig();
if (transformEngines != null)
{
transformEngines.stream()
.sorted(Comparator.comparing(TransformEngine::getTransformEngineName))
.forEach(transformEngine -> {
TransformConfig transformConfig = transformEngine.getTransformConfig();
if (transformConfig != null)
{
setCoreVersionOnSingleStepTransformers(transformConfig, coreVersion);
combinedTransformConfig.addTransformConfig(transformConfig,
transformEngine.getTransformEngineName(), "---", this);
}
});
}
transformConfigBeforeIncompleteTransformsAreRemoved = combinedTransformConfig.buildTransformConfig();
combinedTransformConfig.combineTransformerConfig(this);
combinedTransformConfig.registerCombinedTransformers(this);
}
// Unlike other subclasses this class does not extend Data or replace it at run time.
private TransformCache data = new TransformCache();
public TransformConfig getTransformConfig()
{
return transformConfigBeforeIncompleteTransformsAreRemoved;
}
@Override
public TransformCache getData()
{
return data;
}
@Override
protected void logError(String msg)
{
log.error(msg);
}
@Override
protected void logWarn(String msg)
{
log.warn(msg);
}
}

103
t-engine-base/src/main/java/org/alfresco/transform/base/clients/AlfrescoSharedFileStoreClient.java Normal file
View File

@@ -0,0 +1,103 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.base.clients;
import static org.springframework.http.HttpMethod.POST;
import static org.springframework.http.MediaType.MULTIPART_FORM_DATA;
import java.io.File;
import org.alfresco.transform.common.TransformException;
import org.alfresco.transform.base.model.FileRefResponse;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.core.io.FileSystemResource;
import org.springframework.core.io.Resource;
import org.springframework.http.HttpEntity;
import org.springframework.http.HttpHeaders;
import org.springframework.http.ResponseEntity;
import org.springframework.util.LinkedMultiValueMap;
import org.springframework.web.client.HttpClientErrorException;
import org.springframework.web.client.RestTemplate;
/**
* Simple Rest client that call Alfresco Shared File Store
*/
public class AlfrescoSharedFileStoreClient
{
@Value("${fileStoreUrl}")
private String fileStoreUrl;
@Autowired
private RestTemplate restTemplate;
/**
* Retrieves a file from Shared File Store using given file reference
*
* @param fileRef File reference
* @return ResponseEntity<Resource>
*/
public ResponseEntity<Resource> retrieveFile(String fileRef)
{
try
{
return restTemplate.getForEntity(fileStoreUrl + "/" + fileRef,
org.springframework.core.io.Resource.class);
}
catch (HttpClientErrorException e)
{
throw new TransformException(e.getStatusCode().value(), e.getMessage(), e);
}
}
/**
* Stores given file in Shared File Store
*
* @param file File to be stored
* @return A FileRefResponse containing detail about file's reference
*/
public FileRefResponse saveFile(File file)
{
try
{
FileSystemResource value = new FileSystemResource(file.getAbsolutePath());
LinkedMultiValueMap<String, Object> map = new LinkedMultiValueMap<>();
map.add("file", value);
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MULTIPART_FORM_DATA);
HttpEntity<LinkedMultiValueMap<String, Object>> requestEntity = new HttpEntity<>(map,
headers);
ResponseEntity<FileRefResponse> responseEntity = restTemplate
.exchange(fileStoreUrl, POST, requestEntity, FileRefResponse.class);
return responseEntity.getBody();
}
catch (HttpClientErrorException e)
{
throw new TransformException(e.getStatusCode().value(), e.getMessage(), e);
}
}
}

95
t-engine-base/src/main/java/org/alfresco/transform/base/config/WebApplicationConfig.java Normal file
View File

@@ -0,0 +1,95 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.base.config;
import org.alfresco.transform.base.TransformInterceptor;
import org.alfresco.transform.base.TransformRegistryImpl;
import org.alfresco.transform.base.clients.AlfrescoSharedFileStoreClient;
import org.alfresco.transform.common.TransformerDebug;
import org.alfresco.transform.messages.TransformRequestValidator;
import org.alfresco.transform.registry.TransformServiceRegistry;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.env.Environment;
import org.springframework.web.client.RestTemplate;
import org.springframework.web.servlet.config.annotation.InterceptorRegistry;
import org.springframework.web.servlet.config.annotation.WebMvcConfigurer;
import static org.alfresco.transform.common.RequestParamMap.ENDPOINT_TRANSFORM;
@Configuration
@ComponentScan(basePackages = {"org.alfresco.transform"})
public class WebApplicationConfig implements WebMvcConfigurer
{
@Override
public void addInterceptors(InterceptorRegistry registry)
{
registry
.addInterceptor(transformInterceptor())
.addPathPatterns(ENDPOINT_TRANSFORM, "/live", "/ready");
}
@Bean
public TransformInterceptor transformInterceptor()
{
return new TransformInterceptor();
}
@Bean
public RestTemplate restTemplate()
{
return new RestTemplate();
}
@Bean
public AlfrescoSharedFileStoreClient alfrescoSharedFileStoreClient()
{
return new AlfrescoSharedFileStoreClient();
}
@Bean
public TransformRequestValidator transformRequestValidator()
{
return new TransformRequestValidator();
}
@Autowired Environment env;
@Bean
public TransformServiceRegistry transformRegistry()
{
return new TransformRegistryImpl();
}
@Bean
public TransformerDebug transformerDebug()
{
return new TransformerDebug().setIsTEngine(true);
}
}

91
t-engine-base/src/main/java/org/alfresco/transform/base/executors/AbstractCommandExecutor.java Normal file
View File

@@ -0,0 +1,91 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.base.executors;
import static org.alfresco.transform.base.executors.RuntimeExec.ExecutionResult;
import static org.springframework.http.HttpStatus.BAD_REQUEST;
import static org.springframework.http.HttpStatus.INTERNAL_SERVER_ERROR;
import java.io.File;
import java.util.Map;
import org.alfresco.transform.common.TransformException;
/**
*
*/
public abstract class AbstractCommandExecutor implements CommandExecutor
{
protected RuntimeExec transformCommand = createTransformCommand();
protected RuntimeExec checkCommand = createCheckCommand();
protected abstract RuntimeExec createTransformCommand();
protected abstract RuntimeExec createCheckCommand();
@Override
public void run(Map<String, String> properties, File targetFile, Long timeout)
{
timeout = timeout != null && timeout > 0 ? timeout : 0;
final ExecutionResult result = transformCommand.execute(properties, timeout);
if (result.getExitValue() != 0 && result.getStdErr() != null && result.getStdErr().length() > 0)
{
throw new TransformException(BAD_REQUEST.value(),
"Transformer exit code was not 0: \n" + result.getStdErr());
}
if (!targetFile.exists() || targetFile.length() == 0)
{
throw new TransformException(INTERNAL_SERVER_ERROR.value(),
"Transformer failed to create an output file");
}
}
@Override
public String version()
{
if (checkCommand != null)
{
final ExecutionResult result = checkCommand.execute();
if (result.getExitValue() != 0 && result.getStdErr() != null && result.getStdErr().length() > 0)
{
throw new TransformException(INTERNAL_SERVER_ERROR.value(),
"Transformer version check exit code was not 0: \n" + result);
}
final String version = result.getStdOut().trim();
if (version.isEmpty())
{
throw new TransformException(INTERNAL_SERVER_ERROR.value(),
"Transformer version check failed to create any output");
}
return version;
}
return "Version not checked";
}
}

71
t-engine-base/src/main/java/org/alfresco/transform/base/executors/CommandExecutor.java Normal file
View File

@@ -0,0 +1,71 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.base.executors;
import org.alfresco.transform.base.logging.LogEntry;
import java.io.File;
import java.util.HashMap;
import java.util.Map;
/**
* Basic interface for executing transformations via Shell commands
*
* @author Cezar Leahu
*/
public interface CommandExecutor extends Transformer
{
void run(Map<String, String> properties, File targetFile, Long timeout);
String version();
default void run(String options, File sourceFile, File targetFile,
Long timeout)
{
LogEntry.setOptions(options);
Map<String, String> properties = new HashMap<>();
properties.put("options", options);
properties.put("source", sourceFile.getAbsolutePath());
properties.put("target", targetFile.getAbsolutePath());
run(properties, targetFile, timeout);
}
default void run(String options, File sourceFile, String pageRange, File targetFile,
Long timeout)
{
LogEntry.setOptions(pageRange + (pageRange.isEmpty() ? "" : " ") + options);
Map<String, String> properties = new HashMap<>();
properties.put("options", options);
properties.put("source", sourceFile.getAbsolutePath() + pageRange);
properties.put("target", targetFile.getAbsolutePath());
run(properties, targetFile, timeout);
}
}

360
t-engine-base/src/main/java/org/alfresco/transform/base/executors/ExecParameterTokenizer.java Normal file
View File

@@ -0,0 +1,360 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.base.executors;
import static java.util.Collections.singletonList;
import java.io.Serializable;
import java.util.ArrayList;
import java.util.List;
import java.util.Objects;
import java.util.StringTokenizer;
/**
* DUPLICATED FROM *alfresco-core*.
*
* This class is used to tokenize strings used as parameters for {@link RuntimeExec} objects.
* Examples of such strings are as follows (ImageMagick-like parameters):
* <ul>
* <li><tt>-font Helvetica -pointsize 50</tt></li>
* <li><tt>-font Helvetica -pointsize 50 -draw "circle 100,100 150,150"</tt></li>
* <li><tt>-font Helvetica -pointsize 50 -draw "gravity south fill black text 0,12 'CopyRight'"</tt></li>
* </ul>
* The first is the simple case which would be parsed into Strings as follows:
* <tt>"-font", "Helvetica", "-pointsize", "50"</tt>
* <p/>
* The second is more complex in that it includes a quoted parameter, which would be parsed as a single String:
* <tt>"-font", "Helvetica", "-pointsize", "50", "circle 100,100 150,150"</tt>
* Note however that the quotation characters will be stripped from the token.
* <p/>
* The third shows an example with embedded quotation marks, which would parse to:
* <tt>"-font", "Helvetica", "-pointsize", "50", "gravity south fill black text 0,12 'CopyRight'"</tt>
* In this case, the embedded quotation marks (which must be different from those surrounding the parameter)
* are preserved in the extracted token.
* <p/>
* The class does not understand escaped quotes such as <tt>p1 p2 "a b c \"hello\" d" p4</tt>
*
* @author Neil Mc Erlean
* @since 3.4.2
*/
public class ExecParameterTokenizer
{
/**
* The string to be tokenized.
*/
private final String str;
/**
* The list of tokens, which will take account of quoted sections.
*/
private List<String> tokens;
public ExecParameterTokenizer(String str)
{
this.str = str;
}
/**
* This method returns the tokens in a parameter string.
* Any tokens not contained within single or double quotes will be tokenized in the normal
* way i.e. by using whitespace separators and the standard StringTokenizer algorithm.
* Any tokens which are contained within single or double quotes will be returned as single
* String instances and will have their quote marks removed.
* <p/>
* See above for examples.
*
* @throws NullPointerException if the string to be tokenized was null.
*/
public List<String> getAllTokens()
{
if (this.str == null)
{
throw new NullPointerException("Illegal null string cannot be tokenized.");
}
if (tokens == null)
{
tokens = new ArrayList<>();
// Preserve original behaviour from RuntimeExec.
if (str.indexOf('\'') == -1 && str.indexOf('"') == -1)
{
// Contains no quotes.
for (StringTokenizer standardTokenizer = new StringTokenizer(
str); standardTokenizer.hasMoreTokens(); )
{
tokens.add(standardTokenizer.nextToken());
}
}
else
{
// There are either single or double quotes or both.
// So we need to identify the quoted regions within the string.
List<Pair<Integer, Integer>> quotedRegions = new ArrayList<>();
for (Pair<Integer, Integer> next = identifyNextQuotedRegion(str, 0); next != null; )
{
quotedRegions.add(next);
next = identifyNextQuotedRegion(str, next.getSecond() + 1);
}
// Now we've got a List of index pairs identifying the quoted regions.
// We need to get substrings of quoted and unquoted blocks, whilst maintaining order.
List<Substring> substrings = getSubstrings(str, quotedRegions);
for (Substring r : substrings)
{
tokens.addAll(r.getTokens());
}
}
}
return this.tokens;
}
/**
* The substrings will be a list of quoted and unquoted substrings.
* The unquoted ones need to be further tokenized in the normal way.
* The quoted ones must not be tokenized, but need their quotes stripped off.
*/
private List<Substring> getSubstrings(String str,
List<Pair<Integer, Integer>> quotedRegionIndices)
{
List<Substring> result = new ArrayList<>();
int cursorPosition = 0;
for (Pair<Integer, Integer> nextQuotedRegionIndices : quotedRegionIndices)
{
if (cursorPosition < nextQuotedRegionIndices.getFirst())
{
int startIndexOfNextQuotedRegion = nextQuotedRegionIndices.getFirst() - 1;
result.add(new UnquotedSubstring(
str.substring(cursorPosition, startIndexOfNextQuotedRegion)));
}
result.add(new QuotedSubstring(str.substring(nextQuotedRegionIndices.getFirst(),
nextQuotedRegionIndices.getSecond())));
cursorPosition = nextQuotedRegionIndices.getSecond();
}
// We've processed all the quoted regions, but there may be a final unquoted region
if (cursorPosition < str.length() - 1)
{
result.add(new UnquotedSubstring(str.substring(cursorPosition, str.length() - 1)));
}
return result;
}
private Pair<Integer, Integer> identifyNextQuotedRegion(String str, int startingIndex)
{
int indexOfNextSingleQuote = str.indexOf('\'', startingIndex);
int indexOfNextDoubleQuote = str.indexOf('"', startingIndex);
if (indexOfNextSingleQuote == -1 && indexOfNextDoubleQuote == -1)
{
// If there are no more quoted regions
return null;
}
else if (indexOfNextSingleQuote > -1 && indexOfNextDoubleQuote > -1)
{
// If there are both single and double quotes in the remainder of the string
// Then select the closest quote.
int indexOfNextQuote = Math.min(indexOfNextSingleQuote, indexOfNextDoubleQuote);
char quoteChar = str.charAt(indexOfNextQuote);
return findIndexOfClosingQuote(str, indexOfNextQuote, quoteChar);
}
else
{
// Only one of the quote characters is present.
int indexOfNextQuote = Math.max(indexOfNextSingleQuote, indexOfNextDoubleQuote);
char quoteChar = str.charAt(indexOfNextQuote);
return findIndexOfClosingQuote(str, indexOfNextQuote, quoteChar);
}
}
private Pair<Integer, Integer> findIndexOfClosingQuote(String str, int indexOfStartingQuote,
char quoteChar)
{
// So we know which type of quote char we're dealing with. Either ' or ".
// Now we need to find the closing quote.
int indexAfterClosingQuote = str.indexOf(quoteChar,
indexOfStartingQuote + 1) + 1; // + 1 to search after opening quote. + 1 to give result including closing quote.
if (indexAfterClosingQuote == 0) // -1 + 1
{
// If no closing quote.
throw new IllegalArgumentException("No closing " + quoteChar + "quote in" + str);
}
return new Pair<>(indexOfStartingQuote, indexAfterClosingQuote);
}
/**
* Utility interface for a substring in a parameter string.
*/
public interface Substring
{
/**
* Gets all the tokens in a parameter string.
*/
List<String> getTokens();
}
/**
* A substring that is not surrounded by (single or double) quotes.
*/
public class UnquotedSubstring implements Substring
{
private final String regionString;
public UnquotedSubstring(String str)
{
this.regionString = str;
}
public List<String> getTokens()
{
StringTokenizer t = new StringTokenizer(regionString);
List<String> result = new ArrayList<>();
while (t.hasMoreTokens())
{
result.add(t.nextToken());
}
return result;
}
public String toString()
{
return UnquotedSubstring.class.getSimpleName() + ": '" + regionString + '\'';
}
}
/**
* A substring that is surrounded by (single or double) quotes.
*/
public class QuotedSubstring implements Substring
{
private final String regionString;
public QuotedSubstring(String str)
{
this.regionString = str;
}
public List<String> getTokens()
{
String stringWithoutQuotes = regionString.substring(1, regionString.length() - 1);
return singletonList(stringWithoutQuotes);
}
public String toString()
{
return QuotedSubstring.class.getSimpleName() + ": '" + regionString + '\'';
}
}
public static final class Pair<F, S> implements Serializable
{
private static final long serialVersionUID = -7406248421185630612L;
/**
* The first member of the pair.
*/
private F first;
/**
* The second member of the pair.
*/
private S second;
/**
* Make a new one.
*
* @param first The first member.
* @param second The second member.
*/
public Pair(F first, S second)
{
this.first = first;
this.second = second;
}
/**
* Get the first member of the tuple.
*
* @return The first member.
*/
public final F getFirst()
{
return first;
}
/**
* Get the second member of the tuple.
*
* @return The second member.
*/
public final S getSecond()
{
return second;
}
public final void setFirst(F first)
{
this.first = first;
}
public final void setSecond(S second)
{
this.second = second;
}
@Override public boolean equals(Object o)
{
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
Pair<?, ?> pair = (Pair<?, ?>) o;
return Objects.equals(first, pair.first) &&
Objects.equals(second, pair.second);
}
@Override public int hashCode()
{
return Objects.hash(first, second);
}
@Override
public String toString()
{
return "(" + first + ", " + second + ")";
}
}
}

40
t-engine-base/src/main/java/org/alfresco/transform/base/executors/JavaExecutor.java Normal file
View File

@@ -0,0 +1,40 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.base.executors;
import java.io.File;
/**
* Basic interface for executing transformations inside Java/JVM.
*
* @author Cezar Leahu
* @author adavis
*/
public interface JavaExecutor extends Transformer
{
void call(File sourceFile, File targetFile, String... args) throws Exception;
}

986
t-engine-base/src/main/java/org/alfresco/transform/base/executors/RuntimeExec.java Normal file
View File

@@ -0,0 +1,986 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.base.executors;
import static java.util.Collections.emptyMap;
import java.io.BufferedInputStream;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.Charset;
import java.nio.charset.UnsupportedCharsetException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashMap;
import java.util.HashSet;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.StringTokenizer;
import java.util.Timer;
import java.util.TimerTask;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* DUPLICATED FROM *alfresco-core*.
*
* This acts as a session similar to the <code>java.lang.Process</code>, but
* logs the system standard and error streams.
* <p>
* The bean can be configured to execute a command directly, or be given a map
* of commands keyed by the <i>os.name</i> Java system property. In this map,
* the default key that is used when no match is found is the
* <b>{@link #KEY_OS_DEFAULT *}</b> key.
* <p>
* Use the {@link #setProcessDirectory(String) processDirectory} property to change the default location
* from which the command executes. The process's environment can be configured using the
* {@link #setProcessProperties(Map) processProperties} property.
* <p>
* Commands may use placeholders, e.g.
* <pre><code>
* find
* -name
* ${filename}
* </code></pre>
* The <b>filename</b> property will be substituted for any supplied value prior to
* each execution of the command. Currently, no checks are made to get or check the
* properties contained within the command string. It is up to the client code to
* dynamically extract the properties required if the required properties are not
* known up front.
* <p>
* Sometimes, a variable may contain several arguments. . In this case, the arguments
* need to be tokenized using a standard <tt>StringTokenizer</tt>. To force tokenization
* of a value, use:
* <pre><code>
* SPLIT:${userArgs}
* </code></pre>
* You should not use this just to split up arguments that are known to require tokenization
* up front. The <b>SPLIT:</b> directive works for the entire argument and will not do anything
* if it is not at the beginning of the argument. Do not use <b>SPLIT:</b> to break up arguments
* that are fixed, so avoid doing this:
* <pre><code>
* SPLIT:ls -lih
* </code></pre>
* Instead, break the command up explicitly:
* <pre><code>
* ls
* -lih
* </code></pre>
*
* Tokenization of quoted parameter values is handled by ExecParameterTokenizer, which
* describes the support in more detail.
*
* @author Derek Hulley
*/
public class RuntimeExec
{
private static final Logger logger = LoggerFactory.getLogger(RuntimeExec.class);
/**
* the key to use when specifying a command for any other OS: <b>*</b>
*/
private static final String KEY_OS_DEFAULT = "*";
private static final String KEY_OS_NAME = "os.name";
private static final int BUFFER_SIZE = 1024;
private static final String VAR_OPEN = "${";
private static final String VAR_CLOSE = "}";
private static final String DIRECTIVE_SPLIT = "SPLIT:";
private String[] command;
private Charset charset = Charset.defaultCharset();
private boolean waitForCompletion = true;
private Map<String, String> defaultProperties = emptyMap();
private String[] processProperties;
private File processDirectory;
private final Set<Integer> errCodes;
private final Timer timer = new Timer(true);
/**
* Default constructor. Initialize this instance by setting individual properties.
*/
public RuntimeExec()
{
// set default error codes
errCodes = new HashSet<>(2);
errCodes.add(1);
errCodes.add(2);
}
public String toString()
{
final StringBuilder sb = new StringBuilder(256);
sb.append("RuntimeExec:\n").append(" command: ");
if (command == null)
{
// command is 'null', so there's nothing to toString
sb.append("'null'\n");
}
else
{
for (String cmdStr : command)
{
sb.append(cmdStr).append(" ");
}
sb.append("\n");
}
sb.append(" env props: ").append(Arrays.toString(processProperties)).append("\n")
.append(" dir: ").append(processDirectory).append("\n")
.append(" os: ").append(System.getProperty(KEY_OS_NAME)).append("\n");
return sb.toString();
}
/**
* Set the command to execute regardless of operating system
*
* @param command an array of strings representing the command (first entry) and arguments
* @since 3.0
*/
public void setCommand(String[] command)
{
this.command = command;
}
/**
* Sets the assumed charset of OUT and ERR streams generated by the executed command.
* This defaults to the system default charset: {@link Charset#defaultCharset()}.
*
* @param charsetCode a supported character set code
* @throws UnsupportedCharsetException if the characterset code is not recognised by Java
*/
public void setCharset(String charsetCode)
{
this.charset = Charset.forName(charsetCode);
}
/**
* Set whether to wait for completion of the command or not. If there is no wait for completion,
* then the return value of <i>out</i> and <i>err</i> buffers cannot be relied upon as the
* command may still be in progress. Failure is therefore not possible unless the calling thread
* waits for execution.
*
* @param waitForCompletion <tt>true</tt> (default) is to wait for the command to exit,
* or <tt>false</tt> to just return an exit code of 0 and whatever
* output is available at that point.
* @since 2.1
*/
public void setWaitForCompletion(boolean waitForCompletion)
{
this.waitForCompletion = waitForCompletion;
}
/**
* Supply a choice of commands to execute based on a mapping from the <i>os.name</i> system
* property to the command to execute. The {@link #KEY_OS_DEFAULT *} key can be used
* to get a command where there is not direct match to the operating system key.
* <p>
* Each command is an array of strings, the first of which represents the command and all subsequent
* entries in the array represent the arguments. All elements of the array will be checked for
* the presence of any substitution parameters (e.g. '{dir}'). The parameters can be set using the
* {@link #setDefaultProperties(Map) defaults} or by passing the substitution values into the
* {@link #execute(Map)} command.
* <p>
* If parameters passed may be multiple arguments, or if the values provided in the map are themselves
* collections of arguments (not recommended), then prefix the value with <b>SPLIT:</b> to ensure that
* the value is tokenized before being passed to the command. Any values that are not split, will be
* passed to the command as single arguments. For example:<br>
* '<b>SPLIT: dir . ..</b>' becomes '<b>dir</b>', '<b>.</b>' and '<b>..</b>'.<br>
* '<b>SPLIT: dir ${path}</b>' (if path is '<b>. ..</b>') becomes '<b>dir</b>', '<b>.</b>' and '<b>..</b>'.<br>
* The splitting occurs post-subtitution. Where the arguments are known, it is advisable to avoid
* <b>SPLIT:</b>.
*
* @param commandsByOS a map of command string arrays, keyed by operating system names
* @see #setDefaultProperties(Map)
* @since 3.0
*/
public void setCommandsAndArguments(Map<String, String[]> commandsByOS)
{
// get the current OS
String serverOs = System.getProperty(KEY_OS_NAME);
// attempt to find a match
String[] command = commandsByOS.get(serverOs);
if (command == null)
{
// go through the commands keys, looking for one that matches by regular expression matching
for (String osName : commandsByOS.keySet())
{
// Ignore * options. It is dealt with later.
if (osName.equals(KEY_OS_DEFAULT))
{
continue;
}
// Do regex match
if (serverOs.matches(osName))
{
command = commandsByOS.get(osName);
break;
}
}
// if there is still no command, then check for the wildcard
if (command == null)
{
command = commandsByOS.get(KEY_OS_DEFAULT);
}
}
// check
if (command == null)
{
throw new RuntimeException(
"No command found for OS " + serverOs + " or '" + KEY_OS_DEFAULT + "': \n" +
" commands: " + commandsByOS);
}
this.command = command;
}
/**
* Supply a choice of commands to execute based on a mapping from the <i>os.name</i> system
* property to the command to execute. The {@link #KEY_OS_DEFAULT *} key can be used
* to get a command where there is not direct match to the operating system key.
*
* @param commandsByOS a map of command string keyed by operating system names
* @deprecated Use {@link #setCommandsAndArguments(Map)}
*/
public void setCommandMap(Map<String, String> commandsByOS)
{
// This is deprecated, so issue a warning
logger.warn(
"The bean RuntimeExec property 'commandMap' has been deprecated;" +
" use 'commandsAndArguments' instead. See https://issues.alfresco.com/jira/browse/ETHREEOH-579.");
Map<String, String[]> fixed = new LinkedHashMap<>();
for (Map.Entry<String, String> entry : commandsByOS.entrySet())
{
String os = entry.getKey();
String unparsedCmd = entry.getValue();
StringTokenizer tokenizer = new StringTokenizer(unparsedCmd);
String[] cmd = new String[tokenizer.countTokens()];
for (int i = 0; i < cmd.length; i++)
{
cmd[i] = tokenizer.nextToken();
}
fixed.put(os, cmd);
}
setCommandsAndArguments(fixed);
}
/**
* Set the default command-line properties to use when executing the command.
* These are properties that substitute variables defined in the command string itself.
* Properties supplied during execution will overwrite the default properties.
* <p>
* <code>null</code> properties will be treated as an empty string for substitution
* purposes.
*
* @param defaultProperties property values
*/
public void setDefaultProperties(Map<String, String> defaultProperties)
{
this.defaultProperties = defaultProperties;
}
/**
* Set additional runtime properties (environment properties) that will used
* by the executing process.
* <p>
* Any keys or properties that start and end with <b>${...}</b> will be removed on the assumption
* that these are unset properties. <tt>null</tt> values are translated to empty strings.
* All keys and values are trimmed of leading and trailing whitespace.
*
* @param processProperties Runtime process properties
* @see Runtime#exec(String, String[], java.io.File)
*/
public void setProcessProperties(Map<String, String> processProperties)
{
ArrayList<String> processPropList = new ArrayList<>(processProperties.size());
boolean hasPath = false;
String systemPath = System.getenv("PATH");
for (Map.Entry<String, String> entry : processProperties.entrySet())
{
String key = entry.getKey();
String value = entry.getValue();
if (key == null)
{
continue;
}
if (value == null)
{
value = "";
}
key = key.trim();
value = value.trim();
if (key.startsWith(VAR_OPEN) && key.endsWith(VAR_CLOSE))
{
continue;
}
if (value.startsWith(VAR_OPEN) && value.endsWith(VAR_CLOSE))
{
continue;
}
// If a path is specified, prepend it to the existing path
if (key.equals("PATH"))
{
if (systemPath != null && systemPath.length() > 0)
{
processPropList.add(key + "=" + value + File.pathSeparator + systemPath);
}
else
{
processPropList.add(key + "=" + value);
}
hasPath = true;
}
else
{
processPropList.add(key + "=" + value);
}
}
// If a path was not specified, inherit the current one
if (!hasPath && systemPath != null && systemPath.length() > 0)
{
processPropList.add("PATH=" + systemPath);
}
this.processProperties = processPropList.toArray(new String[0]);
}
/**
* Adds a property to existed processProperties.
* Property should not be null or empty.
* If property with the same value already exists then no change is made.
* If property exists with a different value then old value is replaced with the new one.
*
* @param name - property name
* @param value - property value
*/
public void setProcessProperty(String name, String value)
{
boolean set = false;
if (name == null || value == null)
{
return;
}
name = name.trim();
value = value.trim();
if (name.isEmpty() || value.isEmpty())
{
return;
}
String property = name + "=" + value;
for (String prop : this.processProperties)
{
if (prop.equals(property))
{
set = true;
break;
}
if (prop.startsWith(name))
{
String oldValue = prop.split("=")[1];
prop.replace(oldValue, value);
set = true;
}
}
if (!set)
{
String[] existedProperties = this.processProperties;
int epl = existedProperties.length;
String[] newProperties = Arrays.copyOf(existedProperties, epl + 1);
newProperties[epl] = property;
this.processProperties = newProperties;
}
}
/**
* Set the runtime location from which the command is executed.
* <p>
* If the value is an unsubsititued variable (<b>${...}</b>) then it is ignored.
* If the location is not visible at the time of setting, a warning is issued only.
*
* @param processDirectory the runtime location from which to execute the command
*/
public void setProcessDirectory(String processDirectory)
{
if (processDirectory.startsWith(VAR_OPEN) && processDirectory.endsWith(VAR_CLOSE))
{
this.processDirectory = null;
}
else
{
this.processDirectory = new File(processDirectory);
if (!this.processDirectory.exists())
{
logger.warn(
"The runtime process directory is not visible when setting property " +
"'processDirectory': \n{}", this);
}
}
}
/**
* A comma or space separated list of values that, if returned by the executed command,
* indicate an error value. This defaults to <b>"1, 2"</b>.
*
* @param errCodesStr the error codes for the execution
*/
public void setErrorCodes(String errCodesStr)
{
errCodes.clear();
StringTokenizer tokenizer = new StringTokenizer(errCodesStr, " ,");
while (tokenizer.hasMoreElements())
{
String errCodeStr = tokenizer.nextToken();
// attempt to convert it to an integer
try
{
int errCode = Integer.parseInt(errCodeStr);
this.errCodes.add(errCode);
}
catch (NumberFormatException e)
{
throw new RuntimeException(
"Property 'errorCodes' must be comma-separated list of integers: " + errCodesStr);
}
}
}
/**
* Executes the command using the default properties
*
* @see #execute(Map)
*/
public ExecutionResult execute()
{
return execute(defaultProperties);
}
/**
* Executes the statement that this instance was constructed with.
*
* @param properties the properties that the command might be executed with.
* <code>null</code> properties will be treated as an empty string for substitution
* purposes.
* @return Returns the full execution results
*/
public ExecutionResult execute(Map<String, String> properties)
{
return execute(properties, -1);
}
/**
* Executes the statement that this instance was constructed with an optional
* timeout after which the command is asked to
*
* @param properties the properties that the command might be executed with.
* <code>null</code> properties will be treated as an empty string for substitution
* purposes.
* @param timeoutMs a timeout after which {@link Process#destroy()} is called.
* ignored if less than or equal to zero. Note this method does not guarantee
* to terminate the process (it is not a kill -9).
* @return Returns the full execution results
*/
public ExecutionResult execute(Map<String, String> properties, final long timeoutMs)
{
int defaultFailureExitValue = errCodes.size() > 0 ? ((Integer) errCodes.toArray()[0]) : 1;
// check that the command has been set
if (command == null)
{
throw new RuntimeException("Runtime command has not been set: \n" + this);
}
// create the properties
Runtime runtime = Runtime.getRuntime();
Process process;
String[] commandToExecute = null;
try
{
// execute the command with full property replacement
commandToExecute = getCommand(properties);
final Process thisProcess = runtime.exec(commandToExecute, processProperties,
processDirectory);
process = thisProcess;
if (timeoutMs > 0)
{
final String[] command = commandToExecute;
timer.schedule(new TimerTask()
{
@Override
public void run()
{
// Only try to kill the process if it is still running
try
{
thisProcess.exitValue();
}
catch (IllegalThreadStateException stillRunning)
{
logger.debug(
"Process has taken too long ({} seconds). Killing process {}",
timeoutMs / 1000, Arrays.deepToString(command));
}
}
}, timeoutMs);
}
}
catch (IOException e)
{
// The process could not be executed here, so just drop out with an appropriate error state
String execOut = "";
String execErr = e.getMessage();
ExecutionResult result = new ExecutionResult(null, commandToExecute, errCodes,
defaultFailureExitValue, execOut, execErr);
logFullEnvironmentDump(result);
return result;
}
// create the stream gobblers
InputStreamReaderThread stdOutGobbler = new InputStreamReaderThread(
process.getInputStream(), charset);
InputStreamReaderThread stdErrGobbler = new InputStreamReaderThread(
process.getErrorStream(), charset);
// start gobbling
stdOutGobbler.start();
stdErrGobbler.start();
// wait for the process to finish
int exitValue = 0;
try
{
if (waitForCompletion)
{
exitValue = process.waitFor();
}
}
catch (InterruptedException e)
{
// process was interrupted - generate an error message
stdErrGobbler.addToBuffer(e.toString());
exitValue = defaultFailureExitValue;
}
if (waitForCompletion)
{
// ensure that the stream gobblers get to finish
stdOutGobbler.waitForCompletion();
stdErrGobbler.waitForCompletion();
}
// get the stream values
String execOut = stdOutGobbler.getBuffer();
String execErr = stdErrGobbler.getBuffer();
// construct the return value
ExecutionResult result = new ExecutionResult(process, commandToExecute, errCodes, exitValue,
execOut, execErr);
// done
logFullEnvironmentDump(result);
return result;
}
/**
* Dump the full environment in debug mode
*/
private void logFullEnvironmentDump(ExecutionResult result)
{
if (logger.isTraceEnabled())
{
StringBuilder sb = new StringBuilder();
sb.append(result);
// Environment variables modified by Alfresco
if (processProperties != null && processProperties.length > 0)
{
sb.append("\n modified environment: ");
for (String property : processProperties)
{
sb.append("\n ");
sb.append(property);
}
}
// Dump the full environment
sb.append("\n existing environment: ");
Map<String, String> envVariables = System.getenv();
for (Map.Entry<String, String> entry : envVariables.entrySet())
{
String name = entry.getKey();
String value = entry.getValue();
sb.append("\n ");
sb.append(name).append("=").append(value);
}
logger.trace(sb.toString());
}
logger.debug("Result: " + result.toString());
// close output stream (connected to input stream of native subprocess)
}
/**
* @return Returns the command that will be executed if no additional properties
* were to be supplied
*/
public String[] getCommand()
{
return getCommand(defaultProperties);
}
/**
* Get the command that will be executed post substitution.
* <p>
* <code>null</code> properties will be treated as an empty string for substitution
* purposes.
*
* @param properties the properties that the command might be executed with
* @return Returns the command that will be executed should the additional properties
* be supplied
*/
public String[] getCommand(Map<String, String> properties)
{
Map<String, String> execProperties;
if (properties == defaultProperties)
{
// we are just using the default properties
execProperties = defaultProperties;
}
else
{
execProperties = new HashMap<>(defaultProperties);
// overlay the supplied properties
execProperties.putAll(properties);
}
// Perform the substitution for each element of the command
ArrayList<String> adjustedCommandElements = new ArrayList<>(20);
for (String s : command)
{
StringBuilder sb = new StringBuilder(s);
for (Map.Entry<String, String> entry : execProperties.entrySet())
{
String key = entry.getKey();
String value = entry.getValue();
// ignore null
if (value == null)
{
value = "";
}
// progressively replace the property in the command
key = (VAR_OPEN + key + VAR_CLOSE);
int index = sb.indexOf(key);
while (index > -1)
{
// replace
sb.replace(index, index + key.length(), value);
// get the next one
index = sb.indexOf(key, index + 1);
}
}
String adjustedValue = sb.toString();
// Now SPLIT: it
if (adjustedValue.startsWith(DIRECTIVE_SPLIT))
{
String unsplitAdjustedValue = sb.substring(DIRECTIVE_SPLIT.length());
// There may be quoted arguments here (see ALF-7482)
ExecParameterTokenizer quoteAwareTokenizer = new ExecParameterTokenizer(
unsplitAdjustedValue);
List<String> tokens = quoteAwareTokenizer.getAllTokens();
adjustedCommandElements.addAll(tokens);
}
else
{
adjustedCommandElements.add(adjustedValue);
}
}
// done
return adjustedCommandElements.toArray(new String[0]);
}
/**
* Object to carry the results of an execution to the caller.
*
* @author Derek Hulley
*/
public static class ExecutionResult
{
private final Process process;
private final String[] command;
private final Set<Integer> errCodes;
private final int exitValue;
private final String stdOut;
private final String stdErr;
/**
* @param process the process attached to Java - <tt>null</tt> is allowed
*/
private ExecutionResult(
final Process process,
final String[] command,
final Set<Integer> errCodes,
final int exitValue,
final String stdOut,
final String stdErr)
{
this.process = process;
this.command = command;
this.errCodes = errCodes;
this.exitValue = exitValue;
this.stdOut = stdOut;
this.stdErr = stdErr;
}
@Override
public String toString()
{
String out = stdOut.length() > 250 ? stdOut.substring(0, 250) : stdOut;
String err = stdErr.length() > 250 ? stdErr.substring(0, 250) : stdErr;
StringBuilder sb = new StringBuilder(128);
sb.append("Execution result: \n")
.append(" os: ").append(System.getProperty(KEY_OS_NAME)).append("\n")
.append(" command: ");
appendCommand(sb, command).append("\n")
.append(" succeeded: ").append(getSuccess()).append("\n")
.append(" exit code: ").append(exitValue).append("\n")
.append(" out: ").append(out).append("\n")
.append(" err: ").append(err);
return sb.toString();
}
/**
* Appends the command in a form that make running from the command line simpler.
* It is not a real attempt at making a command given all the operating system
* and shell options, but makes copy, paste and edit a bit simpler.
*/
private StringBuilder appendCommand(StringBuilder sb, String[] command)
{
boolean arg = false;
for (String element : command)
{
if (element == null)
{
continue;
}
if (arg)
{
sb.append(' ');
}
else
{
arg = true;
}
boolean escape = element.indexOf(' ') != -1 || element.indexOf('>') != -1;
if (escape)
{
sb.append("\"");
}
sb.append(element);
if (escape)
{
sb.append("\"");
}
}
return sb;
}
/**
* A helper method to force a kill of the process that generated this result. This is
* useful in cases where the process started is not expected to exit, or doesn't exit
* quickly. If the {@linkplain RuntimeExec#setWaitForCompletion(boolean) "wait for completion"}
* flag is <tt>false</tt> then the process may still be running when this result is returned.
*
* @return <tt>true</tt> if the process was killed, otherwise <tt>false</tt>
*/
public boolean killProcess()
{
if (process == null)
{
return true;
}
try
{
process.destroy();
return true;
}
catch (Throwable e)
{
logger.warn(e.getMessage());
return false;
}
}
/**
* @param exitValue the command exit value
* @return Returns true if the code is a listed failure code
* @see #setErrorCodes(String)
*/
private boolean isFailureCode(int exitValue)
{
return errCodes.contains(exitValue);
}
/**
* @return Returns true if the command was deemed to be successful according to the
* failure codes returned by the execution.
*/
public boolean getSuccess()
{
return !isFailureCode(exitValue);
}
public int getExitValue()
{
return exitValue;
}
public String getStdOut()
{
return stdOut;
}
public String getStdErr()
{
return stdErr;
}
}
/**
* Gobbles an <code>InputStream</code> and writes it into a
* <code>StringBuffer</code>
* <p>
* The reading of the input stream is buffered.
*/
public static class InputStreamReaderThread extends Thread
{
private final InputStream is;
private final Charset charset;
private final StringBuffer buffer; // we require the synchronization
private boolean completed;
/**
* @param is an input stream to read - it will be wrapped in a buffer
* for reading
*/
public InputStreamReaderThread(InputStream is, Charset charset)
{
super();
setDaemon(true); // must not hold up the VM if it is terminating
this.is = is;
this.charset = charset;
this.buffer = new StringBuffer(BUFFER_SIZE);
this.completed = false;
}
public synchronized void run()
{
completed = false;
byte[] bytes = new byte[BUFFER_SIZE];
try (InputStream tempIs = new BufferedInputStream(is, BUFFER_SIZE))
{
int count = -2;
while (count != -1)
{
// do we have something previously read?
if (count > 0)
{
String toWrite = new String(bytes, 0, count, charset.name());
buffer.append(toWrite);
}
// read the next set of bytes
count = tempIs.read(bytes);
}
// done
}
catch (IOException e)
{
throw new RuntimeException("Unable to read stream", e);
}
finally
{
// The thread has finished consuming the stream
completed = true;
// Notify waiters
this.notifyAll(); // Note: Method is synchronized
}
}
/**
* Waits for the run to complete.
* <p>
* <b>Remember to <code>start</code> the thread first
*/
public synchronized void waitForCompletion()
{
while (!completed)
{
try
{
// release our lock and wait a bit
this.wait(1000L); // 200 ms
}
catch (InterruptedException ignore)
{
}
}
}
/**
* @param msg the message to add to the buffer
*/
public void addToBuffer(String msg)
{
buffer.append(msg);
}
public boolean isComplete()
{
return completed;
}
/**
* @return Returns the current state of the buffer
*/
public String getBuffer()
{
return buffer.toString();
}
}
}

130
t-engine-base/src/main/java/org/alfresco/transform/base/executors/Transformer.java Normal file
View File

@@ -0,0 +1,130 @@
package org.alfresco.transform.base.executors;
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
import org.alfresco.transform.common.TransformException;
import java.io.File;
import java.util.Map;
import static org.alfresco.transform.common.Mimetype.MIMETYPE_METADATA_EMBED;
import static org.alfresco.transform.common.Mimetype.MIMETYPE_METADATA_EXTRACT;
import static org.alfresco.transform.base.util.RequestParamMap.TRANSFORM_NAME_PARAMETER;
import static org.springframework.http.HttpStatus.BAD_REQUEST;
import static org.springframework.http.HttpStatus.INTERNAL_SERVER_ERROR;
/**
* Basic interface for executing transforms and metadata extract or embed actions.
*
* @author adavis
*/
public interface Transformer
{
/**
* @return A unique transformer id,
*
*/
String getTransformerId();
default void transform(String sourceMimetype, String targetMimetype, Map<String, String> transformOptions,
File sourceFile, File targetFile) throws TransformException {
final String transformName = transformOptions.remove(TRANSFORM_NAME_PARAMETER);
transformExtractOrEmbed(transformName, sourceMimetype, targetMimetype, transformOptions, sourceFile, targetFile);
}
default void transformExtractOrEmbed(String transformName, String sourceMimetype, String targetMimetype,
Map<String, String> transformOptions,
File sourceFile, File targetFile) throws TransformException
{
try
{
if (MIMETYPE_METADATA_EXTRACT.equals(targetMimetype))
{
extractMetadata(transformName, sourceMimetype, targetMimetype, transformOptions, sourceFile, targetFile);
}
else if (MIMETYPE_METADATA_EMBED.equals(targetMimetype))
{
embedMetadata(transformName, sourceMimetype, targetMimetype, transformOptions, sourceFile, targetFile);
}
else
{
transform(transformName, sourceMimetype, targetMimetype, transformOptions, sourceFile, targetFile);
}
}
catch (TransformException e)
{
throw e;
}
catch (IllegalArgumentException e)
{
throw new TransformException(BAD_REQUEST.value(), getMessage(e), e);
}
catch (Exception e)
{
throw new TransformException(INTERNAL_SERVER_ERROR.value(), getMessage(e), e);
}
if (!targetFile.exists())
{
throw new TransformException(INTERNAL_SERVER_ERROR.value(),
"Transformer failed to create an output file. Target file does not exist.");
}
if (sourceFile.length() > 0 && targetFile.length() == 0)
{
throw new TransformException(INTERNAL_SERVER_ERROR.value(),
"Transformer failed to create an output file. Target file is empty but source file was not empty.");
}
}
private static String getMessage(Exception e)
{
return e.getMessage() == null ? e.getClass().getSimpleName() : e.getMessage();
}
default void transform(String transformName, String sourceMimetype, String targetMimetype,
Map<String, String> transformOptions,
File sourceFile, File targetFile) throws Exception
{
}
default void extractMetadata(String transformName, String sourceMimetype, String targetMimetype,
Map<String, String> transformOptions,
File sourceFile, File targetFile) throws Exception
{
}
/**
* @deprecated The content repository has no non test embed metadata implementations.
* This code exists in case there are custom implementations, that need to be converted to T-Engines.
* It is simply a copy and paste from the content repository and has received limited testing.
*/
default void embedMetadata(String transformName, String sourceMimetype, String targetMimetype,
Map<String, String> transformOptions,
File sourceFile, File targetFile) throws Exception
{
}
}

249
t-engine-base/src/main/java/org/alfresco/transform/base/fs/FileManager.java Normal file
View File

@@ -0,0 +1,249 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.base.fs;
import static org.springframework.http.HttpHeaders.CONTENT_DISPOSITION;
import static org.springframework.http.HttpStatus.BAD_REQUEST;
import static org.springframework.http.HttpStatus.INSUFFICIENT_STORAGE;
import static org.springframework.http.HttpStatus.INTERNAL_SERVER_ERROR;
import static org.springframework.util.StringUtils.getFilename;
import static org.springframework.util.StringUtils.getFilenameExtension;
import java.io.File;
import java.io.IOException;
import java.net.MalformedURLException;
import java.nio.file.Files;
import java.nio.file.StandardCopyOption;
import java.util.Arrays;
import javax.servlet.http.HttpServletRequest;
import org.alfresco.transform.common.ExtensionService;
import org.alfresco.transform.common.TransformException;
import org.alfresco.transform.base.logging.LogEntry;
import org.springframework.core.io.Resource;
import org.springframework.core.io.UrlResource;
import org.springframework.http.HttpHeaders;
import org.springframework.http.ResponseEntity;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.util.UriUtils;
public class FileManager
{
public static final String SOURCE_FILE = "sourceFile";
public static final String TARGET_FILE = "targetFile";
private static final String FILENAME = "filename=";
public static File createTargetFile(HttpServletRequest request, String filename)
{
File file = buildFile(filename);
request.setAttribute(TARGET_FILE, file);
return file;
}
public static File buildFile(String filename)
{
filename = checkFilename(false, filename);
LogEntry.setTarget(filename);
return TempFileProvider.createTempFile("target_", "_" + filename);
}
public static void deleteFile(final File file) throws Exception
{
if (!file.delete())
{
throw new Exception("Failed to delete file");
}
}
private static String checkFilename(boolean source, String filename)
{
filename = getFilename(filename);
if (filename == null || filename.isEmpty())
{
String sourceOrTarget = source ? "source" : "target";
int statusCode = source ? BAD_REQUEST.value() : INTERNAL_SERVER_ERROR.value();
throw new TransformException(statusCode,
"The " + sourceOrTarget + " filename was not supplied");
}
return filename;
}
private static void save(MultipartFile multipartFile, File file)
{
try
{
Files.copy(multipartFile.getInputStream(), file.toPath(),
StandardCopyOption.REPLACE_EXISTING);
}
catch (IOException e)
{
throw new TransformException(INSUFFICIENT_STORAGE.value(),
"Failed to store the source file", e);
}
}
public static void save(Resource body, File file)
{
try
{
Files.copy(body.getInputStream(), file.toPath(), StandardCopyOption.REPLACE_EXISTING);
}
catch (IOException e)
{
throw new TransformException(INSUFFICIENT_STORAGE.value(),
"Failed to store the source file", e);
}
}
private static Resource load(File file)
{
try
{
Resource resource = new UrlResource(file.toURI());
if (resource.exists() || resource.isReadable())
{
return resource;
}
else
{
throw new TransformException(INTERNAL_SERVER_ERROR.value(),
"Could not read the target file: " + file.getPath());
}
}
catch (MalformedURLException e)
{
throw new TransformException(INTERNAL_SERVER_ERROR.value(),
"The target filename was malformed: " + file.getPath(), e);
}
}
public static String getFilenameFromContentDisposition(HttpHeaders headers)
{
String filename = "";
String contentDisposition = headers.getFirst(CONTENT_DISPOSITION);
if (contentDisposition != null)
{
String[] strings = contentDisposition.split("; *");
filename = Arrays.stream(strings)
.filter(s -> s.startsWith(FILENAME))
.findFirst()
.map(s -> s.substring(FILENAME.length()))
.orElse("");
}
return filename;
}
public static String createTargetFileName(final String fileName, String sourceMimetype, String targetMimetype)
{
String targetExtension = ExtensionService.getExtensionForTargetMimetype(targetMimetype, sourceMimetype);
final String sourceFilename = getFilename(fileName);
if (sourceFilename == null || sourceFilename.isEmpty())
{
return null;
}
final String ext = getFilenameExtension(sourceFilename);
if (ext == null || ext.isEmpty())
{
return sourceFilename + '.' + targetExtension;
}
return sourceFilename.substring(0, sourceFilename.length() - ext.length() - 1) + '.' + targetExtension;
}
public static File createSourceFile(HttpServletRequest request, MultipartFile multipartFile)
{
String filename = multipartFile.getOriginalFilename();
long size = multipartFile.getSize();
filename = checkFilename(true, filename);
File file = TempFileProvider.createTempFile("source_", "_" + filename);
request.setAttribute(SOURCE_FILE, file);
save(multipartFile, file);
LogEntry.setSource(filename, size);
return file;
}
@SuppressWarnings("ResultOfMethodCallIgnored")
public static void deleteFile(HttpServletRequest request, String attributeName)
{
File file = (File) request.getAttribute(attributeName);
if (file != null)
{
file.delete();
}
}
public static ResponseEntity<Resource> createAttachment(String targetFilename, File
targetFile)
{
Resource targetResource = load(targetFile);
targetFilename = UriUtils.encodePath(getFilename(targetFilename), "UTF-8");
return ResponseEntity.ok().header(CONTENT_DISPOSITION,
"attachment; filename*= UTF-8''" + targetFilename).body(targetResource);
}
/**
* TempFileProvider - Duplicated and adapted from alfresco-core.
*/
public static class TempFileProvider
{
public static File createTempFile(final String prefix, final String suffix)
{
final File directory = getTempDir();
try
{
return File.createTempFile(prefix, suffix, directory);
}
catch (IOException e)
{
throw new RuntimeException(
"Failed to created temp file: \n prefix: " + prefix +
"\n suffix: " + suffix + "\n directory: " + directory, e);
}
}
private static File getTempDir()
{
final String dirName = "Alfresco";
final String systemTempDirPath = System.getProperty("java.io.tmpdir");
if (systemTempDirPath == null)
{
throw new RuntimeException("System property not available: java.io.tmpdir");
}
final File systemTempDir = new File(systemTempDirPath);
final File tempDir = new File(systemTempDir, dirName);
if (!tempDir.exists() && !tempDir.mkdirs() && !tempDir.exists())
{
throw new RuntimeException("Failed to create temp directory: " + tempDir);
}
return tempDir;
}
}
}

285
t-engine-base/src/main/java/org/alfresco/transform/base/logging/LogEntry.java Normal file
View File

@@ -0,0 +1,285 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.base.logging;
import static java.lang.Math.max;
import static org.springframework.http.HttpStatus.OK;
import java.text.SimpleDateFormat;
import java.util.Collection;
import java.util.Date;
import java.util.Deque;
import java.util.concurrent.ConcurrentLinkedDeque;
import java.util.concurrent.atomic.AtomicInteger;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* Provides setter and getter methods to allow the current Thread to set various log properties and for these
* values to be retrieved. The {@link #complete()} method should be called at the end of a request to flush the
* current entry to an internal log Collection of the latest entries. The {@link #getLog()} method is used to obtain
* access to this collection.
*/
public final class LogEntry
{
private static final Logger logger = LoggerFactory.getLogger(LogEntry.class);
// TODO allow ProbeTestTransform to find out if there are any transforms running longer than the max time.
private static final AtomicInteger count = new AtomicInteger(0);
private static final Deque<LogEntry> log = new ConcurrentLinkedDeque<>();
private static final int MAX_LOG_SIZE = 10;
private static final SimpleDateFormat HH_MM_SS = new SimpleDateFormat("HH:mm:ss");
private static final ThreadLocal<LogEntry> currentLogEntry = ThreadLocal.withInitial(() -> {
LogEntry logEntry = new LogEntry();
if (log.size() >= MAX_LOG_SIZE)
{
log.removeLast();
}
log.addFirst(logEntry);
return logEntry;
});
private final int id = count.incrementAndGet();
private final long start = System.currentTimeMillis();
private int statusCode;
private long durationStreamIn;
private long durationTransform = -1;
private long durationStreamOut = -1;
private String source;
private long sourceSize;
private String target;
private long targetSize = -1;
private String options;
private String message;
@Override
public String toString()
{
StringBuilder sb = new StringBuilder();
append(sb, Integer.toString(getId()));
append(sb, HH_MM_SS.format(getDate()));
append(sb, Integer.toString(getStatusCode()));
append(sb, getDuration());
append(sb, getSource());
append(sb, getSourceSize());
append(sb, getTarget());
append(sb, getTargetSize());
append(sb, getOptions());
sb.append(getMessage());
return sb.toString();
}
private void append(StringBuilder sb, String value)
{
if (value != null && !value.isEmpty() && !"0bytes".equals(value))
{
sb.append(value);
sb.append(' ');
}
}
public static Collection<LogEntry> getLog()
{
return log;
}
public static void start()
{
currentLogEntry.get();
}
public static void setSource(String source, long sourceSize)
{
LogEntry logEntry = currentLogEntry.get();
logEntry.source = getExtension(source);
logEntry.sourceSize = sourceSize;
logEntry.durationStreamIn = System.currentTimeMillis() - logEntry.start;
}
public static void setTarget(String target)
{
currentLogEntry.get().target = getExtension(target);
}
private static String getExtension(String filename)
{
int i = filename.lastIndexOf('.');
if (i != -1)
{
filename = filename.substring(i + 1);
}
return filename;
}
public static void setTargetSize(long targetSize)
{
currentLogEntry.get().targetSize = targetSize;
}
public static void setOptions(String options)
{
currentLogEntry.get().options = options;
}
public static long setStatusCodeAndMessage(int statusCode, String message)
{
LogEntry logEntry = currentLogEntry.get();
logEntry.statusCode = statusCode;
logEntry.message = message;
logEntry.durationTransform = System.currentTimeMillis() - logEntry.start - logEntry.durationStreamIn;
return logEntry.durationTransform;
}
public static void complete()
{
LogEntry logEntry = currentLogEntry.get();
if (logEntry.statusCode == OK.value())
{
logEntry.durationStreamOut = System.currentTimeMillis() - logEntry.start -
logEntry.durationStreamIn - max(logEntry.durationTransform, 0);
}
currentLogEntry.remove();
if (logger.isDebugEnabled())
{
logger.debug(logEntry.toString());
}
}
public int getId()
{
return id;
}
public Date getDate()
{
return new Date(start);
}
public int getStatusCode()
{
return statusCode;
}
public String getDuration()
{
long duration = durationStreamIn + max(durationTransform, 0) + max(durationStreamOut, 0);
return duration <= 5
? ""
: time(duration) +
" (" +
(time(durationStreamIn) + ' ' +
time(durationTransform) + ' ' +
time(durationStreamOut)).trim() +
")";
}
public String getSource()
{
return source;
}
public String getSourceSize()
{
return size(sourceSize);
}
public String getTarget()
{
return target;
}
public String getTargetSize()
{
return size(targetSize);
}
public String getOptions()
{
return options;
}
public String getMessage()
{
return message;
}
private String time(long ms)
{
return ms == -1 ? "" : size(ms, "1ms",
new String[]{"ms", "s", "min", "hr"},
new long[]{1000, 60 * 1000, 60 * 60 * 1000, Long.MAX_VALUE});
}
private String size(long size)
{
// TODO fix numeric overflow in TB expression
return size == -1 ? "" : size(size, "1 byte",
new String[]{"bytes", " KB", " MB", " GB", " TB"},
new long[]{1024, 1024 * 1024, 1024 * 1024 * 1024, 1024 * 1024 * 1024 * 1024, Long.MAX_VALUE});
}
private String size(long size, String singleValue, String[] units, long[] dividers)
{
if (size == 1)
{
return singleValue;
}
long divider = 1;
for (int i = 0; i < units.length - 1; i++)
{
long nextDivider = dividers[i];
if (size < nextDivider)
{
return unitFormat(size, divider, units[i]);
}
divider = nextDivider;
}
return unitFormat(size, divider, units[units.length - 1]);
}
private String unitFormat(long size, long divider, String unit)
{
size = size * 10 / divider;
int decimalPoint = (int) size % 10;
StringBuilder sb = new StringBuilder();
sb.append(size / 10);
if (decimalPoint != 0)
{
sb.append(".");
sb.append(decimalPoint);
}
sb.append(unit);
return sb.toString();
}
}

35
t-engine-base/src/main/java/org/alfresco/transform/base/logging/StandardMessages.java Normal file
View File

@@ -0,0 +1,35 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.base.logging;
public interface StandardMessages
{
String LICENCE =
"If the Alfresco software was purchased under a paid Alfresco license, the terms of the paid license agreement \n" +
"will prevail. Otherwise, the software is provided under terms of the GNU LGPL v3 license. \n" +
"See the license at http://www.gnu.org/licenses/lgpl-3.0.txt. or in /LICENSE.txt \n\n";
}

108
t-engine-base/src/main/java/org/alfresco/transform/base/messaging/MessagingConfig.java Normal file
View File

@@ -0,0 +1,108 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.base.messaging;
import javax.jms.ConnectionFactory;
import javax.jms.Queue;
import org.alfresco.transform.messages.TransformRequestValidator;
import org.apache.activemq.command.ActiveMQQueue;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.jms.annotation.JmsListenerConfigurer;
import org.springframework.jms.config.DefaultJmsListenerContainerFactory;
import org.springframework.jms.config.JmsListenerEndpointRegistrar;
import org.springframework.jms.connection.JmsTransactionManager;
import org.springframework.lang.NonNull;
import org.springframework.messaging.handler.annotation.support.DefaultMessageHandlerMethodFactory;
import org.springframework.transaction.PlatformTransactionManager;
/**
* JMS and messaging configuration for the T-Engines. Contains the basic config in order to have the
* T-Engine able to read from queues and send a reply back.
*
* @author Lucian Tuca
* created on 18/12/2018
*/
@Configuration
@ConditionalOnProperty(name = "activemq.url")
public class MessagingConfig implements JmsListenerConfigurer
{
private static final Logger logger = LoggerFactory.getLogger(MessagingConfig.class);
@Override
public void configureJmsListeners(@NonNull JmsListenerEndpointRegistrar registrar)
{
registrar.setMessageHandlerMethodFactory(methodFactory());
}
@Bean
@ConditionalOnProperty(name = "activemq.url")
public DefaultMessageHandlerMethodFactory methodFactory()
{
DefaultMessageHandlerMethodFactory factory = new DefaultMessageHandlerMethodFactory();
factory.setValidator(new TransformRequestValidator());
return factory;
}
@Bean
@ConditionalOnProperty(name = "activemq.url")
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory(
final ConnectionFactory connectionFactory,
final TransformMessageConverter transformMessageConverter)
{
final DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory);
factory.setMessageConverter(transformMessageConverter);
factory.setErrorHandler(t -> logger.error("JMS error: " + t.getMessage(), t));
factory.setTransactionManager(transactionManager(connectionFactory));
return factory;
}
@Bean
@ConditionalOnProperty(name = "activemq.url")
public PlatformTransactionManager transactionManager(final ConnectionFactory connectionFactory)
{
final JmsTransactionManager transactionManager = new JmsTransactionManager();
transactionManager.setConnectionFactory(connectionFactory);
return transactionManager;
}
@Bean
@ConditionalOnProperty(name = "activemq.url")
public Queue engineRequestQueue(
@Value("${queue.engineRequestQueue}") String engineRequestQueueValue)
{
return new ActiveMQQueue(engineRequestQueueValue);
}
}

67
t-engine-base/src/main/java/org/alfresco/transform/base/messaging/MessagingInfo.java Normal file
View File

@@ -0,0 +1,67 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.base.messaging;
import javax.annotation.PostConstruct;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Configuration;
/**
* Prints JMS status information at application startup.
*
* @author Cezar Leahu
*/
@Configuration
public class MessagingInfo
{
private static final Logger logger = LoggerFactory.getLogger(MessagingInfo.class);
@Value("${activemq.url:}")
private String activemqUrl;
@PostConstruct
public void init()
{
// For backwards-compatibility, we continue to rely on setting ACTIVEMQ_URL environment variable (see application.yaml)
// The MessagingConfig class uses on ConditionalOnProperty (ie. activemq.url is set and not false)
// Note: as per application.yaml the broker url is appended with "?jms.watchTopicAdvisories=false". If this needs to be fully
// overridden then it would require explicitly setting both "spring.activemq.broker-url" *and* "activemq.url" (latter to non-false value).
if ((activemqUrl != null) && (! activemqUrl.equals("false")))
{
logger.info("JMS client is ENABLED - ACTIVEMQ_URL ='{}'", activemqUrl);
}
else
{
logger.info("JMS client is DISABLED - ACTIVEMQ_URL is not set");
}
}
}

102
t-engine-base/src/main/java/org/alfresco/transform/base/messaging/TransformMessageConverter.java Normal file
View File

@@ -0,0 +1,102 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.base.messaging;
import javax.jms.JMSException;
import javax.jms.Message;
import javax.jms.Session;
import org.alfresco.transform.client.model.TransformReply;
import org.alfresco.transform.client.model.TransformRequest;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.jms.support.converter.MappingJackson2MessageConverter;
import org.springframework.jms.support.converter.MessageConversionException;
import org.springframework.jms.support.converter.MessageConverter;
import org.springframework.jms.support.converter.MessageType;
import org.springframework.lang.NonNull;
import org.springframework.stereotype.Service;
import com.fasterxml.jackson.databind.JavaType;
import com.fasterxml.jackson.databind.type.TypeFactory;
import com.google.common.collect.ImmutableMap;
/**
* TODO: Duplicated from the Router
* Custom wrapper over MappingJackson2MessageConverter for T-Request/T-Reply objects.
*
* @author Cezar Leahu
*/
@Service
public class TransformMessageConverter implements MessageConverter
{
private static final Logger logger = LoggerFactory.getLogger(TransformMessageConverter.class);
private static final MappingJackson2MessageConverter converter;
private static final JavaType TRANSFORM_REQUEST_TYPE =
TypeFactory.defaultInstance().constructType(TransformRequest.class);
static
{
converter = new MappingJackson2MessageConverter()
{
@Override
@NonNull
protected JavaType getJavaTypeForMessage(final Message message) throws JMSException
{
if (message.getStringProperty("_type") == null)
{
return TRANSFORM_REQUEST_TYPE;
}
return super.getJavaTypeForMessage(message);
}
};
converter.setTargetType(MessageType.BYTES);
converter.setTypeIdPropertyName("_type");
converter.setTypeIdMappings(ImmutableMap.of(
TransformRequest.class.getName(), TransformRequest.class,
TransformReply.class.getName(), TransformReply.class)
);
}
@Override
@NonNull
public Message toMessage(
@NonNull final Object object,
@NonNull final Session session) throws JMSException, MessageConversionException
{
return converter.toMessage(object, session);
}
@Override
@NonNull
public Object fromMessage(@NonNull final Message message) throws JMSException
{
return converter.fromMessage(message);
}
}

77
t-engine-base/src/main/java/org/alfresco/transform/base/messaging/TransformReplySender.java Normal file
View File

@@ -0,0 +1,77 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.base.messaging;
import javax.jms.Destination;
import org.alfresco.transform.client.model.TransformReply;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.jms.core.JmsTemplate;
import org.springframework.stereotype.Component;
/**
* TODO: Duplicated from the Router
* TransformReplySender Bean
* <p/>
* JMS message sender/publisher
*
* @author Cezar Leahu
*/
@Component
public class TransformReplySender
{
private static final Logger logger = LoggerFactory.getLogger(TransformReplySender.class);
@Autowired
private JmsTemplate jmsTemplate;
public void send(final Destination destination, final TransformReply reply)
{
send(destination, reply, reply.getRequestId());
}
public void send(final Destination destination, final TransformReply reply,
final String correlationId)
{
try
{
//jmsTemplate.setSessionTransacted(true); // do we need this?
jmsTemplate.convertAndSend(destination, reply, m -> {
m.setJMSCorrelationID(correlationId);
return m;
});
logger.trace("Sent: {} - with correlation ID {}", reply, correlationId);
}
catch (Exception e)
{
logger.error(
"Failed to send T-Reply " + reply + " - for correlation ID " + correlationId, e);
}
}
}

599
t-engine-base/src/main/java/org/alfresco/transform/base/metadataExtractors/AbstractMetadataExtractor.java Normal file
View File

@@ -0,0 +1,599 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005-2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.base.metadataExtractors;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.alfresco.transform.common.TransformException;
import org.slf4j.Logger;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.Serializable;
import java.lang.reflect.Array;
import java.util.Arrays;
import java.util.Collection;
import java.util.Collections;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Properties;
import java.util.Set;
import java.util.StringTokenizer;
import java.util.TreeMap;
/**
* Helper methods for metadata extract and embed.
* <p>
* <i>Much of the code is based on AbstractMappingMetadataExtracter from the
* content repository. The code has been simplified to only set up mapping one way.</i>
* <p>
* If a transform specifies that it can convert from {@code "<MIMETYPE>"} to {@code "alfresco-metadata-extract"}
* (specified in the {@code engine_config.json}), it is indicating that it can extract metadata from {@code <MIMETYPE>}.
*
* The transform results in a Map of extracted properties encoded as json being returned to the content repository.
* <ul>
* <li>The content repository will use a transform in preference to any metadata extractors it might have defined
* locally for the same MIMETYPE.</li>
* <li>The T-Engine's Controller class will call a method in a class that extends {@link AbstractMetadataExtractor}
* based on the source and target mediatypes in the normal way.</li>
* <li>The method extracts ALL available metadata is extracted from the document and then calls
* {@link #mapMetadataAndWrite(File, Map, Map)}.</li>
* <li>Selected values from the available metadata are mapped into content repository property names and values,
* depending on what is defined in a {@code "<classname>_metadata_extract.properties"} file.</li>
* <li>The selected values are set back to the content repository as a JSON representation of a Map, where the values
* are applied to the source node.</li>
* </ul>
* To support the same functionality as metadata extractors configured inside the content repository,
* extra key value pairs may be returned from {@link #extractMetadata}. These are:
* <ul>
* <li>{@code "sys:overwritePolicy"} which can specify the
* {@code org.alfresco.repo.content.metadata.MetadataExtracter.OverwritePolicy} name. Defaults to "PRAGMATIC".</li>
* <li>{@code "sys:enableStringTagging"} if {@code "true"} finds or creates tags for each string mapped to
* {@code cm:taggable}. Defaults to {@code "false"} to ignore mapping strings to tags.</li>
* <li>{@code "sys:carryAspectProperties"} </li>
* <li>{@code "sys:stringTaggingSeparators"} </li>
* </ul>
*
* If a transform specifies that it can convert from {@code "<MIMETYPE>"} to {@code "alfresco-metadata-embed"}, it is
* indicating that it can embed metadata in {@code <MIMETYPE>}.
*
* The transform results in a new version of supplied source file that contains the metadata supplied in the transform
* options.
*
* @author Jesper Steen Møller
* @author Derek Hulley
* @author adavis
*/
public abstract class AbstractMetadataExtractor
{
private static final String EXTRACT = "extract";
private static final String EMBED = "embed";
private static final String METADATA = "metadata";
private static final String EXTRACT_MAPPING = "extractMapping";
private static final String NAMESPACE_PROPERTY_PREFIX = "namespace.prefix.";
private static final char NAMESPACE_PREFIX = ':';
private static final char NAMESPACE_BEGIN = '{';
private static final char NAMESPACE_END = '}';
private static final List<String> SYS_PROPERTIES = Arrays.asList(
"sys:overwritePolicy",
"sys:enableStringTagging",
"sys:carryAspectProperties",
"sys:stringTaggingSeparators");
private static final ObjectMapper jsonObjectMapper = new ObjectMapper();
protected final Logger logger;
private Map<String, Set<String>> defaultExtractMapping;
private ThreadLocal<Map<String, Set<String>>> extractMapping = new ThreadLocal<>();
private Map<String, Set<String>> embedMapping;
public AbstractMetadataExtractor(Logger logger)
{
this.logger = logger;
defaultExtractMapping = Collections.emptyMap();
embedMapping = Collections.emptyMap();
try
{
defaultExtractMapping = buildExtractMapping();
embedMapping = buildEmbedMapping();
}
catch (Exception e)
{
logger.error("Failed to read config", e);
}
}
public abstract Map<String, Serializable> extractMetadata(String sourceMimetype, Map<String, String> transformOptions,
File sourceFile) throws Exception;
public void embedMetadata(String sourceMimetype, String targetMimetype, Map<String, String> transformOptions,
File sourceFile, File targetFile) throws Exception
{
// Default nothing, as embedding is not supported in most cases
}
protected Map<String, Serializable> getMetadata(Map<String, String> transformOptions)
{
String metadataAsJson = transformOptions.get(METADATA);
if (metadataAsJson == null)
{
throw new IllegalArgumentException("No metadata in embed request");
}
try
{
TypeReference<HashMap<String, Serializable>> typeRef = new TypeReference<>() {};
HashMap<String, Serializable> systemProperties = jsonObjectMapper.readValue(metadataAsJson, typeRef);
Map<String, Serializable> rawProperties = mapSystemToRaw(systemProperties);
return rawProperties;
}
catch (JsonProcessingException e)
{
throw new IllegalArgumentException("Failed to read metadata from request", e);
}
}
private Map<String, Serializable> mapSystemToRaw(Map<String, Serializable> systemMetadata)
{
Map<String, Serializable> metadataProperties = new HashMap<>(systemMetadata.size() * 2 + 1);
for (Map.Entry<String, Serializable> entry : systemMetadata.entrySet())
{
String modelProperty = entry.getKey();
// Check if there is a mapping for this
if (!embedMapping.containsKey(modelProperty))
{
// No mapping - ignore
continue;
}
Serializable documentValue = entry.getValue();
Set<String> metadataKeys = embedMapping.get(modelProperty);
for (String metadataKey : metadataKeys)
{
metadataProperties.put(metadataKey, documentValue);
}
}
// Done
if (logger.isDebugEnabled())
{
logger.debug(
"Converted system model values to metadata values: \n" +
" System Properties: " + systemMetadata + "\n" +
" Metadata Properties: " + metadataProperties);
}
return metadataProperties;
}
protected Map<String, Set<String>> getExtractMapping()
{
return Collections.unmodifiableMap(extractMapping.get());
}
public Map<String, Set<String>> getEmbedMapping()
{
return Collections.unmodifiableMap(embedMapping);
}
/**
* Based on AbstractMappingMetadataExtracter#getDefaultMapping.
*
* This method provides a <i>mapping</i> of where to store the values extracted from the documents. The list of
* properties need <b>not</b> include all metadata values extracted from the document. This mapping should be
* defined in a file based on the class name: {@code "<classname>_metadata_extract.properties"}
* @return Returns a static mapping. It may not be null.
*/
private Map<String, Set<String>> buildExtractMapping()
{
String filename = getPropertiesFilename(EXTRACT);
Properties properties = readProperties(filename);
if (properties == null)
{
logger.error("Failed to read "+filename);
}
Map<String, String> namespacesByPrefix = getNamespaces(properties);
return buildExtractMapping(properties, namespacesByPrefix);
}
private Map<String, Set<String>> buildExtractMapping(Properties properties, Map<String, String> namespacesByPrefix)
{
// Create the mapping
Map<String, Set<String>> convertedMapping = new HashMap<>(17);
for (Map.Entry<Object, Object> entry : properties.entrySet())
{
String documentProperty = (String) entry.getKey();
String qnamesStr = (String) entry.getValue();
if (documentProperty.startsWith(NAMESPACE_PROPERTY_PREFIX))
{
continue;
}
// Create the entry
Set<String> qnames = new HashSet<>(3);
convertedMapping.put(documentProperty, qnames);
// The to value can be a list of QNames
StringTokenizer tokenizer = new StringTokenizer(qnamesStr, ",");
while (tokenizer.hasMoreTokens())
{
String qnameStr = tokenizer.nextToken().trim();
qnameStr = getQNameString(namespacesByPrefix, entry, qnameStr, EXTRACT);
qnames.add(qnameStr);
}
if (logger.isTraceEnabled())
{
logger.trace("Added mapping from " + documentProperty + " to " + qnames);
}
}
return convertedMapping;
}
/**
* Based on AbstractMappingMetadataExtracter#getDefaultEmbedMapping.
*
* This method provides a <i>mapping</i> of model properties that should be embedded in the content. The list of
* properties need <b>not</b> include all properties. This mapping should be defined in a file based on the class
* name: {@code "<classname>_metadata_embed.properties"}
* <p>
* If no {@code "<classname>_metadata_embed.properties"} file is found, a reverse of the
* {@code "<classname>_metadata_extract.properties"} will be assumed. A last win approach will be used for handling
* duplicates.
* @return Returns a static mapping. It may not be null.
*/
private Map<String, Set<String>> buildEmbedMapping()
{
String filename = getPropertiesFilename(EMBED);
Properties properties = readProperties(filename);
Map<String, Set<String>> embedMapping;
if (properties != null)
{
Map<String, String> namespacesByPrefix = getNamespaces(properties);
embedMapping = buildEmbedMapping(properties, namespacesByPrefix);
}
else
{
if (logger.isDebugEnabled())
{
logger.debug("No " + filename + ", assuming reverse of extract mapping");
}
embedMapping = buildEmbedMappingByReversingExtract();
}
return embedMapping;
}
private Map<String, Set<String>> buildEmbedMapping(Properties properties, Map<String, String> namespacesByPrefix)
{
Map<String, Set<String>> convertedMapping = new HashMap<>(17);
for (Map.Entry<Object, Object> entry : properties.entrySet())
{
String modelProperty = (String) entry.getKey();
String metadataKeysString = (String) entry.getValue();
if (modelProperty.startsWith(NAMESPACE_PROPERTY_PREFIX))
{
continue;
}
modelProperty = getQNameString(namespacesByPrefix, entry, modelProperty, EMBED);
String[] metadataKeysArray = metadataKeysString.split(",");
Set<String> metadataKeys = new HashSet<String>(metadataKeysArray.length);
for (String metadataKey : metadataKeysArray) {
metadataKeys.add(metadataKey.trim());
}
// Create the entry
convertedMapping.put(modelProperty, metadataKeys);
if (logger.isTraceEnabled())
{
logger.trace("Added mapping from " + modelProperty + " to " + metadataKeysString);
}
}
return convertedMapping;
}
private Map<String, Set<String>> buildEmbedMappingByReversingExtract()
{
Map<String, Set<String>> extractMapping = buildExtractMapping();
Map<String, Set<String>> embedMapping;
embedMapping = new HashMap<>(extractMapping.size());
for (String metadataKey : extractMapping.keySet())
{
if (extractMapping.get(metadataKey) != null && extractMapping.get(metadataKey).size() > 0)
{
String modelProperty = extractMapping.get(metadataKey).iterator().next();
Set<String> metadataKeys = embedMapping.get(modelProperty);
if (metadataKeys == null)
{
metadataKeys = new HashSet<String>(1);
embedMapping.put(modelProperty, metadataKeys);
}
metadataKeys.add(metadataKey);
if (logger.isTraceEnabled())
{
logger.trace("Added mapping from " + modelProperty + " to " + metadataKeys.toString());
}
}
}
return embedMapping;
}
private String getPropertiesFilename(String suffix)
{
String className = this.getClass().getName();
String shortClassName = className.split("\\.")[className.split("\\.").length - 1];
shortClassName = shortClassName.replace('$', '-');
return shortClassName + "_metadata_" + suffix + ".properties";
}
private Properties readProperties(String filename)
{
Properties properties = null;
try
{
InputStream inputStream = AbstractMetadataExtractor.class.getClassLoader().getResourceAsStream(filename);
if (inputStream != null)
{
properties = new Properties();
properties.load(inputStream);
}
}
catch (IOException ignore)
{
}
return properties;
}
private Map<String, String> getNamespaces(Properties properties)
{
Map<String, String> namespacesByPrefix = new HashMap<String, String>(5);
for (Map.Entry<Object, Object> entry : properties.entrySet())
{
String propertyName = (String) entry.getKey();
if (propertyName.startsWith(NAMESPACE_PROPERTY_PREFIX))
{
String prefix = propertyName.substring(17);
String namespace = (String) entry.getValue();
namespacesByPrefix.put(prefix, namespace);
}
}
return namespacesByPrefix;
}
private String getQNameString(Map<String, String> namespacesByPrefix, Map.Entry<Object, Object> entry, String qnameStr, String type)
{
// Check if we need to resolve a namespace reference
int index = qnameStr.indexOf(NAMESPACE_PREFIX);
if (index > -1 && qnameStr.charAt(0) != NAMESPACE_BEGIN)
{
String prefix = qnameStr.substring(0, index);
String suffix = qnameStr.substring(index + 1);
// It is prefixed
String uri = namespacesByPrefix.get(prefix);
if (uri == null)
{
throw new IllegalArgumentException("No prefix mapping for " + type + " property mapping: \n" +
" Extractor: " + this + "\n" +
" Mapping: " + entry);
}
qnameStr = NAMESPACE_BEGIN + uri + NAMESPACE_END + suffix;
}
return qnameStr;
}
/**
* Adds a value to the map, conserving null values. Values are converted to null if:
* <ul>
* <li>it is an empty string value after trimming</li>
* <li>it is an empty collection</li>
* <li>it is an empty array</li>
* </ul>
* String values are trimmed before being put into the map.
* Otherwise, it is up to the extracter to ensure that the value is a <tt>Serializable</tt>.
* It is not appropriate to implicitly convert values in order to make them <tt>Serializable</tt>
* - the best conversion method will depend on the value's specific meaning.
*
* @param key the destination key
* @param value the serializable value
* @param destination the map to put values into
* @return Returns <tt>true</tt> if set, otherwise <tt>false</tt>
*/
// Copied from the content repository's AbstractMappingMetadataExtracter.
protected boolean putRawValue(String key, Serializable value, Map<String, Serializable> destination)
{
if (value == null)
{
// Just keep this
}
else if (value instanceof String)
{
String valueStr = ((String) value).trim();
if (valueStr.length() == 0)
{
value = null;
}
else
{
if (valueStr.indexOf("\u0000") != -1)
{
valueStr = valueStr.replaceAll("\u0000", "");
}
// Keep the trimmed value
value = valueStr;
}
}
else if (value instanceof Collection)
{
Collection<?> valueCollection = (Collection<?>) value;
if (valueCollection.isEmpty())
{
value = null;
}
}
else if (value.getClass().isArray())
{
if (Array.getLength(value) == 0)
{
value = null;
}
}
// It passed all the tests
destination.put(key, value);
return true;
}
public void extractMetadata(String sourceMimetype, Map<String, String> transformOptions,
String sourceEncoding, InputStream inputStream,
String targetEncoding, OutputStream outputStream) throws Exception
{
// TODO
throw new TransformException(500, "TODO extractMetadata");
}
/**
* The {@code transformOptions} may contain a replacement set of mappings. These will be used in place of the
* default mappings from read from file if supplied.
*/
public void extractMetadata(String sourceMimetype, Map<String, String> transformOptions, File sourceFile,
File targetFile) throws Exception
{
Map<String, Set<String>> mapping = getExtractMappingFromOptions(transformOptions, defaultExtractMapping);
// Use a ThreadLocal to avoid changing method signatures of methods that currently call getExtractMapping.
try
{
extractMapping.set(mapping);
Map<String, Serializable> metadata = extractMetadata(sourceMimetype, transformOptions, sourceFile);
mapMetadataAndWrite(targetFile, metadata, mapping);
}
finally
{
extractMapping.set(null);
}
}
private Map<String, Set<String>> getExtractMappingFromOptions(Map<String, String> transformOptions, Map<String,
Set<String>> defaultExtractMapping)
{
String extractMappingOption = transformOptions.get(EXTRACT_MAPPING);
if (extractMappingOption != null)
{
try
{
TypeReference<HashMap<String, Set<String>>> typeRef = new TypeReference<>() {};
return jsonObjectMapper.readValue(extractMappingOption, typeRef);
}
catch (JsonProcessingException e)
{
throw new IllegalArgumentException("Failed to read "+ EXTRACT_MAPPING +" from request", e);
}
}
return defaultExtractMapping;
}
/**
* @deprecated use {@link #extractMetadata(String, Map, File, File)} rather than calling this method.
* By default call the overloaded method with the default {@code extractMapping}.
*/
@Deprecated
public void mapMetadataAndWrite(File targetFile, Map<String, Serializable> metadata) throws IOException
{
mapMetadataAndWrite(targetFile, metadata, defaultExtractMapping);
}
public void mapMetadataAndWrite(File targetFile, Map<String, Serializable> metadata,
Map<String, Set<String>> extractMapping) throws IOException
{
if (logger.isDebugEnabled())
{
logger.debug("Raw metadata:");
metadata.forEach((k,v) -> logger.debug(" "+k+"="+v));
}
metadata = mapRawToSystem(metadata, extractMapping);
writeMetadata(targetFile, metadata);
}
/**
* Based on AbstractMappingMetadataExtracter#mapRawToSystem.
*
* @param rawMetadata Metadata keyed by document properties
* @param extractMapping Mapping between document ans system properties
* @return Returns the metadata keyed by the system properties
*/
private Map<String, Serializable> mapRawToSystem(Map<String, Serializable> rawMetadata,
Map<String, Set<String>> extractMapping)
{
boolean debugEnabled = logger.isDebugEnabled();
if (debugEnabled)
{
logger.debug("Returned metadata:");
}
Map<String, Serializable> systemProperties = new HashMap<String, Serializable>(rawMetadata.size() * 2 + 1);
for (Map.Entry<String, Serializable> entry : rawMetadata.entrySet())
{
String documentKey = entry.getKey();
Serializable documentValue = entry.getValue();
if (SYS_PROPERTIES.contains(documentKey))
{
systemProperties.put(documentKey, documentValue);
if (debugEnabled)
{
logger.debug(" " + documentKey + "=" + documentValue);
}
continue;
}
// Check if there is a mapping for this
if (!extractMapping.containsKey(documentKey))
{
// No mapping - ignore
continue;
}
Set<String> systemQNames = extractMapping.get(documentKey);
for (String systemQName : systemQNames)
{
if (debugEnabled)
{
logger.debug(" "+systemQName+"="+documentValue+" ("+documentKey+")");
}
systemProperties.put(systemQName, documentValue);
}
}
return new TreeMap<String, Serializable>(systemProperties);
}
private void writeMetadata(File targetFile, Map<String, Serializable> results)
throws IOException
{
jsonObjectMapper.writeValue(targetFile, results);
}
}

77
t-engine-base/src/main/java/org/alfresco/transform/base/model/FileRefEntity.java Normal file
View File

@@ -0,0 +1,77 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.base.model;
import java.util.Objects;
/**
* TODO: Copied from org.alfresco.store.entity (alfresco-shared-file-store). To be discussed
*
* POJO that represents content reference ({@link java.util.UUID})
*/
public class FileRefEntity
{
private String fileRef;
public FileRefEntity() {}
public FileRefEntity(String fileRef)
{
this.fileRef = fileRef;
}
public void setFileRef(String fileRef)
{
this.fileRef = fileRef;
}
public String getFileRef()
{
return fileRef;
}
@Override
public boolean equals(Object o)
{
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
FileRefEntity that = (FileRefEntity) o;
return Objects.equals(fileRef, that.fileRef);
}
@Override
public int hashCode()
{
return Objects.hash(fileRef);
}
@Override
public String toString()
{
return fileRef;
}
}

54
t-engine-base/src/main/java/org/alfresco/transform/base/model/FileRefResponse.java Normal file
View File

@@ -0,0 +1,54 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.base.model;
/**
* TODO: Copied from org.alfresco.store.entity (alfresco-shared-file-store). To be discussed
*
* POJO that describes the ContentRefEntry response, contains {@link FileRefEntity} according to API spec
*/
public class FileRefResponse
{
private FileRefEntity entry;
public FileRefResponse() {}
public FileRefResponse(FileRefEntity entry)
{
this.entry = entry;
}
public FileRefEntity getEntry()
{
return entry;
}
public void setEntry(FileRefEntity entry)
{
this.entry = entry;
}
}

391
t-engine-base/src/main/java/org/alfresco/transform/base/probes/ProbeTestTransform.java Normal file
View File

@@ -0,0 +1,391 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.base.probes;
import static org.alfresco.transform.base.fs.FileManager.SOURCE_FILE;
import static org.alfresco.transform.base.fs.FileManager.TARGET_FILE;
import static org.alfresco.transform.base.fs.FileManager.TempFileProvider.createTempFile;
import static org.springframework.http.HttpStatus.BAD_REQUEST;
import static org.springframework.http.HttpStatus.INSUFFICIENT_STORAGE;
import static org.springframework.http.HttpStatus.INTERNAL_SERVER_ERROR;
import static org.springframework.http.HttpStatus.OK;
import static org.springframework.http.HttpStatus.TOO_MANY_REQUESTS;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.StandardCopyOption;
import java.util.Map;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicLong;
import javax.servlet.http.HttpServletRequest;
import org.alfresco.transform.base.TransformController;
import org.alfresco.transform.common.TransformException;
import org.alfresco.transform.registry.TransformServiceRegistry;
import org.alfresco.transform.base.logging.LogEntry;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
/**
* Provides test transformations and the logic used by k8 liveness and readiness probes.
*
* <p><b>K8s probes</b>: A readiness probe indicates if the pod should accept request. <b>It does not indicate that a pod is
* ready after startup</b>. The liveness probe indicates when to kill the pod. <b>Both probes are called throughout the
* lifetime of the pod</b> and a <b>liveness probes can take place before a readiness probe.</b> The k8s
* <b>initialDelaySeconds field is not fully honoured</b> as it is multiplied by a random number, so is
* actually a maximum initial delay in seconds, but could be 0. </p>
*
* <p>Live and readiness probes do test transforms. The first 6 requests result in a transformation of a small test
* file. The average time and size is remembered, but excludes the first one as it is normally slower. This is
* used in future requests to discover if transformations are becoming slower or unexpectedly change size.</p>
*
* <p>If a transform longer than a maximum time, a maximum number of transforms have been performed, a test transform is
* an unexpected size or a test transform takes an unexpected time, then a non 200 status code is returned resulting in
* k8s terminating the pod. These are controlled by:</p>
* <ul>
* <li>expectedLength the expected length of the target file after a test transform</li>
* <li>plusOrMinus allows for variation in the transformed size - generally caused by dates</li>
* <li>livenessPercent allows for variation in transform time. Up to 2 and a half times is not
* unreasonable under load</li>
* <li>maxTransforms the maximum number of transforms (not just test ones) before a restart is triggered</li>
* <li>maxTransformSeconds a maximum time any transform (not just test ones) is allowed to take before
* a restart is triggered.</li>
* <li>livenessTransformPeriodSeconds The number of seconds between test transforms done for live probes</li>
* </ul>
*/
public class ProbeTestTransform
{
private final Logger logger = LoggerFactory.getLogger(ProbeTestTransform.class);
@Autowired
private TransformServiceRegistry transformRegistry;
private static final int AVERAGE_OVER_TRANSFORMS = 5;
private final String sourceFilename;
private final String targetFilename;
private final String sourceMimetype;
private final String targetMimetype;
private final Map<String, String> transformOptions;
private final long minExpectedLength;
private final long maxExpectedLength;
private int livenessPercent;
private long probeCount;
private int transCount;
private long normalTime;
private long maxTime = Long.MAX_VALUE;
private long nextTransformTime;
private final boolean livenessTransformEnabled;
private final long livenessTransformPeriod;
private final long maxTransformCount;
private long maxTransformTime;
private final AtomicBoolean initialised = new AtomicBoolean(false);
private final AtomicBoolean readySent = new AtomicBoolean(false);
private final AtomicLong transformCount = new AtomicLong(0);
private final AtomicBoolean die = new AtomicBoolean(false);
public int getLivenessPercent()
{
return livenessPercent;
}
public long getMaxTime()
{
return maxTime;
}
public ProbeTestTransform(String sourceFilename, String targetFilename,
String sourceMimetype, String targetMimetype, Map<String, String> transformOptions,
long expectedLength, long plusOrMinus, int livenessPercent, long maxTransforms, long maxTransformSeconds,
long livenessTransformPeriodSeconds)
{
this.sourceFilename = sourceFilename;
this.targetFilename = targetFilename;
this.sourceMimetype = sourceMimetype;
this.targetMimetype = targetMimetype;
this.transformOptions = transformOptions;
minExpectedLength = Math.max(0, expectedLength - plusOrMinus);
maxExpectedLength = expectedLength + plusOrMinus;
this.livenessPercent = (int) getPositiveLongEnv("livenessPercent", livenessPercent);
maxTransformCount = getPositiveLongEnv("maxTransforms", maxTransforms);
maxTransformTime = getPositiveLongEnv("maxTransformSeconds", maxTransformSeconds) * 1000;
livenessTransformPeriod = getPositiveLongEnv("livenessTransformPeriodSeconds",
livenessTransformPeriodSeconds) * 1000;
livenessTransformEnabled = getBooleanEnvVar("livenessTransformEnabled", false);
}
private boolean getBooleanEnvVar(final String name, final boolean defaultValue)
{
try
{
return Boolean.parseBoolean(System.getenv(name));
}
catch (Exception ignore)
{
}
return defaultValue;
}
private long getPositiveLongEnv(String name, long defaultValue)
{
long l = -1;
String env = System.getenv(name);
if (env != null)
{
try
{
l = Long.parseLong(env);
}
catch (NumberFormatException ignore)
{
}
}
if (l <= 0)
{
l = defaultValue;
}
logger.trace("Probe: {}={}", name, l);
return l;
}
// We don't want to be doing test transforms every few seconds, but do want frequent live probes.
public String doTransformOrNothing(HttpServletRequest request, boolean isLiveProbe, TransformController controller)
{
// If not initialised OR it is a live probe and we are scheduled to to do a test transform.
probeCount++;
// TODO: update/fix/refactor liveness probes as part of ATS-138
if (isLiveProbe && !livenessTransformEnabled)
{
return doNothing(true);
}
return (isLiveProbe && livenessTransformPeriod > 0 &&
(transCount <= AVERAGE_OVER_TRANSFORMS || nextTransformTime < System.currentTimeMillis()))
|| !initialised.get()
? doTransform(request, isLiveProbe, controller)
: doNothing(isLiveProbe);
}
private String doNothing(boolean isLiveProbe)
{
String probeMessage = getProbeMessage(isLiveProbe);
String message = "Success - No transform.";
LogEntry.setStatusCodeAndMessage(OK.value(), probeMessage + message);
if (!isLiveProbe && !readySent.getAndSet(true))
{
logger.trace("{}{}", probeMessage, message);
}
return message;
}
private String doTransform(HttpServletRequest request, boolean isLiveProbe, TransformController controller)
{
checkMaxTransformTimeAndCount(isLiveProbe);
long start = System.currentTimeMillis();
if (nextTransformTime != 0)
{
do
{
nextTransformTime += livenessTransformPeriod;
}
while (nextTransformTime < start);
}
File sourceFile = getSourceFile(request, isLiveProbe);
File targetFile = getTargetFile(request);
String transformName = getTransformerName(sourceFile, sourceMimetype, targetMimetype, transformOptions);
controller.transformImpl(transformName, sourceMimetype, targetMimetype, transformOptions, sourceFile, targetFile);
long time = System.currentTimeMillis() - start;
String message = "Transform " + time + "ms";
checkTargetFile(targetFile, isLiveProbe, message);
recordTransformTime(time);
calculateMaxTime(time, isLiveProbe);
if (time > maxTime)
{
throw new TransformException(INTERNAL_SERVER_ERROR.value(),
getMessagePrefix(isLiveProbe) +
message + " which is more than " + livenessPercent +
"% slower than the normal value of " + normalTime + "ms");
}
// We don't care if the ready or live probe works out if we are 'ready' to take requests.
initialised.set(true);
checkMaxTransformTimeAndCount(isLiveProbe);
return getProbeMessage(isLiveProbe) + message;
}
private String getTransformerName(final File sourceFile, final String sourceMimetype,
final String targetMimetype, final Map<String, String> transformOptions)
{
final long sourceSizeInBytes = sourceFile.length();
final String transformerName = transformRegistry.findTransformerName(sourceMimetype,
sourceSizeInBytes, targetMimetype, transformOptions, null);
if (transformerName == null)
{
throw new TransformException(BAD_REQUEST.value(), "No transforms were able to handle the request");
}
return transformerName;
}
private void checkMaxTransformTimeAndCount(boolean isLiveProbe)
{
if (die.get())
{
throw new TransformException(TOO_MANY_REQUESTS.value(),
getMessagePrefix(isLiveProbe) + "Transformer requested to die. A transform took " +
"longer than " + (maxTransformTime * 1000) + " seconds");
}
if (maxTransformCount > 0 && transformCount.get() > maxTransformCount)
{
throw new TransformException(TOO_MANY_REQUESTS.value(),
getMessagePrefix(isLiveProbe) + "Transformer requested to die. It has performed " +
"more than " + maxTransformCount + " transformations");
}
}
private File getSourceFile(HttpServletRequest request, boolean isLiveProbe)
{
incrementTransformerCount();
File sourceFile = createTempFile("source_", "_" + sourceFilename);
request.setAttribute(SOURCE_FILE, sourceFile);
try (InputStream inputStream = this.getClass().getResourceAsStream('/' + sourceFilename))
{
Files.copy(inputStream, sourceFile.toPath(), StandardCopyOption.REPLACE_EXISTING);
}
catch (IOException e)
{
throw new TransformException(INSUFFICIENT_STORAGE.value(),
getMessagePrefix(isLiveProbe) + "Failed to store the source file", e);
}
long length = sourceFile.length();
LogEntry.setSource(sourceFilename, length);
return sourceFile;
}
private File getTargetFile(HttpServletRequest request)
{
File targetFile = createTempFile("target_", "_" + targetFilename);
request.setAttribute(TARGET_FILE, targetFile);
LogEntry.setTarget(targetFilename);
return targetFile;
}
public void recordTransformTime(long time)
{
if (maxTransformTime > 0 && time > maxTransformTime)
{
die.set(true);
}
}
public void calculateMaxTime(long time, boolean isLiveProbe)
{
if (transCount <= AVERAGE_OVER_TRANSFORMS)
{
// Take the average of the first few transforms as the normal time. The initial transform might be slower
// so is ignored. Later ones are not included in case we have a gradual performance problem.
String message = getMessagePrefix(isLiveProbe) + "Success - Transform " + time + "ms";
if (++transCount > 1)
{
normalTime = (normalTime * (transCount - 2) + time) / (transCount - 1);
maxTime = (normalTime * (livenessPercent + 100)) / 100;
if ((!isLiveProbe && !readySent.getAndSet(
true)) || transCount > AVERAGE_OVER_TRANSFORMS)
{
nextTransformTime = System.currentTimeMillis() + livenessTransformPeriod;
logger.trace("{} - {}ms+{}%={}ms", message, normalTime, livenessPercent,
maxTime);
}
}
else if (!isLiveProbe && !readySent.getAndSet(true))
{
logger.trace(message);
}
}
}
private void checkTargetFile(File targetFile, boolean isLiveProbe, String message)
{
String probeMessage = getProbeMessage(isLiveProbe);
if (!targetFile.exists() || !targetFile.isFile())
{
throw new TransformException(INTERNAL_SERVER_ERROR.value(),
probeMessage + "Target File \"" + targetFile.getAbsolutePath() + "\" did not exist");
}
long length = targetFile.length();
if (length < minExpectedLength || length > maxExpectedLength)
{
throw new TransformException(INTERNAL_SERVER_ERROR.value(),
probeMessage + "Target File \"" + targetFile.getAbsolutePath() +
"\" was the wrong size (" + length + "). Needed to be between " +
minExpectedLength + " and " + maxExpectedLength);
}
LogEntry.setTargetSize(length);
LogEntry.setStatusCodeAndMessage(OK.value(), probeMessage + "Success - " + message);
}
private String getMessagePrefix(boolean isLiveProbe)
{
return Long.toString(probeCount) + ' ' + getProbeMessage(isLiveProbe);
}
private String getProbeMessage(boolean isLiveProbe)
{
return (isLiveProbe ? "Live Probe: " : "Ready Probe: ");
}
public void incrementTransformerCount()
{
transformCount.incrementAndGet();
}
public void setLivenessPercent(int livenessPercent)
{
this.livenessPercent = livenessPercent;
}
public long getNormalTime()
{
return normalTime;
}
}

41
t-engine-base/src/main/java/org/alfresco/transform/base/util/RequestParamMap.java Normal file
View File

@@ -0,0 +1,41 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.base.util;
/**
* Extends the list of transform options with historic request parameters or 'extra' parameters used in testing
* or communication in the all-in-one transformer.
*/
public interface RequestParamMap extends org.alfresco.transform.common.RequestParamMap
{
String TRANSFORM_NAME_PARAMETER = "alfresco.transform-name-parameter";
String FILE = "file";
String SOURCE_EXTENSION = "sourceExtension";
String SOURCE_MIMETYPE = "sourceMimetype";
String TARGET_MIMETYPE = "targetMimetype";
}

63
t-engine-base/src/main/java/org/alfresco/transform/base/util/Util.java Normal file
View File

@@ -0,0 +1,63 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.base.util;
public class Util
{
/**
* Safely converts a {@link String} to an {@link Integer}
*
* @param param String to be converted
* @return Null if param is null or converted value as {@link Integer}
*/
public static Integer stringToInteger(final String param)
{
return param == null ? null : Integer.parseInt(param);
}
/**
* Safely converts a {@link String} to a {@link Boolean}
*
* @param param String to be converted
* @return Null if param is null or converted value as {@link Boolean}
*/
public static Boolean stringToBoolean(final String param)
{
return param == null ? null : Boolean.parseBoolean(param);
}
/**
* Safely converts a {@link String} to a {@link Long}
*
* @param param String to be converted
* @return Null if param is null or converted value as {@link Boolean}
*/
public static Long stringToLong(final String param)
{
return param == null ? null : Long.parseLong(param);
}
}

61
t-engine-base/src/main/resources/application.yaml Normal file
View File

@@ -0,0 +1,61 @@
---
spring:
servlet:
multipart:
max-file-size: 8192MB
max-request-size: 8192MB
activemq:
broker-url: ${ACTIVEMQ_URL:nio://localhost:61616}?jms.watchTopicAdvisories=false
user: ${ACTIVEMQ_USER:admin}
password: ${ACTIVEMQ_PASSWORD:admin}
pool:
enabled: true
max-connections: 20
jackson:
default-property-inclusion: non_empty
activemq:
url: ${ACTIVEMQ_URL:false}
server:
port: ${SERVER_PORT:8090}
error:
include-message: ALWAYS
transform:
core:
version: @project.version@
logging:
level:
# org.alfresco.util.exec.RuntimeExec: debug
org.alfresco.transformer.LibreOfficeController: debug
org.alfresco.transformer.JodConverterSharedInstance: debug
org.alfresco.transformer.AlfrescoPdfRendererController: debug
org.alfresco.transformer.ImageMagickController: debug
org.alfresco.transformer.TikaController: debug
org.alfresco.transformer.MiscellaneousTransformersController: debug
org.alfresco.transform.common.TransformerDebug: debug
fileStoreUrl: ${FILE_STORE_URL:http://localhost:8099/alfresco/api/-default-/private/sfs/versions/1/file}
jms-listener:
concurrency: ${JMS_LISTENER_CONCURRENCY:1-10}
management:
endpoints:
web:
exposure:
include:
- metrics
- prometheus
- health
metrics:
enable[http]: false
enable[logback]: false
enable[tomcat]: false
enable[jvm.classes]: false
container:
name: ${HOSTNAME:t-engine}

1
t-engine-base/src/main/resources/engine_config.json Normal file
View File

@@ -0,0 +1 @@
{}

22
t-engine-base/src/main/resources/templates/error.html Normal file
View File

@@ -0,0 +1,22 @@
<html xmlns:th="http://www.thymeleaf.org">
<body>
<div>
<div th:if="${message} == null">
<h2>Error Page</h2>
</div>
<div th:if="${message}">
<h2 th:text="${#strings.substring(message,0,#strings.indexOf(message,' - '))} + ' Error Page'"></h2>
<h3 th:text="${#strings.substring(message,#strings.indexOf(message,' - ')+3)}"></h3>
</div>
<p th:text="${status} + ' - ' + ${error}"></p>
</div>
<div>
<br/>
<a href="/">Test Transformation</a>
<a href="/log">Log entries</a>
</div>
</body>
</html>

43
t-engine-base/src/main/resources/templates/log.html Normal file
View File

@@ -0,0 +1,43 @@
<html xmlns:th="http://www.thymeleaf.org">
<body>
<div>
<h2 th:text="${title}"></h2>
<div th:if="${log}">
<table>
<tr>
<th>Id</th>
<th>Time</th>
<th>Status Code</th>
<th>Duration (ms)</th>
<th>Source</th>
<th></th>
<th>Target</th>
<th></th>
<th>Options</th>
<th>Message</th>
</tr>
<tr th:each="entry : ${log}">
<td th:text="${entry.id}"></td>
<td th:text="${#dates.format(entry.date, 'HH:mm:ss')}"></td>
<td th:text="${entry.statusCode}"></td>
<td th:text="${entry.duration}"></td>
<td th:text="${entry.source}"></td>
<td th:text="${entry.sourceSize}"></td>
<td th:text="${entry.target}"></td>
<td th:text="${entry.targetSize}"></td>
<td th:text="${entry.options}"></td>
<td th:text="${entry.message}"></td>
</tr>
</table>
</div>
</div>
<div>
<br/>
<a href="/">Test Transformation</a>
<a href="/transform/config?configVersion=9999">Transformer Config</a>
</div>
</body>
</html>

177
t-engine-base/src/test/java/org/alfresco/transform/base/AbstractHttpRequestTest.java Normal file
View File

@@ -0,0 +1,177 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.base;
import static org.alfresco.transform.common.RequestParamMap.DIRECT_ACCESS_URL;
import static org.alfresco.transform.common.RequestParamMap.ENDPOINT_TRANSFORM;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.springframework.http.HttpMethod.POST;
import static org.springframework.http.MediaType.MULTIPART_FORM_DATA;
import static org.springframework.test.util.AssertionErrors.assertTrue;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.web.client.TestRestTemplate;
import org.springframework.boot.web.server.LocalServerPort;
import org.springframework.http.HttpEntity;
import org.springframework.http.HttpHeaders;
import org.springframework.http.ResponseEntity;
import org.springframework.util.LinkedMultiValueMap;
/**
* Super class for testing controllers with a server. Includes tests for the Controller itself.
* Note: Currently uses json rather than HTML as json is returned by this spring boot test harness.
*/
public abstract class AbstractHttpRequestTest
{
@LocalServerPort
private int port;
@Autowired
private TestRestTemplate restTemplate;
protected abstract String getTransformerName();
protected abstract String getSourceExtension();
@Test
public void testPageExists()
{
String result = restTemplate.getForObject("http://localhost:" + port + "/", String.class);
String title = getTransformerName() + ' ' + "Test Transformation";
assertTrue("\"" + title + "\" should be part of the page title", result.contains(title));
}
@Test
public void logPageExists()
{
String result = restTemplate.getForObject("http://localhost:" + port + "/log",
String.class);
String title = getTransformerName() + ' ' + "Log";
assertTrue("\"" + title + "\" should be part of the page title", result.contains(title));
}
@Test
public void errorPageExists()
{
String result = restTemplate.getForObject("http://localhost:" + port + "/error",
String.class);
String title = getTransformerName() + ' ' + "Error Page";
assertTrue("\"" + title + "\" should be part of the page title",
result.contains("Error Page"));
}
@Test
public void noFileError()
{
LinkedMultiValueMap<String, Object> parameters = new LinkedMultiValueMap<>();
parameters.add("targetExtension", ".tmp");
assertTransformError(false,
getTransformerName() + " - Required request part 'file' is not present",
parameters);
}
@Test
public void noTargetExtensionError()
{
assertMissingParameter("targetExtension");
}
private void assertMissingParameter(String name)
{
assertTransformError(true,
getTransformerName() + " - Request parameter '" + name + "' is missing", null);
}
protected void assertTransformError(boolean addFile,
String errorMessage,
LinkedMultiValueMap<String, Object> additionalParams)
{
LinkedMultiValueMap<String, Object> parameters = new LinkedMultiValueMap<>();
if (addFile)
{
parameters.add("file",
new org.springframework.core.io.ClassPathResource("quick." + getSourceExtension()));
}
if (additionalParams != null)
{
parameters.addAll(additionalParams);
}
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MULTIPART_FORM_DATA);
HttpEntity<LinkedMultiValueMap<String, Object>> entity = new HttpEntity<>(parameters,
headers);
sendTranformationRequest(entity, errorMessage);
}
@Test
public void httpTransformRequestDirectAccessUrlNotFoundTest()
{
String directUrl = "https://expired/direct/access/url";
LinkedMultiValueMap<String, Object> parameters = new LinkedMultiValueMap<>();
parameters.add("targetExtension", ".tmp");
parameters.add(DIRECT_ACCESS_URL, directUrl);
assertTransformError(false,
getTransformerName() + " - Direct Access Url not found.",
parameters);
}
protected void sendTranformationRequest(
final HttpEntity<LinkedMultiValueMap<String, Object>> entity, final String errorMessage)
{
final ResponseEntity<String> response = restTemplate.exchange(ENDPOINT_TRANSFORM, POST, entity,
String.class, "");
assertEquals(errorMessage, getErrorMessage(response.getBody()));
}
// Strip out just the error message from the returned json content body
// Had been expecting the Error page to be returned, but we end up with the json in this test harness.
// Is correct if run manually, so not worrying too much about this.
private String getErrorMessage(String content)
{
String message = "";
int i = content.indexOf("\"message\":\"");
if (i != -1)
{
int j = content.indexOf("\",\"path\":", i);
if (j != -1)
{
message = content.substring(i + 11, j);
}
}
return message;
}
}

134
t-engine-base/src/test/java/org/alfresco/transform/base/AbstractMetadataExtractsIT.java Normal file
View File

@@ -0,0 +1,134 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.base;
import static java.text.MessageFormat.format;
import static org.alfresco.transform.base.EngineClient.sendTRequest;
import static org.alfresco.transform.common.Mimetype.MIMETYPE_METADATA_EXTRACT;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.fail;
import static org.springframework.http.HttpStatus.OK;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.io.Serializable;
import java.util.HashMap;
import java.util.Map;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.springframework.core.io.Resource;
import org.springframework.http.ResponseEntity;
/**
* Super class of metadata integration tests. Sub classes should provide the following:
* <p>
* <ul>
* <li>A method providing a
* Stream of test files: {@code public static Stream<TestFileInfo> engineTransformations()}; </li>
* <li> Provide expected json files (&lt;sourceFilename>"_metadata.json") as resources on the classpath.</li>
* <li> Override the method {@code testTransformation(TestFileInfo testFileInfo)} such that it calls
* the super method as a {@code @ParameterizedTest} for example:</li> </ul>
* <pre>
* &#64;ParameterizedTest
*
* &#64;MethodSource("engineTransformations")
*
* &#64;Override
* public void testTransformation(TestFileInfo testFileInfo)
*
* {
* super.testTransformation(TestFileInfo testFileInfo)
* }
* </pre>
*
* @author adavis
* @author dedwards
*/
public abstract class AbstractMetadataExtractsIT
{
private static final String ENGINE_URL = "http://localhost:8090";
// These are normally variable, hence the lowercase.
private static final String targetMimetype = MIMETYPE_METADATA_EXTRACT;
private static final String targetExtension = "json";
private final ObjectMapper jsonObjectMapper = new ObjectMapper();
public void testTransformation(TestFileInfo testFileInfo)
{
final String sourceMimetype = testFileInfo.getMimeType();
final String sourceFile = testFileInfo.getPath();
final String descriptor = format("Transform ({0}, {1} -> {2}, {3})",
sourceFile, sourceMimetype, targetMimetype, targetExtension);
try
{
final ResponseEntity<Resource> response = sendTRequest(ENGINE_URL, sourceFile,
sourceMimetype, targetMimetype, targetExtension);
assertEquals(OK, response.getStatusCode(), descriptor);
String metadataFilename = sourceFile + "_metadata.json";
Map<String, Serializable> actualMetadata = readMetadata(response.getBody().getInputStream());
File actualMetadataFile = new File(metadataFilename);
jsonObjectMapper.writerWithDefaultPrettyPrinter().writeValue(actualMetadataFile, actualMetadata);
Map<String, Serializable> expectedMetadata = readExpectedMetadata(metadataFilename, actualMetadataFile);
assertEquals(expectedMetadata, actualMetadata,
sourceFile+": The metadata did not match the expected value. It has been saved in "+actualMetadataFile.getAbsolutePath());
actualMetadataFile.delete();
}
catch (Exception e)
{
e.printStackTrace();
fail(descriptor + " exception: " + e.getMessage());
}
}
private Map<String, Serializable> readExpectedMetadata(String filename, File actualMetadataFile) throws IOException
{
try (InputStream inputStream = this.getClass().getClassLoader().getResourceAsStream(filename))
{
if (inputStream == null)
{
fail("The expected metadata file "+filename+" did not exist.\n"+
"The actual metadata has been saved in "+actualMetadataFile.getAbsoluteFile());
}
return readMetadata(inputStream);
}
}
private Map<String, Serializable> readMetadata(InputStream inputStream) throws IOException
{
TypeReference<HashMap<String, Serializable>> typeRef = new TypeReference<HashMap<String, Serializable>>() {};
return jsonObjectMapper.readValue(inputStream, typeRef);
}
}

74
t-engine-base/src/test/java/org/alfresco/transform/base/AbstractQueueTransformServiceIT.java Normal file
View File

@@ -0,0 +1,74 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2021 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.base;
import static org.junit.jupiter.api.Assertions.assertEquals;
import javax.jms.Queue;
import org.alfresco.transform.client.model.TransformReply;
import org.alfresco.transform.client.model.TransformRequest;
import org.apache.activemq.command.ActiveMQQueue;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.jms.core.JmsTemplate;
/**
* @author Lucian Tuca
* created on 15/01/2019
*/
@SpringBootTest(properties = {"activemq.url=nio://localhost:61616"})
public abstract class AbstractQueueTransformServiceIT
{
@Autowired
private Queue engineRequestQueue;
@Autowired
private JmsTemplate jmsTemplate;
private final ActiveMQQueue testingQueue = new ActiveMQQueue(
"org.alfresco.transform.engine.IT");
@Test
public void queueTransformServiceIT()
{
TransformRequest request = buildRequest();
jmsTemplate.convertAndSend(engineRequestQueue, request, m -> {
m.setJMSCorrelationID(request.getRequestId());
m.setJMSReplyTo(testingQueue);
return m;
});
this.jmsTemplate.setReceiveTimeout(1_000);
TransformReply reply = (TransformReply) this.jmsTemplate.receiveAndConvert(testingQueue);
assertEquals(request.getRequestId(), reply.getRequestId());
}
protected abstract TransformRequest buildRequest();
}

686
t-engine-base/src/test/java/org/alfresco/transform/base/AbstractTransformerControllerTest.java Normal file
View File

@@ -0,0 +1,686 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.base;
import static java.nio.file.StandardCopyOption.REPLACE_EXISTING;
import static org.alfresco.transform.common.RequestParamMap.DIRECT_ACCESS_URL;
import static org.alfresco.transform.common.RequestParamMap.ENDPOINT_TRANSFORM;
import static org.alfresco.transform.common.RequestParamMap.ENDPOINT_TRANSFORM_CONFIG_LATEST;
import static org.alfresco.transform.common.RequestParamMap.ENDPOINT_TRANSFORM_CONFIG;
import static org.hamcrest.Matchers.containsString;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.when;
import static org.springframework.http.HttpHeaders.ACCEPT;
import static org.springframework.http.HttpHeaders.CONTENT_TYPE;
import static org.springframework.http.HttpStatus.BAD_REQUEST;
import static org.springframework.http.HttpStatus.CREATED;
import static org.springframework.http.HttpStatus.INTERNAL_SERVER_ERROR;
import static org.springframework.http.HttpStatus.OK;
import static org.springframework.http.MediaType.APPLICATION_JSON_VALUE;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.content;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.header;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.net.URL;
import java.nio.channels.FileChannel;
import java.nio.file.Files;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.UUID;
import org.alfresco.transform.base.TransformController;
import org.alfresco.transform.base.probes.ProbeTestTransform;
import org.alfresco.transform.client.model.InternalContext;
import org.alfresco.transform.client.model.TransformReply;
import org.alfresco.transform.client.model.TransformRequest;
import org.alfresco.transform.config.SupportedSourceAndTarget;
import org.alfresco.transform.config.TransformConfig;
import org.alfresco.transform.config.TransformOption;
import org.alfresco.transform.config.TransformOptionGroup;
import org.alfresco.transform.config.TransformOptionValue;
import org.alfresco.transform.config.Transformer;
import org.alfresco.transform.registry.TransformServiceRegistry;
import org.alfresco.transform.messages.TransformStack;
import org.alfresco.transform.base.clients.AlfrescoSharedFileStoreClient;
import org.alfresco.transform.base.model.FileRefEntity;
import org.alfresco.transform.base.model.FileRefResponse;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.test.mock.mockito.MockBean;
import org.springframework.boot.test.mock.mockito.SpyBean;
import org.springframework.core.io.ClassPathResource;
import org.springframework.mock.web.MockMultipartFile;
import org.springframework.test.util.ReflectionTestUtils;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.ResultActions;
import org.springframework.test.web.servlet.request.MockHttpServletRequestBuilder;
import org.springframework.test.web.servlet.request.MockMvcRequestBuilders;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
/**
* Super class for testing controllers without a server. Includes tests for the Controller itself.
*/
public abstract class AbstractTransformerControllerTest
{
@TempDir // added as part of ATS-702 to allow test resources to be read from the imported jar files to prevent test resource duplication
public File tempDir;
@Autowired
TransformController controller;
@Autowired
protected MockMvc mockMvc;
@Autowired
protected ObjectMapper objectMapper;
@MockBean
protected AlfrescoSharedFileStoreClient alfrescoSharedFileStoreClient;
@SpyBean
protected TransformServiceRegistry transformRegistry;
@Value("${transform.core.version}")
private String coreVersion;
protected String sourceExtension;
protected String targetExtension;
protected String sourceMimetype;
protected String targetMimetype;
protected HashMap<String, String> options = new HashMap<>();
protected MockMultipartFile sourceFile;
protected String expectedOptions;
protected String expectedSourceSuffix;
protected Long expectedTimeout = 0L;
protected byte[] expectedSourceFileBytes;
/**
* The expected result. Taken resting target quick file's bytes.
*
* Note: These checks generally don't work on Windows (Mac and Linux are okay). Possibly to do with byte order
* loading.
*/
protected byte[] expectedTargetFileBytes;
// Called by sub class
protected abstract void mockTransformCommand(String sourceExtension,
String targetExtension, String sourceMimetype,
boolean readTargetFileBytes) throws IOException;
protected TransformController getController()
{
return controller;
}
protected ProbeTestTransform getProbeTestTransform()
{
return controller.probeTestTransform;
}
protected abstract void updateTransformRequestWithSpecificOptions(
TransformRequest transformRequest);
/**
* This method ends up being the core of the mock.
* It copies content from an existing file in the resources folder to the desired location
* in order to simulate a successful transformation.
*
* @param actualTargetExtension Requested extension.
* @param testFile The test file (transformed) - basically the result.
* @param targetFile The location where the content from the testFile should be copied
* @throws IOException in case of any errors.
*/
public void generateTargetFileFromResourceFile(String actualTargetExtension, File testFile,
File targetFile) throws IOException
{
if (testFile != null)
{
try (var inputStream = new FileInputStream(testFile);
var outputStream = new FileOutputStream(targetFile))
{
FileChannel source = inputStream.getChannel();
FileChannel target = outputStream.getChannel();
target.transferFrom(source, 0, source.size());
} catch (Exception e)
{
throw e;
}
}
else
{
testFile = getTestFile("quick." + actualTargetExtension, false);
if (testFile != null)
{
try (var inputStream = new FileInputStream(testFile);
var outputStream = new FileOutputStream(targetFile))
{
FileChannel source = inputStream.getChannel();
FileChannel target = outputStream.getChannel();
target.transferFrom(source, 0, source.size());
} catch (Exception e)
{
throw e;
}
}
}
}
protected byte[] readTestFile(String extension) throws IOException
{
return Files.readAllBytes(getTestFile("quick." + extension, true).toPath());
}
protected File getTestFile(String testFilename, boolean required) throws IOException
{
File testFile = null;
ClassLoader classLoader = getClass().getClassLoader();
URL testFileUrl = classLoader.getResource(testFilename);
if (required && testFileUrl == null)
{
throw new IOException("The test file " + testFilename +
" does not exist in the resources directory");
}
// added as part of ATS-702 to allow test resources to be read from the imported jar files to prevent test resource duplication
if (testFileUrl!=null)
{
// Each use of the tempDir should result in a unique directory being used
testFile = new File(tempDir, testFilename);
Files.copy(classLoader.getResourceAsStream(testFilename), testFile.toPath(),REPLACE_EXISTING);
}
return testFileUrl == null ? null : testFile;
}
protected MockHttpServletRequestBuilder mockMvcRequest(String url, MockMultipartFile sourceFile,
String... params)
{
if (sourceFile == null)
{
return mockMvcRequestWithoutMockMultipartFile(url, params);
}
else
{
return mockMvcRequestWithMockMultipartFile(url, sourceFile, params);
}
}
private MockHttpServletRequestBuilder mockMvcRequestWithoutMockMultipartFile(String url,
String... params)
{
MockHttpServletRequestBuilder builder = MockMvcRequestBuilders.multipart(ENDPOINT_TRANSFORM);
if (params.length % 2 != 0)
{
throw new IllegalArgumentException("each param should have a name and value.");
}
for (int i = 0; i < params.length; i += 2)
{
builder = builder.param(params[i], params[i + 1]);
}
return builder;
}
private MockHttpServletRequestBuilder mockMvcRequestWithMockMultipartFile(String url, MockMultipartFile sourceFile,
String... params)
{
MockHttpServletRequestBuilder builder = MockMvcRequestBuilders.multipart(ENDPOINT_TRANSFORM).file(
sourceFile);
if (params.length % 2 != 0)
{
throw new IllegalArgumentException("each param should have a name and value.");
}
for (int i = 0; i < params.length; i += 2)
{
builder = builder.param(params[i], params[i + 1]);
}
return builder;
}
protected TransformRequest createTransformRequest(String sourceFileRef, File sourceFile)
{
TransformRequest transformRequest = new TransformRequest();
transformRequest.setRequestId("1");
transformRequest.setSchema(1);
transformRequest.setClientData("Alfresco Digital Business Platform");
transformRequest.setTransformRequestOptions(options);
transformRequest.setSourceReference(sourceFileRef);
transformRequest.setSourceExtension(sourceExtension);
transformRequest.setSourceMediaType(sourceMimetype);
transformRequest.setSourceSize(sourceFile.length());
transformRequest.setTargetExtension(targetExtension);
transformRequest.setTargetMediaType(targetMimetype);
transformRequest.setInternalContext(InternalContext.initialise(null));
transformRequest.getInternalContext().getMultiStep().setInitialRequestId("123");
transformRequest.getInternalContext().getMultiStep().setInitialSourceMediaType(sourceMimetype);
TransformStack.setInitialTransformRequestOptions(transformRequest.getInternalContext(), options);
TransformStack.setInitialSourceReference(transformRequest.getInternalContext(), sourceFileRef);
TransformStack.addTransformLevel(transformRequest.getInternalContext(),
TransformStack.levelBuilder(TransformStack.PIPELINE_FLAG)
.withStep("transformerName", sourceMimetype, targetMimetype));
return transformRequest;
}
@Test
public void simpleTransformTest() throws Exception
{
mockMvc.perform(
mockMvcRequest(ENDPOINT_TRANSFORM, sourceFile, "targetExtension", targetExtension))
.andExpect(status().is(OK.value()))
.andExpect(content().bytes(expectedTargetFileBytes))
.andExpect(header().string("Content-Disposition",
"attachment; filename*= UTF-8''quick." + targetExtension));
}
@Test
public void testDelayTest() throws Exception
{
long start = System.currentTimeMillis();
mockMvc.perform(mockMvcRequest(ENDPOINT_TRANSFORM, sourceFile, "targetExtension", targetExtension,
"testDelay", "400"))
.andExpect(status().is(OK.value()))
.andExpect(content().bytes(expectedTargetFileBytes))
.andExpect(header().string("Content-Disposition",
"attachment; filename*= UTF-8''quick." + targetExtension));
long ms = System.currentTimeMillis() - start;
System.out.println("Transform incluing test delay was " + ms);
assertTrue(ms >= 400, "Delay sending the result back was too small " + ms);
assertTrue(ms <= 500,"Delay sending the result back was too big " + ms);
}
@Test
public void noTargetFileTest() throws Exception
{
mockMvc.perform(mockMvcRequest(ENDPOINT_TRANSFORM, sourceFile, "targetExtension", "xxx"))
.andExpect(status().is(INTERNAL_SERVER_ERROR.value()));
}
@Test
// Looks dangerous but is okay as we only use the final filename
public void dotDotSourceFilenameTest() throws Exception
{
sourceFile = new MockMultipartFile("file", "../quick." + sourceExtension, sourceMimetype,
expectedSourceFileBytes);
mockMvc.perform(
mockMvcRequest(ENDPOINT_TRANSFORM, sourceFile, "targetExtension", targetExtension))
.andExpect(status().is(OK.value()))
.andExpect(content().bytes(expectedTargetFileBytes))
.andExpect(header().string("Content-Disposition",
"attachment; filename*= UTF-8''quick." + targetExtension));
}
@Test
// Is okay, as the target filename is built up from the whole source filename and the targetExtension
public void noExtensionSourceFilenameTest() throws Exception
{
sourceFile = new MockMultipartFile("file", "../quick", sourceMimetype,
expectedSourceFileBytes);
mockMvc.perform(
mockMvcRequest(ENDPOINT_TRANSFORM, sourceFile, "targetExtension", targetExtension))
.andExpect(status().is(OK.value()))
.andExpect(content().bytes(expectedTargetFileBytes))
.andExpect(header().string("Content-Disposition",
"attachment; filename*= UTF-8''quick." + targetExtension));
}
@Test
// Invalid file name that ends in /
public void badSourceFilenameTest() throws Exception
{
sourceFile = new MockMultipartFile("file", "abc/", sourceMimetype, expectedSourceFileBytes);
mockMvc.perform(
mockMvcRequest(ENDPOINT_TRANSFORM, sourceFile, "targetExtension", targetExtension))
.andExpect(status().is(BAD_REQUEST.value()))
.andExpect(status().reason(containsString("The source filename was not supplied")));
}
@Test
public void blankSourceFilenameTest() throws Exception
{
sourceFile = new MockMultipartFile("file", "", sourceMimetype, expectedSourceFileBytes);
mockMvc.perform(
mockMvcRequest(ENDPOINT_TRANSFORM, sourceFile, "targetExtension", targetExtension))
.andExpect(status().is(BAD_REQUEST.value()));
}
@Test
public void noTargetExtensionTest() throws Exception
{
mockMvc.perform(mockMvcRequest(ENDPOINT_TRANSFORM, sourceFile))
.andExpect(status().is(BAD_REQUEST.value()))
.andExpect(status().reason(
containsString("Request parameter 'targetExtension' is missing")));
}
@Test
public void calculateMaxTime() throws Exception
{
ProbeTestTransform probeTestTransform = getController().probeTestTransform;
probeTestTransform.setLivenessPercent(110);
long[][] values = new long[][]{
{5000, 0, Long.MAX_VALUE}, // 1st transform is ignored
{1000, 1000, 2100}, // 1000 + 1000*1.1
{3000, 2000, 4200}, // 2000 + 2000*1.1
{2000, 2000, 4200},
{6000, 3000, 6300},
{8000, 4000, 8400},
{4444, 4000, 8400}, // no longer in the first few, so normal and max times don't change
{5555, 4000, 8400}
};
for (long[] v : values)
{
long time = v[0];
long expectedNormalTime = v[1];
long expectedMaxTime = v[2];
probeTestTransform.calculateMaxTime(time, true);
assertEquals(expectedNormalTime, probeTestTransform.getNormalTime());
assertEquals(expectedMaxTime, probeTestTransform.getMaxTime());
}
}
@Test
public void testEmptyPojoTransform() throws Exception
{
// Transformation Request POJO
TransformRequest transformRequest = new TransformRequest();
// Serialize and call the transformer
String tr = objectMapper.writeValueAsString(transformRequest);
String transformationReplyAsString = mockMvc
.perform(MockMvcRequestBuilders
.post(ENDPOINT_TRANSFORM)
.header(ACCEPT, APPLICATION_JSON_VALUE)
.header(CONTENT_TYPE, APPLICATION_JSON_VALUE)
.content(tr))
.andExpect(status().is(BAD_REQUEST.value()))
.andReturn().getResponse().getContentAsString();
TransformReply transformReply = objectMapper.readValue(transformationReplyAsString,
TransformReply.class);
// Assert the reply
assertEquals(BAD_REQUEST.value(), transformReply.getStatus());
}
/**
*
* @return transformer specific engine config name
*/
public String getEngineConfigName()
{
return "engine_config.json";
}
@Test
public void testGetTransformConfigInfo() throws Exception
{
TransformConfig expectedTransformConfig = objectMapper
.readValue(getTestFile(getEngineConfigName(), true),
TransformConfig.class);
expectedTransformConfig.getTransformers().forEach(transformer -> {
transformer.setCoreVersion(coreVersion);
transformer.getTransformOptions().add(DIRECT_ACCESS_URL);
});
expectedTransformConfig.getTransformOptions().put(DIRECT_ACCESS_URL, Set.of(new TransformOptionValue(false, DIRECT_ACCESS_URL)));
ReflectionTestUtils.setField(transformRegistry, "engineConfig",
new ClassPathResource(getEngineConfigName()));
String response = mockMvc
.perform(MockMvcRequestBuilders.get(ENDPOINT_TRANSFORM_CONFIG_LATEST))
.andExpect(status().is(OK.value()))
.andExpect(header().string(CONTENT_TYPE, APPLICATION_JSON_VALUE))
.andReturn().getResponse().getContentAsString();
TransformConfig transformConfig = objectMapper.readValue(response, TransformConfig.class);
assertEquals(expectedTransformConfig, transformConfig);
}
@Test
// Test for case when T-Router or Repository is a version that does not expect it
public void testGetTransformConfigInfoExcludingCoreVersion() throws Exception
{
TransformConfig expectedTransformConfig = objectMapper
.readValue(getTestFile(getEngineConfigName(), true),
TransformConfig.class);
ReflectionTestUtils.setField(transformRegistry, "engineConfig",
new ClassPathResource(getEngineConfigName()));
String response = mockMvc
.perform(MockMvcRequestBuilders.get(ENDPOINT_TRANSFORM_CONFIG))
.andExpect(status().is(OK.value()))
.andExpect(header().string(CONTENT_TYPE, APPLICATION_JSON_VALUE))
.andReturn().getResponse().getContentAsString();
TransformConfig transformConfig = objectMapper.readValue(response, TransformConfig.class);
assertEquals(expectedTransformConfig, transformConfig);
}
@Test
public void testGetInfoFromConfigWithDuplicates() throws Exception
{
TransformConfig expectedResult = buildCompleteTransformConfig();
ReflectionTestUtils.setField(transformRegistry, "engineConfig",
new ClassPathResource("engine_config_with_duplicates.json"));
String response = mockMvc
.perform(MockMvcRequestBuilders.get(ENDPOINT_TRANSFORM_CONFIG))
.andExpect(status().is(OK.value()))
.andExpect(header().string(CONTENT_TYPE, APPLICATION_JSON_VALUE))
.andReturn().getResponse().getContentAsString();
TransformConfig transformConfig = objectMapper.readValue(response, TransformConfig.class);
assertNotNull(transformConfig);
assertEquals(expectedResult, transformConfig);
assertEquals(3, transformConfig.getTransformOptions().get("engineXOptions").size());
assertEquals(1,
transformConfig.getTransformers().get(0).getSupportedSourceAndTargetList().size());
assertEquals(1,
transformConfig.getTransformers().get(0).getTransformOptions().size());
}
@Test
public void testGetInfoFromConfigWithEmptyTransformOptions() throws Exception
{
Transformer transformer = buildTransformer("application/pdf", "image/png");
TransformConfig expectedResult = new TransformConfig();
expectedResult.setTransformers(ImmutableList.of(transformer));
ReflectionTestUtils.setField(transformRegistry, "engineConfig",
new ClassPathResource("engine_config_incomplete.json"));
String response = mockMvc
.perform(MockMvcRequestBuilders.get(ENDPOINT_TRANSFORM_CONFIG))
.andExpect(status().is(OK.value()))
.andExpect(header().string(CONTENT_TYPE, APPLICATION_JSON_VALUE))
.andReturn().getResponse().getContentAsString();
TransformConfig transformConfig = objectMapper.readValue(response, TransformConfig.class);
assertNotNull(transformConfig);
assertEquals(expectedResult, transformConfig);
}
@Test
public void testGetInfoFromConfigWithNoTransformOptions() throws Exception
{
Transformer transformer = buildTransformer("application/pdf", "image/png");
transformer.setTransformerName("engineX");
TransformConfig expectedResult = new TransformConfig();
expectedResult.setTransformers(ImmutableList.of(transformer));
ReflectionTestUtils.setField(transformRegistry, "engineConfig",
new ClassPathResource("engine_config_no_transform_options.json"));
String response = mockMvc
.perform(MockMvcRequestBuilders.get(ENDPOINT_TRANSFORM_CONFIG))
.andExpect(status().is(OK.value()))
.andExpect(header().string(CONTENT_TYPE, APPLICATION_JSON_VALUE))
.andReturn().getResponse().getContentAsString();
TransformConfig transformConfig = objectMapper.readValue(response, TransformConfig.class);
assertNotNull(transformConfig);
assertEquals(expectedResult, transformConfig);
}
private TransformConfig buildCompleteTransformConfig()
{
TransformConfig expectedResult = new TransformConfig();
Set<TransformOption> transformOptionGroup = ImmutableSet.of(
new TransformOptionValue(false, "cropGravity"));
Set<TransformOption> transformOptions = ImmutableSet.of(
new TransformOptionValue(false, "page"),
new TransformOptionValue(false, "width"),
new TransformOptionGroup(false, transformOptionGroup));
Map<String, Set<TransformOption>> transformOptionsMap = ImmutableMap.of("engineXOptions",
transformOptions);
Transformer transformer = buildTransformer("application/pdf", "image/png", "engineXOptions",
"engineX");
List<Transformer> transformers = ImmutableList.of(transformer);
expectedResult.setTransformOptions(transformOptionsMap);
expectedResult.setTransformers(transformers);
return expectedResult;
}
private Transformer buildTransformer(String sourceMediaType, String targetMediaType,
String transformOptions, String transformerName)
{
Transformer transformer = buildTransformer(sourceMediaType, targetMediaType);
transformer.setTransformerName(transformerName);
transformer.setTransformOptions(ImmutableSet.of(transformOptions));
return transformer;
}
private Transformer buildTransformer(String sourceMediaType, String targetMediaType)
{
Set<SupportedSourceAndTarget> supportedSourceAndTargetList = ImmutableSet.of(
SupportedSourceAndTarget.builder()
.withSourceMediaType(sourceMediaType)
.withTargetMediaType(targetMediaType)
.build());
Transformer transformer = new Transformer();
transformer.setSupportedSourceAndTargetList(supportedSourceAndTargetList);
return transformer;
}
@Test
public void queueTransformRequestUsingDirectAccessUrlTest() throws Exception
{
// Files
String sourceFileRef = UUID.randomUUID().toString();
File sourceFile = getTestFile("quick." + sourceExtension, true);
String targetFileRef = UUID.randomUUID().toString();
TransformRequest transformRequest = createTransformRequest(sourceFileRef, sourceFile);
Map<String, String> transformRequestOptions = transformRequest.getTransformRequestOptions();
String directUrl = "file://" + sourceFile.toPath();
transformRequestOptions.put(DIRECT_ACCESS_URL, directUrl);
transformRequest.setTransformRequestOptions(transformRequestOptions);
when(alfrescoSharedFileStoreClient.saveFile(any()))
.thenReturn(new FileRefResponse(new FileRefEntity(targetFileRef)));
// Update the Transformation Request with any specific params before sending it
updateTransformRequestWithSpecificOptions(transformRequest);
// Serialize and call the transformer
String tr = objectMapper.writeValueAsString(transformRequest);
String transformationReplyAsString = mockMvc
.perform(MockMvcRequestBuilders
.post("/transform")
.header(ACCEPT, APPLICATION_JSON_VALUE)
.header(CONTENT_TYPE, APPLICATION_JSON_VALUE)
.content(tr))
.andExpect(status().is(CREATED.value()))
.andReturn().getResponse().getContentAsString();
TransformReply transformReply = objectMapper.readValue(transformationReplyAsString,
TransformReply.class);
// Assert the reply
assertEquals(transformRequest.getRequestId(), transformReply.getRequestId());
assertEquals(transformRequest.getClientData(), transformReply.getClientData());
assertEquals(transformRequest.getSchema(), transformReply.getSchema());
}
@Test
public void httpTransformRequestUsingDirectAccessUrlTest() throws Exception
{
File dauSourceFile = getTestFile("quick." + sourceExtension, true);
String directUrl = "file://" + dauSourceFile.toPath();
ResultActions resultActions = mockMvc.perform(
mockMvcRequest(ENDPOINT_TRANSFORM, null)
.param("targetExtension", targetExtension)
.param(DIRECT_ACCESS_URL, directUrl))
.andExpect(status().is(OK.value()));
if (expectedTargetFileBytes != null)
{
resultActions.andExpect(content().bytes(expectedTargetFileBytes));
}
}
}

69
t-engine-base/src/test/java/org/alfresco/transform/base/EngineClient.java Normal file
View File

@@ -0,0 +1,69 @@
/*
* Copyright 2015-2022 Alfresco Software, Ltd. All rights reserved.
*
* License rights for this program may be obtained from Alfresco Software, Ltd.
* pursuant to a written agreement and any use of this program without such an
* agreement is prohibited.
*/
package org.alfresco.transform.base;
import static java.util.Collections.emptyMap;
import static org.alfresco.transform.common.RequestParamMap.ENDPOINT_TRANSFORM;
import static org.springframework.http.MediaType.MULTIPART_FORM_DATA;
import java.util.Map;
import org.springframework.core.io.ClassPathResource;
import org.springframework.core.io.Resource;
import org.springframework.http.HttpEntity;
import org.springframework.http.HttpHeaders;
import org.springframework.http.ResponseEntity;
import org.springframework.util.LinkedMultiValueMap;
import org.springframework.util.MultiValueMap;
import org.springframework.web.client.RestTemplate;
/**
* @author Cezar Leahu
*/
public class EngineClient
{
private static final RestTemplate REST_TEMPLATE = new RestTemplate();
public static ResponseEntity<Resource> sendTRequest(
final String engineUrl, final String sourceFile,
final String sourceMimetype, final String targetMimetype, final String targetExtension)
{
return sendTRequest(engineUrl, sourceFile, sourceMimetype, targetMimetype, targetExtension,
emptyMap());
}
public static ResponseEntity<Resource> sendTRequest(
final String engineUrl, final String sourceFile,
final String sourceMimetype, final String targetMimetype, final String targetExtension,
final Map<String, String> transformOptions)
{
final HttpHeaders headers = new HttpHeaders();
headers.setContentType(MULTIPART_FORM_DATA);
//headers.setAccept(ImmutableList.of(MULTIPART_FORM_DATA));
final MultiValueMap<String, Object> body = new LinkedMultiValueMap<>();
body.add("file", new ClassPathResource(sourceFile));
if (sourceMimetype != null && !sourceMimetype.trim().isEmpty())
{
body.add("sourceMimetype", sourceMimetype);
}
if (targetMimetype != null && !targetMimetype.trim().isEmpty())
{
body.add("targetMimetype", targetMimetype);
}
if (targetExtension != null && !targetExtension.trim().isEmpty())
{
body.add("targetExtension", targetExtension);
}
transformOptions.forEach(body::add);
final HttpEntity<MultiValueMap<String, Object>> entity = new HttpEntity<>(body, headers);
return REST_TEMPLATE.postForEntity(engineUrl + ENDPOINT_TRANSFORM, entity, Resource.class);
}
}

240
t-engine-base/src/test/java/org/alfresco/transform/base/QueueTransformServiceTest.java Normal file
View File

@@ -0,0 +1,240 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2021 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.base;
import static org.mockito.Mockito.doReturn;
import static org.mockito.Mockito.doThrow;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.verifyNoMoreInteractions;
import static org.springframework.http.HttpStatus.BAD_REQUEST;
import static org.springframework.http.HttpStatus.CREATED;
import static org.springframework.http.HttpStatus.INTERNAL_SERVER_ERROR;
import javax.jms.Destination;
import javax.jms.JMSException;
import javax.jms.Message;
import org.alfresco.transform.client.model.TransformReply;
import org.alfresco.transform.client.model.TransformRequest;
import org.alfresco.transform.base.messaging.TransformMessageConverter;
import org.alfresco.transform.base.messaging.TransformReplySender;
import org.apache.activemq.command.ActiveMQObjectMessage;
import org.apache.activemq.command.ActiveMQQueue;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.MockitoAnnotations;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.jms.support.converter.MessageConversionException;
public class QueueTransformServiceTest
{
@Mock
private TransformController transformController;
@Mock
private TransformMessageConverter transformMessageConverter;
@Mock
private TransformReplySender transformReplySender;
@InjectMocks
private QueueTransformService queueTransformService;
@BeforeEach
public void setup()
{
MockitoAnnotations.initMocks(this);
}
@Test
public void testWhenReceiveNullMessageThenStopFlow()
{
queueTransformService.receive(null);
verifyNoMoreInteractions(transformController);
verifyNoMoreInteractions(transformMessageConverter);
verifyNoMoreInteractions(transformReplySender);
}
@Test
public void testWhenReceiveMessageWithNoReplyToQueueThenStopFlow()
{
queueTransformService.receive(new ActiveMQObjectMessage());
verifyNoMoreInteractions(transformController);
verifyNoMoreInteractions(transformMessageConverter);
verifyNoMoreInteractions(transformReplySender);
}
@Test
public void testConvertMessageReturnsNullThenReplyWithInternalServerError() throws JMSException
{
ActiveMQObjectMessage msg = new ActiveMQObjectMessage();
msg.setCorrelationId("1234");
ActiveMQQueue destination = new ActiveMQQueue();
msg.setJMSReplyTo(destination);
TransformReply reply = TransformReply
.builder()
.withStatus(INTERNAL_SERVER_ERROR.value())
.withErrorDetails(
"JMS exception during T-Request deserialization of message with correlationID "
+ msg.getCorrelationId() + ": null")
.build();
doReturn(null).when(transformMessageConverter).fromMessage(msg);
queueTransformService.receive(msg);
verify(transformMessageConverter).fromMessage(msg);
verify(transformReplySender).send(destination, reply, msg.getCorrelationId());
verifyNoMoreInteractions(transformController);
}
@Test
public void testConvertMessageThrowsMessageConversionExceptionThenReplyWithBadRequest()
throws JMSException
{
ActiveMQObjectMessage msg = new ActiveMQObjectMessage();
msg.setCorrelationId("1234");
ActiveMQQueue destination = new ActiveMQQueue();
msg.setJMSReplyTo(destination);
TransformReply reply = TransformReply
.builder()
.withStatus(BAD_REQUEST.value())
.withErrorDetails(
"Message conversion exception during T-Request deserialization of message with correlationID"
+ msg.getCorrelationId() + ": null")
.build();
doThrow(MessageConversionException.class).when(transformMessageConverter).fromMessage(msg);
queueTransformService.receive(msg);
verify(transformMessageConverter).fromMessage(msg);
verify(transformReplySender).send(destination, reply, msg.getCorrelationId());
verifyNoMoreInteractions(transformController);
}
@Test
public void testConvertMessageThrowsJMSExceptionThenReplyWithInternalServerError()
throws JMSException
{
ActiveMQObjectMessage msg = new ActiveMQObjectMessage();
msg.setCorrelationId("1234");
ActiveMQQueue destination = new ActiveMQQueue();
msg.setJMSReplyTo(destination);
TransformReply reply = TransformReply
.builder()
.withStatus(INTERNAL_SERVER_ERROR.value())
.withErrorDetails(
"JMSException during T-Request deserialization of message with correlationID " +
msg.getCorrelationId() + ": null")
.build();
doThrow(JMSException.class).when(transformMessageConverter).fromMessage(msg);
queueTransformService.receive(msg);
verify(transformMessageConverter).fromMessage(msg);
verify(transformReplySender).send(destination, reply, msg.getCorrelationId());
verifyNoMoreInteractions(transformController);
}
@Test
public void testWhenReceiveValidTransformRequestThenReplyWithSuccess() throws JMSException
{
ActiveMQObjectMessage msg = new ActiveMQObjectMessage();
ActiveMQQueue destination = new ActiveMQQueue();
msg.setJMSReplyTo(destination);
TransformRequest request = new TransformRequest();
TransformReply reply = TransformReply
.builder()
.withStatus(CREATED.value())
.build();
doReturn(request).when(transformMessageConverter).fromMessage(msg);
doReturn(new ResponseEntity<>(reply, HttpStatus.valueOf(reply.getStatus())))
.when(transformController).transform(request, null);
queueTransformService.receive(msg);
verify(transformMessageConverter).fromMessage(msg);
verify(transformController).transform(request, null);
verify(transformReplySender).send(destination, reply);
}
@Test
public void testWhenJMSExceptionOnMessageIsThrownThenStopFlow() throws JMSException
{
Message msg = mock(Message.class);
doThrow(JMSException.class).when(msg).getJMSReplyTo();
queueTransformService.receive(msg);
verifyNoMoreInteractions(transformController);
verifyNoMoreInteractions(transformMessageConverter);
verifyNoMoreInteractions(transformReplySender);
}
@Test
public void testWhenExceptionOnCorrelationIdIsThrownThenContinueFlowWithNullCorrelationId()
throws JMSException
{
Message msg = mock(Message.class);
Destination destination = mock(Destination.class);
doThrow(JMSException.class).when(msg).getJMSCorrelationID();
doReturn(destination).when(msg).getJMSReplyTo();
TransformRequest request = new TransformRequest();
TransformReply reply = TransformReply
.builder()
.withStatus(CREATED.value())
.build();
doReturn(request).when(transformMessageConverter).fromMessage(msg);
doReturn(new ResponseEntity<>(reply, HttpStatus.valueOf(reply.getStatus())))
.when(transformController).transform(request, null);
queueTransformService.receive(msg);
verify(transformMessageConverter).fromMessage(msg);
verify(transformController).transform(request, null);
verify(transformReplySender).send(destination, reply);
}
}

76
t-engine-base/src/test/java/org/alfresco/transform/base/SourceTarget.java Normal file
View File

@@ -0,0 +1,76 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2019 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.base;
import java.util.Objects;
/**
* @author Cezar Leahu
*/
public class SourceTarget
{
final String source;
final String target;
private SourceTarget(final String source, final String target)
{
this.source = source;
this.target = target;
}
@Override
public boolean equals(Object o)
{
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
SourceTarget that = (SourceTarget) o;
return Objects.equals(source, that.source) &&
Objects.equals(target, that.target);
}
@Override
public int hashCode()
{
return Objects.hash(source, target);
}
@Override
public String toString()
{
return source + '|' + target;
}
public static SourceTarget of(final String source, final String target)
{
return sourceTarget(source, target);
}
public static SourceTarget sourceTarget(final String source, final String target)
{
return new SourceTarget(source, target);
}
}

85
t-engine-base/src/test/java/org/alfresco/transform/base/TestFileInfo.java Normal file
View File

@@ -0,0 +1,85 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2019 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.base;
/**
* @author Cezar Leahu
*/
public class TestFileInfo
{
private final String mimeType;
private final String extension;
private final String path;
private final boolean exactMimeType;
public TestFileInfo(final String mimeType, final String extension, final String path,
final boolean exactMimeType)
{
this.mimeType = mimeType;
this.extension = extension;
this.path = path;
this.exactMimeType = exactMimeType;
}
public String getMimeType()
{
return mimeType;
}
public String getExtension()
{
return extension;
}
public String getPath()
{
return path;
}
public boolean isExactMimeType()
{
return exactMimeType;
}
public static TestFileInfo testFile(final String mimeType, final String extension,
final String path, final boolean exactMimeType)
{
return new TestFileInfo(mimeType, extension, path, exactMimeType);
}
public static TestFileInfo testFile(final String mimeType, final String extension,
final String path)
{
return new TestFileInfo(mimeType, extension, path, false);
}
@Override
public String toString()
{
return path;
}
}

22
t-engine-base/src/test/resources/engine_config_complete.json Normal file
View File

@@ -0,0 +1,22 @@
{
"transformOptions": {
"engineXOptions": [
{"value": {"name": "page"}},
{"value": {"name": "width"}},
{"group": {"transformOptions": [
{"value": {"name": "cropGravity"}}
]}}
]
},
"transformers": [
{
"transformerName": "engineX",
"supportedSourceAndTargetList": [
{"sourceMediaType": "application/pdf", "targetMediaType": "image/png" }
],
"transformOptions": [
"engineXOptions"
]
}
]
}

10
t-engine-base/src/test/resources/engine_config_incomplete.json Normal file
View File

@@ -0,0 +1,10 @@
{
"transformOptions": {},
"transformers": [
{
"supportedSourceAndTargetList": [
{"sourceMediaType": "application/pdf", "targetMediaType": "image/png" }
]
}
]
}

10
t-engine-base/src/test/resources/engine_config_no_transform_options.json Normal file
View File

@@ -0,0 +1,10 @@
{
"transformers": [
{
"transformerName": "engineX",
"supportedSourceAndTargetList": [
{"sourceMediaType": "application/pdf", "targetMediaType": "image/png" }
]
}
]
}

26
t-engine-base/src/test/resources/engine_config_with_duplicates.json Normal file
View File

@@ -0,0 +1,26 @@
{
"transformOptions": {
"engineXOptions": [
{"value": {"name": "page"}},
{"value": {"name": "page"}},
{"value": {"name": "width"}},
{"group": {"transformOptions": [
{"value": {"name": "cropGravity"}}
]}}
]
},
"transformers": [
{
"transformerName": "engineX",
"supportedSourceAndTargetList": [
{"sourceMediaType": "application/pdf", "targetMediaType": "image/png" },
{"sourceMediaType": "application/pdf", "targetMediaType": "image/png" },
{"sourceMediaType": "application/pdf", "targetMediaType": "image/png" }
],
"transformOptions": [
"engineXOptions",
"engineXOptions"
]
}
]
}