HXENG-64 refactor ATS (#657)

Refactor to clean up packages in the t-model and to introduce a simpler to implement t-engine base.

The new t-engines (tika, imagemagick, libreoffice, pdfrenderer, misc, aio, aspose) and t-router may be used in combination with older components as the API between the content Repo and between components has not changed. As far as possible the same artifacts are created (the -boot projects no longer exist). They may be used with older ACS repo versions.

The main changes to look for are:
* The introduction of TransformEngine and CustomTransformer interfaces to be implemented.
* The removal in t-engines and t-router of the Controller, Application, test template page, Controller tests and application config, as this is all now done by the t-engine base package.
* The t-router now extends the t-engine base, which also reduced the amount of duplicate code.
* The t-engine base provides the test page, which includes drop downs of known transform options. The t-router is able to use pipeline and failover transformers. This was not possible to do previously as the router had no test UI.
* Resources including licenses are automatically included in the all-in-one t-engine, from the individual t-engines. They just need to be added as dependencies in the pom. 
* The ugly code in the all-in-one t-engine and misc t-engine to pick transformers has gone, as they are now just selected by the transformRegistry.
* The way t-engines respond to http or message queue transform requests has been combined (eliminates the similar but different code that existed before).
* The t-engine base now uses InputStream and OutputStream rather than Files by default. As a result it will be simpler to avoid writing content to a temporary location.
* A number of the Tika and Misc CustomTransforms no longer use Files.
* The original t-engine base still exists so customers can continue to create custom t-engines the way they have done previously. the project has just been moved into a folder called deprecated.
* The folder structure has changed. The long "alfresco-transform-..." names have given way to shorter easier to read and type names.
* The t-engine project structure now has a single project rather than two. 
* The previous config values still exist, but there are now a new set for config values for in files with names that don't misleadingly imply they only contain pipeline of routing information. 
* The concept of 'routing' has much less emphasis in class names as the code just uses the transformRegistry. 
* TransformerConfig may now be read as json or yaml. The restrictions about what could be specified in yaml has gone.
* T-engines and t-router may use transform config from files. Previously it was just the t-router.
* The POC code to do with graphs of possible routes has been removed.
* All master branch changes have been merged in.
* The concept of a single transform request which results in multiple responses (e.g. images from a video) has been added to the core processing of requests in the t-engine base.
* Many SonarCloud linter fixes.
This commit is contained in:
Alan Davis
2022-09-14 13:40:19 +01:00
committed by GitHub
parent ea83ef9ebc
commit babe26b0ba
652 changed files with 19479 additions and 18195 deletions

View File

@@ -0,0 +1,581 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transformer;
import org.alfresco.transform.common.TransformerDebug;
import org.alfresco.transform.client.model.InternalContext;
import org.alfresco.transform.client.model.TransformReply;
import org.alfresco.transform.client.model.TransformRequest;
import org.alfresco.transform.messages.TransformRequestValidator;
import org.alfresco.transform.config.TransformConfig;
import org.alfresco.transform.registry.TransformServiceRegistry;
import org.alfresco.transform.exceptions.TransformException;
import org.alfresco.transformer.clients.AlfrescoSharedFileStoreClient;
import org.alfresco.transformer.logging.LogEntry;
import org.alfresco.transformer.model.FileRefResponse;
import org.codehaus.plexus.util.FileUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.io.Resource;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.validation.DirectFieldBindingResult;
import org.springframework.validation.Errors;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.ResponseBody;
import org.springframework.web.client.HttpClientErrorException;
import org.springframework.web.multipart.MultipartFile;
import javax.servlet.http.HttpServletRequest;
import java.io.File;
import java.io.IOException;
import java.net.URL;
import java.util.Arrays;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.concurrent.atomic.AtomicInteger;
import static java.util.stream.Collectors.joining;
import static org.alfresco.transform.config.CoreVersionDecorator.setOrClearCoreVersion;
import static org.alfresco.transform.common.RequestParamMap.DIRECT_ACCESS_URL;
import static org.alfresco.transform.common.RequestParamMap.CONFIG_VERSION;
import static org.alfresco.transform.common.RequestParamMap.CONFIG_VERSION_DEFAULT;
import static org.alfresco.transform.common.RequestParamMap.ENDPOINT_TRANSFORM;
import static org.alfresco.transform.common.RequestParamMap.ENDPOINT_TRANSFORM_CONFIG;
import static org.alfresco.transformer.fs.FileManager.TempFileProvider.createTempFile;
import static org.alfresco.transformer.fs.FileManager.buildFile;
import static org.alfresco.transformer.fs.FileManager.createAttachment;
import static org.alfresco.transformer.fs.FileManager.createSourceFile;
import static org.alfresco.transformer.fs.FileManager.createTargetFile;
import static org.alfresco.transformer.fs.FileManager.createTargetFileName;
import static org.alfresco.transformer.fs.FileManager.deleteFile;
import static org.alfresco.transformer.fs.FileManager.getFilenameFromContentDisposition;
import static org.alfresco.transformer.fs.FileManager.save;
import static org.alfresco.transformer.util.RequestParamMap.FILE;
import static org.alfresco.transformer.util.RequestParamMap.SOURCE_ENCODING;
import static org.alfresco.transformer.util.RequestParamMap.SOURCE_EXTENSION;
import static org.alfresco.transformer.util.RequestParamMap.SOURCE_MIMETYPE;
import static org.alfresco.transformer.util.RequestParamMap.TARGET_EXTENSION;
import static org.alfresco.transformer.util.RequestParamMap.TARGET_MIMETYPE;
import static org.alfresco.transformer.util.RequestParamMap.TEST_DELAY;
import static org.alfresco.transformer.util.RequestParamMap.TRANSFORM_NAME_PROPERTY;
import static org.springframework.http.HttpStatus.BAD_REQUEST;
import static org.springframework.http.HttpStatus.CREATED;
import static org.springframework.http.HttpStatus.INTERNAL_SERVER_ERROR;
import static org.springframework.http.HttpStatus.OK;
import static org.springframework.http.MediaType.APPLICATION_JSON_VALUE;
import static org.springframework.http.MediaType.MULTIPART_FORM_DATA_VALUE;
import static org.springframework.util.StringUtils.getFilenameExtension;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
*
* <p>Abstract Controller, provides structure and helper methods to sub-class transformer controllers. Sub classes
* should implement {@link #transformImpl(String, String, String, Map, File, File)} and unimplemented methods from
* {@link TransformController}.</p>
*
* <p>Status Codes:</p>
* <ul>
* <li>200 Success</li>
* <li>400 Bad Request: Request parameter <name> is missing (missing mandatory parameter)</li>
* <li>400 Bad Request: Request parameter <name> is of the wrong type</li>
* <li>400 Bad Request: Transformer exit code was not 0 (possible problem with the source file)</li>
* <li>400 Bad Request: The source filename was not supplied</li>
* <li>500 Internal Server Error: (no message with low level IO problems)</li>
* <li>500 Internal Server Error: The target filename was not supplied (should not happen as targetExtension is checked)</li>
* <li>500 Internal Server Error: Transformer version check exit code was not 0</li>
* <li>500 Internal Server Error: Transformer version check failed to create any output</li>
* <li>500 Internal Server Error: Could not read the target file</li>
* <li>500 Internal Server Error: The target filename was malformed (should not happen because of other checks)</li>
* <li>500 Internal Server Error: Transformer failed to create an output file (the exit code was 0, so there should be some content)</li>
* <li>500 Internal Server Error: Filename encoding error</li>
* <li>507 Insufficient Storage: Failed to store the source file</li>
*
* <li>408 Request Timeout -- TODO implement general timeout mechanism rather than depend on transformer timeout
* (might be possible for external processes)</li>
* <li>415 Unsupported Media Type -- TODO possibly implement a check on supported source and target mimetypes (probably not)</li>
* <li>429 Too Many Requests: Returned by liveness probe</li>
* </ul>
* <p>Provides methods to help super classes perform /transform requests. Also responses to /version, /ready and /live
* requests.</p>
*/
@Deprecated
public abstract class AbstractTransformerController implements TransformController
{
private static final Logger logger = LoggerFactory.getLogger(
AbstractTransformerController.class);
// Request parameters that are not part of transform options
public static final List<String> NON_TRANSFORM_OPTION_REQUEST_PARAMETERS = Arrays.asList(SOURCE_EXTENSION,
TARGET_EXTENSION, TARGET_MIMETYPE, SOURCE_MIMETYPE, TEST_DELAY, TRANSFORM_NAME_PROPERTY, DIRECT_ACCESS_URL);
@Autowired
private AlfrescoSharedFileStoreClient alfrescoSharedFileStoreClient;
@Autowired
private TransformRequestValidator transformRequestValidator;
@Autowired
private TransformServiceRegistry transformRegistry;
@Autowired
private TransformerDebug transformerDebug;
private AtomicInteger httpRequestCount = new AtomicInteger(1);
@GetMapping(value = ENDPOINT_TRANSFORM_CONFIG)
public ResponseEntity<TransformConfig> info(
@RequestParam(value = CONFIG_VERSION, defaultValue = CONFIG_VERSION_DEFAULT) int configVersion)
{
logger.info("GET Transform Config version: " + configVersion);
TransformConfig transformConfig = ((TransformRegistryImpl) transformRegistry).getTransformConfig();
transformConfig = setOrClearCoreVersion(transformConfig, configVersion);
return new ResponseEntity<>(transformConfig, OK);
}
@PostMapping(value = ENDPOINT_TRANSFORM, consumes = MULTIPART_FORM_DATA_VALUE)
public ResponseEntity<Resource> transform(HttpServletRequest request,
@RequestParam(value = FILE, required = false) MultipartFile sourceMultipartFile,
@RequestParam(TARGET_EXTENSION) String targetExtension,
@RequestParam(value = SOURCE_MIMETYPE, required = false) String sourceMimetype,
@RequestParam(value = TARGET_MIMETYPE, required = false) String targetMimetype,
@RequestParam Map<String, String> requestParameters,
@RequestParam(value = TEST_DELAY, required = false) Long testDelay,
// The TRANSFORM_NAME_PROPERTY param allows ACS legacy transformers to specify which transform to use,
// It can be removed once legacy transformers are removed from ACS.
@RequestParam(value = TRANSFORM_NAME_PROPERTY, required = false) String requestTransformName)
{
if (logger.isDebugEnabled())
{
logger.debug("Processing request via HTTP endpoint. Params: sourceMimetype: '{}', targetMimetype: '{}', "
+ "targetExtension: '{}', requestParameters: {}", sourceMimetype, targetMimetype, targetExtension, requestParameters);
}
final String directUrl = requestParameters.getOrDefault(DIRECT_ACCESS_URL, "");
File sourceFile;
String sourceFilename;
if (directUrl.isBlank())
{
if (sourceMultipartFile == null)
{
throw new TransformException(BAD_REQUEST, "Required request part 'file' is not present");
}
sourceFile = createSourceFile(request, sourceMultipartFile);
sourceFilename = sourceMultipartFile.getOriginalFilename();
}
else
{
sourceFile = getSourceFileFromDirectUrl(directUrl);
sourceFilename = sourceFile.getName();
}
final String targetFilename = createTargetFileName(sourceFilename, targetExtension);
getProbeTestTransform().incrementTransformerCount();
final File targetFile = createTargetFile(request, targetFilename);
Map<String, String> transformOptions = getTransformOptions(requestParameters);
String transformName = getTransformerName(sourceMimetype, targetMimetype, requestTransformName, sourceFile, transformOptions);
String reference = "e"+httpRequestCount.getAndIncrement();
transformerDebug.pushTransform(reference, sourceMimetype, targetMimetype, sourceFile, transformName);
transformerDebug.logOptions(reference, requestParameters);
try
{
transformImpl(transformName, sourceMimetype, targetMimetype, transformOptions, sourceFile, targetFile);
final ResponseEntity<Resource> body = createAttachment(targetFilename, targetFile);
LogEntry.setTargetSize(targetFile.length());
long time = LogEntry.setStatusCodeAndMessage(OK.value(), "Success");
time += LogEntry.addDelay(testDelay);
getProbeTestTransform().recordTransformTime(time);
transformerDebug.popTransform(reference, time);
return body;
}
catch (Throwable t)
{
transformerDebug.logFailure(reference, t.getMessage());
throw t;
}
}
private File getSourceFileFromDirectUrl(String directUrl)
{
File sourceFile = createTempFile("tmp", ".tmp");
try
{
FileUtils.copyURLToFile(new URL(directUrl), sourceFile);
}
catch (IllegalArgumentException e)
{
throw new TransformException(BAD_REQUEST, "Direct Access Url is invalid.", e);
}
catch (IOException e)
{
throw new TransformException(BAD_REQUEST, "Direct Access Url not found.", e);
}
return sourceFile;
}
protected Map<String, String> getTransformOptions(Map<String, String> requestParameters)
{
Map<String, String> transformOptions = new HashMap<>(requestParameters);
transformOptions.keySet().removeAll(NON_TRANSFORM_OPTION_REQUEST_PARAMETERS);
transformOptions.values().removeIf(v -> v.isEmpty());
return transformOptions;
}
/**
* '/transform' endpoint which consumes and produces 'application/json'
*
* This is the way to tell Spring to redirect the request to this endpoint
* instead of the one which produces 'html'
*
* @param request The transformation request
* @param timeout Transformation timeout
* @return A transformation reply
*/
@PostMapping(value = ENDPOINT_TRANSFORM, produces = APPLICATION_JSON_VALUE)
@ResponseBody
public ResponseEntity<TransformReply> transform(@RequestBody TransformRequest request,
@RequestParam(value = "timeout", required = false) Long timeout)
{
logger.trace("Received {}, timeout {} ms", request, timeout);
final TransformReply reply = new TransformReply();
reply.setRequestId(request.getRequestId());
reply.setSourceReference(request.getSourceReference());
reply.setSchema(request.getSchema());
reply.setClientData(request.getClientData());
final Errors errors = validateTransformRequest(request);
validateInternalContext(request, errors);
initialiseContext(request);
reply.setInternalContext(request.getInternalContext());
if (!errors.getAllErrors().isEmpty())
{
reply.setStatus(BAD_REQUEST.value());
reply.setErrorDetails(errors
.getAllErrors()
.stream()
.map(Object::toString)
.collect(joining(", ")));
transformerDebug.logFailure(reply);
logger.trace("Invalid request, sending {}", reply);
return new ResponseEntity<>(reply, HttpStatus.valueOf(reply.getStatus()));
}
transformerDebug.pushTransform(request);
// Load the source file
File sourceFile;
try
{
final String directUrl = request.getTransformRequestOptions().getOrDefault(DIRECT_ACCESS_URL, "");
if (directUrl.isBlank())
{
sourceFile = loadSourceFile(request.getSourceReference(), request.getSourceExtension());
}
else
{
sourceFile = getSourceFileFromDirectUrl(directUrl);
}
}
catch (TransformException e)
{
reply.setStatus(e.getStatusCode());
reply.setErrorDetails(messageWithCause("Failed at reading the source file", e));
transformerDebug.logFailure(reply);
logger.trace("Failed to load source file (TransformException), sending " + reply);
return new ResponseEntity<>(reply, HttpStatus.valueOf(reply.getStatus()));
}
catch (HttpClientErrorException e)
{
reply.setStatus(e.getStatusCode().value());
reply.setErrorDetails(messageWithCause("Failed at reading the source file", e));
transformerDebug.logFailure(reply);
logger.trace("Failed to load source file (HttpClientErrorException), sending " + reply, e);
return new ResponseEntity<>(reply, HttpStatus.valueOf(reply.getStatus()));
}
catch (Exception e)
{
reply.setStatus(INTERNAL_SERVER_ERROR.value());
reply.setErrorDetails(messageWithCause("Failed at reading the source file", e));
transformerDebug.logFailure(reply);
logger.trace("Failed to load source file (Exception), sending " + reply, e);
return new ResponseEntity<>(reply, HttpStatus.valueOf(reply.getStatus()));
}
// Create local temp target file in order to run the transformation
final String targetFilename = createTargetFileName(sourceFile.getName(),
request.getTargetExtension());
final File targetFile = buildFile(targetFilename);
// Run the transformation
try
{
String targetMimetype = request.getTargetMediaType();
String sourceMimetype = request.getSourceMediaType();
Map<String, String> transformOptions = getTransformOptions(request.getTransformRequestOptions());
transformerDebug.logOptions(request);
String transformName = getTransformerName(sourceFile, sourceMimetype, targetMimetype, transformOptions);
transformImpl(transformName, sourceMimetype, targetMimetype, transformOptions, sourceFile, targetFile);
reply.getInternalContext().setCurrentSourceSize(targetFile.length());
}
catch (TransformException e)
{
reply.setStatus(e.getStatusCode());
reply.setErrorDetails(messageWithCause("Failed at processing transformation", e));
transformerDebug.logFailure(reply);
logger.trace("Failed to perform transform (TransformException), sending " + reply, e);
return new ResponseEntity<>(reply, HttpStatus.valueOf(reply.getStatus()));
}
catch (Exception e)
{
reply.setStatus(INTERNAL_SERVER_ERROR.value());
reply.setErrorDetails(messageWithCause("Failed at processing transformation", e));
transformerDebug.logFailure(reply);
logger.trace("Failed to perform transform (Exception), sending " + reply, e);
return new ResponseEntity<>(reply, HttpStatus.valueOf(reply.getStatus()));
}
// Write the target file
FileRefResponse targetRef;
try
{
targetRef = alfrescoSharedFileStoreClient.saveFile(targetFile);
}
catch (TransformException e)
{
reply.setStatus(e.getStatusCode());
reply.setErrorDetails(messageWithCause("Failed at writing the transformed file", e));
transformerDebug.logFailure(reply);
logger.trace("Failed to save target file (TransformException), sending " + reply, e);
return new ResponseEntity<>(reply, HttpStatus.valueOf(reply.getStatus()));
}
catch (HttpClientErrorException e)
{
reply.setStatus(e.getStatusCode().value());
reply.setErrorDetails(messageWithCause("Failed at writing the transformed file. ", e));
transformerDebug.logFailure(reply);
logger.trace("Failed to save target file (HttpClientErrorException), sending " + reply, e);
return new ResponseEntity<>(reply, HttpStatus.valueOf(reply.getStatus()));
}
catch (Exception e)
{
reply.setStatus(INTERNAL_SERVER_ERROR.value());
reply.setErrorDetails(messageWithCause("Failed at writing the transformed file. ", e));
transformerDebug.logFailure(reply);
logger.trace("Failed to save target file (Exception), sending " + reply, e);
return new ResponseEntity<>(reply, HttpStatus.valueOf(reply.getStatus()));
}
try
{
deleteFile(targetFile);
}
catch (Exception e)
{
logger.error("Failed to delete local temp target file '{}'. Error will be ignored ",
targetFile, e);
}
try
{
deleteFile(sourceFile);
}
catch (Exception e)
{
logger.error("Failed to delete source local temp file " + sourceFile, e);
}
reply.setTargetReference(targetRef.getEntry().getFileRef());
reply.setStatus(CREATED.value());
transformerDebug.popTransform(reply);
logger.trace("Sending successful {}, timeout {} ms", reply, timeout);
return new ResponseEntity<>(reply, HttpStatus.valueOf(reply.getStatus()));
}
private Errors validateTransformRequest(final TransformRequest transformRequest)
{
DirectFieldBindingResult errors = new DirectFieldBindingResult(transformRequest, "request");
transformRequestValidator.validate(transformRequest, errors);
return errors;
}
private void validateInternalContext(TransformRequest request, Errors errors)
{
String errorMessage = InternalContext.checkForBasicErrors(request.getInternalContext(), "T-Request");
if (errorMessage != null)
{
errors.rejectValue("internalContext", null, errorMessage);
}
}
private void initialiseContext(TransformRequest request)
{
// If needed initialise the context enough to allow logging to take place without NPE checks
request.setInternalContext(InternalContext.initialise(request.getInternalContext()));
}
/**
* Loads the file with the specified sourceReference from Alfresco Shared File Store
*
* @param sourceReference reference to the file in Alfresco Shared File Store
* @param sourceExtension default extension if the file in Alfresco Shared File Store has none
* @return the file containing the source content for the transformation
*/
private File loadSourceFile(final String sourceReference, final String sourceExtension)
{
ResponseEntity<Resource> responseEntity = alfrescoSharedFileStoreClient
.retrieveFile(sourceReference);
getProbeTestTransform().incrementTransformerCount();
HttpHeaders headers = responseEntity.getHeaders();
String filename = getFilenameFromContentDisposition(headers);
String extension = getFilenameExtension(filename) != null ? getFilenameExtension(filename) : sourceExtension;
MediaType contentType = headers.getContentType();
long size = headers.getContentLength();
final Resource body = responseEntity.getBody();
if (body == null)
{
String message = "Source file with reference: " + sourceReference + " is null or empty. "
+ "Transformation will fail and stop now as there is no content to be transformed.";
logger.warn(message);
throw new TransformException(BAD_REQUEST, message);
}
final File file = createTempFile("source_", "." + extension);
logger.debug("Read source content {} length={} contentType={}",
sourceReference, size, contentType);
save(body, file);
LogEntry.setSource(filename, size);
return file;
}
private static String messageWithCause(final String prefix, Throwable e)
{
final StringBuilder sb = new StringBuilder();
sb.append(prefix).append(" - ")
.append(e.getClass().getSimpleName()).append(": ")
.append(e.getMessage());
while (e.getCause() != null)
{
e = e.getCause();
sb.append(", cause ")
.append(e.getClass().getSimpleName()).append(": ")
.append(e.getMessage());
}
return sb.toString();
}
private String getTransformerName(String sourceMimetype, String targetMimetype,
String requestTransformName, File sourceFile,
Map<String, String> transformOptions)
{
// Check if transformName was provided in the request (this can happen for ACS legacy transformers)
String transformName = requestTransformName;
if (transformName == null || transformName.isEmpty())
{
transformName = getTransformerName(sourceFile, sourceMimetype, targetMimetype, transformOptions);
}
else if (logger.isInfoEnabled())
{
logger.trace("Using transform name provided in the request: " + requestTransformName);
}
return transformName;
}
protected String getTransformerName(final File sourceFile, final String sourceMimetype,
final String targetMimetype, final Map<String, String> transformOptions)
{
// The transformOptions always contains sourceEncoding when sent to a T-Engine, even though it should not be
// used to select a transformer. Similar to source and target mimetypes and extensions, but these are not
// passed in transformOptions.
String sourceEncoding = transformOptions.remove(SOURCE_ENCODING);
try
{
final long sourceSizeInBytes = sourceFile.length();
final String transformerName = transformRegistry.findTransformerName(sourceMimetype,
sourceSizeInBytes, targetMimetype, transformOptions, null);
if (transformerName == null)
{
throw new TransformException(BAD_REQUEST, "No transforms for:");
}
return transformerName;
}
finally
{
if (sourceEncoding != null)
{
transformOptions.put(SOURCE_ENCODING, sourceEncoding);
}
}
}
protected Map<String, String> createTransformOptions(Object... namesAndValues)
{
if (namesAndValues.length % 2 != 0)
{
logger.error(
"Incorrect number of parameters. Should have an even number as they are names and values.");
}
Map<String, String> transformOptions = new HashMap<>();
for (int i = 0; i < namesAndValues.length; i += 2)
{
String name = namesAndValues[i].toString();
Object value = namesAndValues[i + 1];
if (value != null && (!(value instanceof String) || !((String)value).isBlank()))
{
transformOptions.put(name, value.toString());
}
}
return transformOptions;
}
}

View File

@@ -0,0 +1,208 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transformer;
import static org.springframework.http.HttpStatus.BAD_REQUEST;
import static org.springframework.http.HttpStatus.INTERNAL_SERVER_ERROR;
import java.util.Optional;
import javax.jms.Destination;
import javax.jms.JMSException;
import javax.jms.Message;
import org.alfresco.transform.client.model.TransformReply;
import org.alfresco.transform.client.model.TransformRequest;
import org.alfresco.transform.exceptions.TransformException;
import org.alfresco.transformer.messaging.TransformMessageConverter;
import org.alfresco.transformer.messaging.TransformReplySender;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.http.HttpStatus;
import org.springframework.jms.annotation.JmsListener;
import org.springframework.jms.support.converter.MessageConversionException;
import org.springframework.stereotype.Component;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
*
* Queue Transformer service.
* This service reads all the requests for the particular engine, forwards them to the worker
* component (at this time the injected controller - to be refactored) and sends back the reply
* to the {@link Message#getJMSReplyTo()} value. If this value is missing we've got to a dead end.
*
* @author Lucian Tuca
* created on 18/12/2018
*/
@Deprecated
@Component
@ConditionalOnProperty(name = "activemq.url")
public class QueueTransformService
{
private static final Logger logger = LoggerFactory.getLogger(QueueTransformService.class);
// TODO: I know this is not smart but all the the transformation logic is in the Controller.
// The controller also manages the probes. There's tons of refactoring needed there, hence this. Sorry.
@Autowired
private TransformController transformController;
@Autowired
private TransformMessageConverter transformMessageConverter;
@Autowired
private TransformReplySender transformReplySender;
@JmsListener(destination = "${queue.engineRequestQueue}", concurrency = "${jms-listener.concurrency}")
public void receive(final Message msg)
{
if (msg == null)
{
logger.error("Received null message!");
return;
}
final String correlationId = tryRetrieveCorrelationId(msg);
Destination replyToDestinationQueue;
try
{
replyToDestinationQueue = msg.getJMSReplyTo();
if (replyToDestinationQueue == null)
{
logger.error(
"Cannot find 'replyTo' destination queue for message with correlationID {}. Stopping. ",
correlationId);
return;
}
}
catch (JMSException e)
{
logger.error(
"Cannot find 'replyTo' destination queue for message with correlationID {}. Stopping. ",
correlationId);
return;
}
logger.trace("New T-Request from queue with correlationId: {}", correlationId);
Optional<TransformRequest> transformRequest;
try
{
transformRequest = convert(msg, correlationId);
}
catch (TransformException e)
{
logger.error(e.getMessage(), e);
replyWithError(replyToDestinationQueue, HttpStatus.valueOf(e.getStatusCode()),
e.getMessage(), correlationId);
return;
}
if (!transformRequest.isPresent())
{
logger.error("T-Request from message with correlationID {} is null!", correlationId);
replyWithInternalSvErr(replyToDestinationQueue,
"JMS exception during T-Request deserialization: ", correlationId);
return;
}
TransformReply reply = transformController.transform(transformRequest.get(), null)
.getBody();
transformReplySender.send(replyToDestinationQueue, reply);
}
/**
* Tries to convert the JMS {@link Message} to a {@link TransformRequest}
* If any error occurs, a {@link TransformException} is thrown
*
* @param msg Message to be deserialized
* @return The converted {@link TransformRequest} instance
*/
private Optional<TransformRequest> convert(final Message msg, String correlationId)
{
try
{
TransformRequest request = (TransformRequest) transformMessageConverter.fromMessage(msg);
return Optional.ofNullable(request);
}
catch (MessageConversionException e)
{
String message =
"MessageConversionException during T-Request deserialization of message with correlationID "
+ correlationId + ": ";
throw new TransformException(BAD_REQUEST, message + e.getMessage());
}
catch (JMSException e)
{
String message =
"JMSException during T-Request deserialization of message with correlationID "
+ correlationId + ": ";
throw new TransformException(INTERNAL_SERVER_ERROR, message + e.getMessage());
}
catch (Exception e)
{
String message =
"Exception during T-Request deserialization of message with correlationID "
+ correlationId + ": ";
throw new TransformException(INTERNAL_SERVER_ERROR, message + e.getMessage());
}
}
private void replyWithInternalSvErr(final Destination destination, final String msg,
final String correlationId)
{
replyWithError(destination, INTERNAL_SERVER_ERROR, msg, correlationId);
}
private void replyWithError(final Destination destination, final HttpStatus status,
final String msg,
final String correlationId)
{
final TransformReply reply = TransformReply
.builder()
.withStatus(status.value())
.withErrorDetails(msg)
.build();
transformReplySender.send(destination, reply, correlationId);
}
private static String tryRetrieveCorrelationId(final Message msg)
{
try
{
return msg.getJMSCorrelationID();
}
catch (Exception ignore)
{
return null;
}
}
}

View File

@@ -0,0 +1,227 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transformer;
import static java.text.MessageFormat.format;
import static org.springframework.http.HttpStatus.BAD_REQUEST;
import java.io.File;
import java.io.IOException;
import java.util.Collection;
import java.util.Map;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import org.alfresco.transform.client.model.TransformReply;
import org.alfresco.transform.client.model.TransformRequest;
import org.alfresco.transform.exceptions.TransformException;
import org.alfresco.transformer.logging.LogEntry;
import org.alfresco.transformer.probes.ProbeTestTransform;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.TypeMismatchException;
import org.springframework.http.ResponseEntity;
import org.springframework.ui.Model;
import org.springframework.web.bind.MissingServletRequestParameterException;
import org.springframework.web.bind.annotation.ExceptionHandler;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.ResponseBody;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
*
* TransformController interface.
* <br/>
* It contains much of the common boilerplate code that each of
* its concrete implementations need as default methods.
*/
@Deprecated
public interface TransformController
{
Logger logger = LoggerFactory.getLogger(TransformController.class);
/**
* Should be overridden in subclasses to initiate the transformation.
*
* @param transformName the name of the transformer in the engine_config.json file
* @param sourceMimetype mimetype of the source
* @param targetMimetype mimetype of the target
* @param transformOptions transform options from the client
* @param sourceFile the source file
* @param targetFile the target file
*/
void transformImpl(String transformName, String sourceMimetype, String targetMimetype,
Map<String, String> transformOptions, File sourceFile, File targetFile);
/**
* @deprecated use {@link #transformImpl(String, String, String, Map, File, File)} and timeout should be part of
* the transformOptions created from the TransformRequest.
*/
@Deprecated
ResponseEntity<TransformReply> transform(TransformRequest transformRequest, Long timeout);
/**
* @deprecated use {@link #transformImpl(String, String, String, Map, File, File)}.
*/
@Deprecated
default void processTransform(final File sourceFile, final File targetFile,
final String sourceMimetype, final String targetMimetype,
final Map<String, String> transformOptions, final Long timeout)
{
}
/**
* @return a friendly name for the T-Engine.
*/
String getTransformerName();
/**
* Provides the Kubernetes pod probes.
*/
ProbeTestTransform getProbeTestTransform();
/**
* Method used by Kubernetes pod probes.
*/
default String probe(HttpServletRequest request, boolean isLiveProbe)
{
return getProbeTestTransform().doTransformOrNothing(request, isLiveProbe);
}
/**
* @return a string that may be used by clients in debug. It need not include the version.
*/
@RequestMapping("/version")
@ResponseBody
String version();
/**
* @return the name of a template to test the T-Engine. Defaults to {@code "transformForm"}.
*/
@GetMapping("/")
default String transformForm(Model model)
{
return "transformForm"; // the name of the template
}
/**
* @return the name of a template to display when there is an error when using the test UI for the T-Engine.
* Defaults to {@code "error"}.
* @See #transformForm
*/
@GetMapping("/error")
default String error()
{
return "error"; // the name of the template
}
/**
* @return the name of a template to display log messages when using the test UI for the T-Engine.
* Defaults to {@code "log"}.
* @See #transformForm
*/
@GetMapping("/log")
default String log(Model model)
{
model.addAttribute("title", getTransformerName() + " Log Entries");
Collection<LogEntry> log = LogEntry.getLog();
if (!log.isEmpty())
{
model.addAttribute("log", log);
}
return "log"; // the name of the template
}
/**
* Method used by Kubernetes ready pod probes.
*/
@GetMapping("/ready")
@ResponseBody
default String ready(HttpServletRequest request)
{
return probe(request, false);
}
/**
* Method used by Kubernetes live pod probes.
*/
@GetMapping("/live")
@ResponseBody
default String live(HttpServletRequest request)
{
return probe(request, true);
}
//region [Exception Handlers]
@ExceptionHandler(TypeMismatchException.class)
default void handleParamsTypeMismatch(HttpServletResponse response,
MissingServletRequestParameterException e) throws IOException
{
final String message = format("Request parameter ''{0}'' is of the wrong type", e
.getParameterName());
final int statusCode = BAD_REQUEST.value();
logger.error(message, e);
LogEntry.setStatusCodeAndMessage(statusCode, message);
response.sendError(statusCode, getTransformerName() + " - " + message);
}
@ExceptionHandler(MissingServletRequestParameterException.class)
default void handleMissingParams(HttpServletResponse response,
MissingServletRequestParameterException e) throws IOException
{
final String message = format("Request parameter ''{0}'' is missing", e.getParameterName());
final int statusCode = BAD_REQUEST.value();
logger.error(message, e);
LogEntry.setStatusCodeAndMessage(statusCode, message);
response.sendError(statusCode, getTransformerName() + " - " + message);
}
@ExceptionHandler(TransformException.class)
default void transformExceptionWithMessage(HttpServletResponse response,
TransformException e) throws IOException
{
final String message = e.getMessage();
final int statusCode = e.getStatus().value();
logger.error(message, e);
long time = LogEntry.setStatusCodeAndMessage(statusCode, message);
getProbeTestTransform().recordTransformTime(time);
response.sendError(statusCode, getTransformerName() + " - " + message);
}
//endregion
}

View File

@@ -0,0 +1,67 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transformer;
import static org.alfresco.transformer.fs.FileManager.SOURCE_FILE;
import static org.alfresco.transformer.fs.FileManager.TARGET_FILE;
import static org.alfresco.transformer.fs.FileManager.deleteFile;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import org.alfresco.transformer.logging.LogEntry;
import org.springframework.web.servlet.handler.HandlerInterceptorAdapter;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
*
* TransformInterceptor
* <br/>
* Handles ThreadLocal Log entries for each request.
*/
@Deprecated
public class TransformInterceptor extends HandlerInterceptorAdapter
{
@Override
public boolean preHandle(HttpServletRequest request,
HttpServletResponse response, Object handler)
{
LogEntry.start();
return true;
}
@Override
public void afterCompletion(HttpServletRequest request,
HttpServletResponse response, Object handler, Exception ex)
{
// TargetFile cannot be deleted until completion, otherwise 0 bytes are sent.
deleteFile(request, SOURCE_FILE);
deleteFile(request, TARGET_FILE);
LogEntry.complete();
}
}

View File

@@ -0,0 +1,119 @@
/*
* #%L
* Alfresco Repository
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
*
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transformer;
import static java.nio.charset.StandardCharsets.UTF_8;
import static org.alfresco.transform.config.CoreVersionDecorator.setCoreVersionOnSingleStepTransformers;
import static org.springframework.http.HttpStatus.INTERNAL_SERVER_ERROR;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.Reader;
import javax.annotation.PostConstruct;
import org.alfresco.transform.config.TransformConfig;
import org.alfresco.transform.registry.AbstractTransformRegistry;
import org.alfresco.transform.registry.CombinedTransformConfig;
import org.alfresco.transform.registry.TransformCache;
import org.alfresco.transform.exceptions.TransformException;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.core.io.Resource;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.springframework.core.io.ResourceLoader;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
*
* Used by clients to work out if a transformation is supported based on the engine_config.json.
*/
@Deprecated
public class TransformRegistryImpl extends AbstractTransformRegistry
{
private static final Logger log = LoggerFactory.getLogger(TransformRegistryImpl.class);
@Autowired
ResourceLoader resourceLoader;
@Value("${transform.core.config.location:classpath:engine_config.json}")
private String locationFromProperty;
@Value("${transform.core.version}")
private String coreVersion;
private Resource engineConfig;
@PostConstruct
public void afterPropertiesSet()
{
engineConfig = resourceLoader.getResource(locationFromProperty);
TransformConfig transformConfig = getTransformConfig();
// There is only one TransformConfig in a T-Engine so the following call is fine
CombinedTransformConfig.combineAndRegister(transformConfig, locationFromProperty, "---", this);
}
// Holds the structures used by AbstractTransformRegistry to look up what is supported.
// Unlike other sub classes this class does not extend Data or replace it at run time.
private TransformCache data = new TransformCache();
private ObjectMapper jsonObjectMapper = new ObjectMapper();
TransformConfig getTransformConfig()
{
try (Reader reader = new InputStreamReader(engineConfig.getInputStream(), UTF_8))
{
TransformConfig transformConfig = jsonObjectMapper.readValue(reader, TransformConfig.class);
setCoreVersionOnSingleStepTransformers(transformConfig, coreVersion);
return transformConfig;
}
catch (IOException e)
{
throw new TransformException(INTERNAL_SERVER_ERROR, "Could not read " + locationFromProperty, e);
}
}
@Override
public TransformCache getData()
{
return data;
}
@Override
protected void logError(String msg)
{
log.error(msg);
}
@Override
protected void logWarn(String msg)
{
log.warn(msg);
}
}

View File

@@ -0,0 +1,106 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transformer.clients;
import static org.springframework.http.HttpMethod.POST;
import static org.springframework.http.MediaType.MULTIPART_FORM_DATA;
import java.io.File;
import org.alfresco.transform.exceptions.TransformException;
import org.alfresco.transformer.model.FileRefResponse;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.core.io.FileSystemResource;
import org.springframework.core.io.Resource;
import org.springframework.http.HttpEntity;
import org.springframework.http.HttpHeaders;
import org.springframework.http.ResponseEntity;
import org.springframework.util.LinkedMultiValueMap;
import org.springframework.web.client.HttpClientErrorException;
import org.springframework.web.client.RestTemplate;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
*
* Simple Rest client that call Alfresco Shared File Store
*/
@Deprecated
public class AlfrescoSharedFileStoreClient
{
@Value("${fileStoreUrl}")
private String fileStoreUrl;
@Autowired
private RestTemplate restTemplate;
/**
* Retrieves a file from Shared File Store using given file reference
*
* @param fileRef File reference
* @return ResponseEntity<Resource>
*/
public ResponseEntity<Resource> retrieveFile(String fileRef)
{
try
{
return restTemplate.getForEntity(fileStoreUrl + "/" + fileRef,
org.springframework.core.io.Resource.class);
}
catch (HttpClientErrorException e)
{
throw new TransformException(e.getStatusCode(), e.getMessage(), e);
}
}
/**
* Stores given file in Shared File Store
*
* @param file File to be stored
* @return A FileRefResponse containing detail about file's reference
*/
public FileRefResponse saveFile(File file)
{
try
{
FileSystemResource value = new FileSystemResource(file.getAbsolutePath());
LinkedMultiValueMap<String, Object> map = new LinkedMultiValueMap<>();
map.add("file", value);
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MULTIPART_FORM_DATA);
HttpEntity<LinkedMultiValueMap<String, Object>> requestEntity = new HttpEntity<>(map,
headers);
ResponseEntity<FileRefResponse> responseEntity = restTemplate
.exchange(fileStoreUrl, POST, requestEntity, FileRefResponse.class);
return responseEntity.getBody();
}
catch (HttpClientErrorException e)
{
throw new TransformException(e.getStatusCode(), e.getMessage(), e);
}
}
}

View File

@@ -0,0 +1,93 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transformer.config;
import org.alfresco.transform.messages.TransformRequestValidator;
import org.alfresco.transform.registry.TransformServiceRegistry;
import org.alfresco.transform.common.TransformerDebug;
import org.alfresco.transformer.TransformInterceptor;
import org.alfresco.transformer.TransformRegistryImpl;
import org.alfresco.transformer.clients.AlfrescoSharedFileStoreClient;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.web.client.RestTemplate;
import org.springframework.web.servlet.config.annotation.InterceptorRegistry;
import org.springframework.web.servlet.config.annotation.WebMvcConfigurer;
import static org.alfresco.transform.common.RequestParamMap.ENDPOINT_TRANSFORM;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
*/
@Deprecated
@Configuration
public class WebApplicationConfig implements WebMvcConfigurer
{
@Override
public void addInterceptors(InterceptorRegistry registry)
{
registry
.addInterceptor(transformInterceptor())
.addPathPatterns(ENDPOINT_TRANSFORM, "/live", "/ready");
}
@Bean
public TransformInterceptor transformInterceptor()
{
return new TransformInterceptor();
}
@Bean
public RestTemplate restTemplate()
{
return new RestTemplate();
}
@Bean
public AlfrescoSharedFileStoreClient alfrescoSharedFileStoreClient()
{
return new AlfrescoSharedFileStoreClient();
}
@Bean
public TransformRequestValidator transformRequestValidator()
{
return new TransformRequestValidator();
}
@Bean
public TransformServiceRegistry transformRegistry()
{
return new TransformRegistryImpl();
}
@Bean
public TransformerDebug transformerDebug()
{
return new TransformerDebug().setIsTRouter(false);
}
}

View File

@@ -0,0 +1,90 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transformer.executors;
import static org.alfresco.transformer.executors.RuntimeExec.ExecutionResult;
import static org.springframework.http.HttpStatus.BAD_REQUEST;
import static org.springframework.http.HttpStatus.INTERNAL_SERVER_ERROR;
import java.io.File;
import java.util.Map;
import org.alfresco.transform.exceptions.TransformException;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
*/
@Deprecated
public abstract class AbstractCommandExecutor implements CommandExecutor
{
protected RuntimeExec transformCommand = createTransformCommand();
protected RuntimeExec checkCommand = createCheckCommand();
protected abstract RuntimeExec createTransformCommand();
protected abstract RuntimeExec createCheckCommand();
@Override
public void run(Map<String, String> properties, File targetFile, Long timeout)
{
timeout = timeout != null && timeout > 0 ? timeout : 0;
final ExecutionResult result = transformCommand.execute(properties, timeout);
if (result.getExitValue() != 0 && result.getStdErr() != null && result.getStdErr().length() > 0)
{
throw new TransformException(BAD_REQUEST, "Transformer exit code was not 0: \n" + result.getStdErr());
}
if (!targetFile.exists() || targetFile.length() == 0)
{
throw new TransformException(INTERNAL_SERVER_ERROR, "Transformer failed to create an output file");
}
}
@Override
public String version()
{
if (checkCommand != null)
{
final ExecutionResult result = checkCommand.execute();
if (result.getExitValue() != 0 && result.getStdErr() != null && result.getStdErr().length() > 0)
{
throw new TransformException(INTERNAL_SERVER_ERROR,
"Transformer version check exit code was not 0: \n" + result);
}
final String version = result.getStdOut().trim();
if (version.isEmpty())
{
throw new TransformException(INTERNAL_SERVER_ERROR,
"Transformer version check failed to create any output");
}
return version;
}
return "Version not checked";
}
}

View File

@@ -0,0 +1,74 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transformer.executors;
import org.alfresco.transformer.logging.LogEntry;
import java.io.File;
import java.util.HashMap;
import java.util.Map;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
*
* Basic interface for executing transformations via Shell commands
*
* @author Cezar Leahu
*/
@Deprecated
public interface CommandExecutor extends Transformer
{
void run(Map<String, String> properties, File targetFile, Long timeout);
String version();
default void run(String options, File sourceFile, File targetFile,
Long timeout)
{
LogEntry.setOptions(options);
Map<String, String> properties = new HashMap<>();
properties.put("options", options);
properties.put("source", sourceFile.getAbsolutePath());
properties.put("target", targetFile.getAbsolutePath());
run(properties, targetFile, timeout);
}
default void run(String options, File sourceFile, String pageRange, File targetFile,
Long timeout)
{
LogEntry.setOptions(pageRange + (pageRange.isEmpty() ? "" : " ") + options);
Map<String, String> properties = new HashMap<>();
properties.put("options", options);
properties.put("source", sourceFile.getAbsolutePath() + pageRange);
properties.put("target", targetFile.getAbsolutePath());
run(properties, targetFile, timeout);
}
}

View File

@@ -0,0 +1,363 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transformer.executors;
import static java.util.Collections.singletonList;
import java.io.Serializable;
import java.util.ArrayList;
import java.util.List;
import java.util.Objects;
import java.util.StringTokenizer;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
*
* DUPLICATED FROM *alfresco-core*.
*
* This class is used to tokenize strings used as parameters for {@link RuntimeExec} objects.
* Examples of such strings are as follows (ImageMagick-like parameters):
* <ul>
* <li><tt>-font Helvetica -pointsize 50</tt></li>
* <li><tt>-font Helvetica -pointsize 50 -draw "circle 100,100 150,150"</tt></li>
* <li><tt>-font Helvetica -pointsize 50 -draw "gravity south fill black text 0,12 'CopyRight'"</tt></li>
* </ul>
* The first is the simple case which would be parsed into Strings as follows:
* <tt>"-font", "Helvetica", "-pointsize", "50"</tt>
* <p/>
* The second is more complex in that it includes a quoted parameter, which would be parsed as a single String:
* <tt>"-font", "Helvetica", "-pointsize", "50", "circle 100,100 150,150"</tt>
* Note however that the quotation characters will be stripped from the token.
* <p/>
* The third shows an example with embedded quotation marks, which would parse to:
* <tt>"-font", "Helvetica", "-pointsize", "50", "gravity south fill black text 0,12 'CopyRight'"</tt>
* In this case, the embedded quotation marks (which must be different from those surrounding the parameter)
* are preserved in the extracted token.
* <p/>
* The class does not understand escaped quotes such as <tt>p1 p2 "a b c \"hello\" d" p4</tt>
*
* @author Neil Mc Erlean
* @since 3.4.2
*/
@Deprecated
public class ExecParameterTokenizer
{
/**
* The string to be tokenized.
*/
private final String str;
/**
* The list of tokens, which will take account of quoted sections.
*/
private List<String> tokens;
public ExecParameterTokenizer(String str)
{
this.str = str;
}
/**
* This method returns the tokens in a parameter string.
* Any tokens not contained within single or double quotes will be tokenized in the normal
* way i.e. by using whitespace separators and the standard StringTokenizer algorithm.
* Any tokens which are contained within single or double quotes will be returned as single
* String instances and will have their quote marks removed.
* <p/>
* See above for examples.
*
* @throws NullPointerException if the string to be tokenized was null.
*/
public List<String> getAllTokens()
{
if (this.str == null)
{
throw new NullPointerException("Illegal null string cannot be tokenized.");
}
if (tokens == null)
{
tokens = new ArrayList<>();
// Preserve original behaviour from RuntimeExec.
if (str.indexOf('\'') == -1 && str.indexOf('"') == -1)
{
// Contains no quotes.
for (StringTokenizer standardTokenizer = new StringTokenizer(
str); standardTokenizer.hasMoreTokens(); )
{
tokens.add(standardTokenizer.nextToken());
}
}
else
{
// There are either single or double quotes or both.
// So we need to identify the quoted regions within the string.
List<Pair<Integer, Integer>> quotedRegions = new ArrayList<>();
for (Pair<Integer, Integer> next = identifyNextQuotedRegion(str, 0); next != null; )
{
quotedRegions.add(next);
next = identifyNextQuotedRegion(str, next.getSecond() + 1);
}
// Now we've got a List of index pairs identifying the quoted regions.
// We need to get substrings of quoted and unquoted blocks, whilst maintaining order.
List<Substring> substrings = getSubstrings(str, quotedRegions);
for (Substring r : substrings)
{
tokens.addAll(r.getTokens());
}
}
}
return this.tokens;
}
/**
* The substrings will be a list of quoted and unquoted substrings.
* The unquoted ones need to be further tokenized in the normal way.
* The quoted ones must not be tokenized, but need their quotes stripped off.
*/
private List<Substring> getSubstrings(String str,
List<Pair<Integer, Integer>> quotedRegionIndices)
{
List<Substring> result = new ArrayList<>();
int cursorPosition = 0;
for (Pair<Integer, Integer> nextQuotedRegionIndices : quotedRegionIndices)
{
if (cursorPosition < nextQuotedRegionIndices.getFirst())
{
int startIndexOfNextQuotedRegion = nextQuotedRegionIndices.getFirst() - 1;
result.add(new UnquotedSubstring(
str.substring(cursorPosition, startIndexOfNextQuotedRegion)));
}
result.add(new QuotedSubstring(str.substring(nextQuotedRegionIndices.getFirst(),
nextQuotedRegionIndices.getSecond())));
cursorPosition = nextQuotedRegionIndices.getSecond();
}
// We've processed all the quoted regions, but there may be a final unquoted region
if (cursorPosition < str.length() - 1)
{
result.add(new UnquotedSubstring(str.substring(cursorPosition, str.length() - 1)));
}
return result;
}
private Pair<Integer, Integer> identifyNextQuotedRegion(String str, int startingIndex)
{
int indexOfNextSingleQuote = str.indexOf('\'', startingIndex);
int indexOfNextDoubleQuote = str.indexOf('"', startingIndex);
if (indexOfNextSingleQuote == -1 && indexOfNextDoubleQuote == -1)
{
// If there are no more quoted regions
return null;
}
else if (indexOfNextSingleQuote > -1 && indexOfNextDoubleQuote > -1)
{
// If there are both single and double quotes in the remainder of the string
// Then select the closest quote.
int indexOfNextQuote = Math.min(indexOfNextSingleQuote, indexOfNextDoubleQuote);
char quoteChar = str.charAt(indexOfNextQuote);
return findIndexOfClosingQuote(str, indexOfNextQuote, quoteChar);
}
else
{
// Only one of the quote characters is present.
int indexOfNextQuote = Math.max(indexOfNextSingleQuote, indexOfNextDoubleQuote);
char quoteChar = str.charAt(indexOfNextQuote);
return findIndexOfClosingQuote(str, indexOfNextQuote, quoteChar);
}
}
private Pair<Integer, Integer> findIndexOfClosingQuote(String str, int indexOfStartingQuote,
char quoteChar)
{
// So we know which type of quote char we're dealing with. Either ' or ".
// Now we need to find the closing quote.
int indexAfterClosingQuote = str.indexOf(quoteChar,
indexOfStartingQuote + 1) + 1; // + 1 to search after opening quote. + 1 to give result including closing quote.
if (indexAfterClosingQuote == 0) // -1 + 1
{
// If no closing quote.
throw new IllegalArgumentException("No closing " + quoteChar + "quote in" + str);
}
return new Pair<>(indexOfStartingQuote, indexAfterClosingQuote);
}
/**
* Utility interface for a substring in a parameter string.
*/
public interface Substring
{
/**
* Gets all the tokens in a parameter string.
*/
List<String> getTokens();
}
/**
* A substring that is not surrounded by (single or double) quotes.
*/
public class UnquotedSubstring implements Substring
{
private final String regionString;
public UnquotedSubstring(String str)
{
this.regionString = str;
}
public List<String> getTokens()
{
StringTokenizer t = new StringTokenizer(regionString);
List<String> result = new ArrayList<>();
while (t.hasMoreTokens())
{
result.add(t.nextToken());
}
return result;
}
public String toString()
{
return UnquotedSubstring.class.getSimpleName() + ": '" + regionString + '\'';
}
}
/**
* A substring that is surrounded by (single or double) quotes.
*/
public class QuotedSubstring implements Substring
{
private final String regionString;
public QuotedSubstring(String str)
{
this.regionString = str;
}
public List<String> getTokens()
{
String stringWithoutQuotes = regionString.substring(1, regionString.length() - 1);
return singletonList(stringWithoutQuotes);
}
public String toString()
{
return QuotedSubstring.class.getSimpleName() + ": '" + regionString + '\'';
}
}
public static final class Pair<F, S> implements Serializable
{
private static final long serialVersionUID = -7406248421185630612L;
/**
* The first member of the pair.
*/
private F first;
/**
* The second member of the pair.
*/
private S second;
/**
* Make a new one.
*
* @param first The first member.
* @param second The second member.
*/
public Pair(F first, S second)
{
this.first = first;
this.second = second;
}
/**
* Get the first member of the tuple.
*
* @return The first member.
*/
public final F getFirst()
{
return first;
}
/**
* Get the second member of the tuple.
*
* @return The second member.
*/
public final S getSecond()
{
return second;
}
public final void setFirst(F first)
{
this.first = first;
}
public final void setSecond(S second)
{
this.second = second;
}
@Override public boolean equals(Object o)
{
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
Pair<?, ?> pair = (Pair<?, ?>) o;
return Objects.equals(first, pair.first) &&
Objects.equals(second, pair.second);
}
@Override public int hashCode()
{
return Objects.hash(first, second);
}
@Override
public String toString()
{
return "(" + first + ", " + second + ")";
}
}
}

View File

@@ -0,0 +1,43 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transformer.executors;
import java.io.File;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
*
* Basic interface for executing transformations inside Java/JVM.
*
* @author Cezar Leahu
* @author adavis
*/
@Deprecated
public interface JavaExecutor extends Transformer
{
void call(File sourceFile, File targetFile, String... args) throws Exception;
}

View File

@@ -0,0 +1,989 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transformer.executors;
import static java.util.Collections.emptyMap;
import java.io.BufferedInputStream;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.Charset;
import java.nio.charset.UnsupportedCharsetException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashMap;
import java.util.HashSet;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.StringTokenizer;
import java.util.Timer;
import java.util.TimerTask;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
*
* DUPLICATED FROM *alfresco-core*.
*
* This acts as a session similar to the <code>java.lang.Process</code>, but
* logs the system standard and error streams.
* <p>
* The bean can be configured to execute a command directly, or be given a map
* of commands keyed by the <i>os.name</i> Java system property. In this map,
* the default key that is used when no match is found is the
* <b>{@link #KEY_OS_DEFAULT *}</b> key.
* <p>
* Use the {@link #setProcessDirectory(String) processDirectory} property to change the default location
* from which the command executes. The process's environment can be configured using the
* {@link #setProcessProperties(Map) processProperties} property.
* <p>
* Commands may use placeholders, e.g.
* <pre><code>
* find
* -name
* ${filename}
* </code></pre>
* The <b>filename</b> property will be substituted for any supplied value prior to
* each execution of the command. Currently, no checks are made to get or check the
* properties contained within the command string. It is up to the client code to
* dynamically extract the properties required if the required properties are not
* known up front.
* <p>
* Sometimes, a variable may contain several arguments. . In this case, the arguments
* need to be tokenized using a standard <tt>StringTokenizer</tt>. To force tokenization
* of a value, use:
* <pre><code>
* SPLIT:${userArgs}
* </code></pre>
* You should not use this just to split up arguments that are known to require tokenization
* up front. The <b>SPLIT:</b> directive works for the entire argument and will not do anything
* if it is not at the beginning of the argument. Do not use <b>SPLIT:</b> to break up arguments
* that are fixed, so avoid doing this:
* <pre><code>
* SPLIT:ls -lih
* </code></pre>
* Instead, break the command up explicitly:
* <pre><code>
* ls
* -lih
* </code></pre>
*
* Tokenization of quoted parameter values is handled by ExecParameterTokenizer, which
* describes the support in more detail.
*
* @author Derek Hulley
*/
@Deprecated
public class RuntimeExec
{
private static final Logger logger = LoggerFactory.getLogger(RuntimeExec.class);
/**
* the key to use when specifying a command for any other OS: <b>*</b>
*/
private static final String KEY_OS_DEFAULT = "*";
private static final String KEY_OS_NAME = "os.name";
private static final int BUFFER_SIZE = 1024;
private static final String VAR_OPEN = "${";
private static final String VAR_CLOSE = "}";
private static final String DIRECTIVE_SPLIT = "SPLIT:";
private String[] command;
private Charset charset = Charset.defaultCharset();
private boolean waitForCompletion = true;
private Map<String, String> defaultProperties = emptyMap();
private String[] processProperties;
private File processDirectory;
private final Set<Integer> errCodes;
private final Timer timer = new Timer(true);
/**
* Default constructor. Initialize this instance by setting individual properties.
*/
public RuntimeExec()
{
// set default error codes
errCodes = new HashSet<>(2);
errCodes.add(1);
errCodes.add(2);
}
public String toString()
{
final StringBuilder sb = new StringBuilder(256);
sb.append("RuntimeExec:\n").append(" command: ");
if (command == null)
{
// command is 'null', so there's nothing to toString
sb.append("'null'\n");
}
else
{
for (String cmdStr : command)
{
sb.append(cmdStr).append(" ");
}
sb.append("\n");
}
sb.append(" env props: ").append(Arrays.toString(processProperties)).append("\n")
.append(" dir: ").append(processDirectory).append("\n")
.append(" os: ").append(System.getProperty(KEY_OS_NAME)).append("\n");
return sb.toString();
}
/**
* Set the command to execute regardless of operating system
*
* @param command an array of strings representing the command (first entry) and arguments
* @since 3.0
*/
public void setCommand(String[] command)
{
this.command = command;
}
/**
* Sets the assumed charset of OUT and ERR streams generated by the executed command.
* This defaults to the system default charset: {@link Charset#defaultCharset()}.
*
* @param charsetCode a supported character set code
* @throws UnsupportedCharsetException if the characterset code is not recognised by Java
*/
public void setCharset(String charsetCode)
{
this.charset = Charset.forName(charsetCode);
}
/**
* Set whether to wait for completion of the command or not. If there is no wait for completion,
* then the return value of <i>out</i> and <i>err</i> buffers cannot be relied upon as the
* command may still be in progress. Failure is therefore not possible unless the calling thread
* waits for execution.
*
* @param waitForCompletion <tt>true</tt> (default) is to wait for the command to exit,
* or <tt>false</tt> to just return an exit code of 0 and whatever
* output is available at that point.
* @since 2.1
*/
public void setWaitForCompletion(boolean waitForCompletion)
{
this.waitForCompletion = waitForCompletion;
}
/**
* Supply a choice of commands to execute based on a mapping from the <i>os.name</i> system
* property to the command to execute. The {@link #KEY_OS_DEFAULT *} key can be used
* to get a command where there is not direct match to the operating system key.
* <p>
* Each command is an array of strings, the first of which represents the command and all subsequent
* entries in the array represent the arguments. All elements of the array will be checked for
* the presence of any substitution parameters (e.g. '{dir}'). The parameters can be set using the
* {@link #setDefaultProperties(Map) defaults} or by passing the substitution values into the
* {@link #execute(Map)} command.
* <p>
* If parameters passed may be multiple arguments, or if the values provided in the map are themselves
* collections of arguments (not recommended), then prefix the value with <b>SPLIT:</b> to ensure that
* the value is tokenized before being passed to the command. Any values that are not split, will be
* passed to the command as single arguments. For example:<br>
* '<b>SPLIT: dir . ..</b>' becomes '<b>dir</b>', '<b>.</b>' and '<b>..</b>'.<br>
* '<b>SPLIT: dir ${path}</b>' (if path is '<b>. ..</b>') becomes '<b>dir</b>', '<b>.</b>' and '<b>..</b>'.<br>
* The splitting occurs post-subtitution. Where the arguments are known, it is advisable to avoid
* <b>SPLIT:</b>.
*
* @param commandsByOS a map of command string arrays, keyed by operating system names
* @see #setDefaultProperties(Map)
* @since 3.0
*/
public void setCommandsAndArguments(Map<String, String[]> commandsByOS)
{
// get the current OS
String serverOs = System.getProperty(KEY_OS_NAME);
// attempt to find a match
String[] command = commandsByOS.get(serverOs);
if (command == null)
{
// go through the commands keys, looking for one that matches by regular expression matching
for (String osName : commandsByOS.keySet())
{
// Ignore * options. It is dealt with later.
if (osName.equals(KEY_OS_DEFAULT))
{
continue;
}
// Do regex match
if (serverOs.matches(osName))
{
command = commandsByOS.get(osName);
break;
}
}
// if there is still no command, then check for the wildcard
if (command == null)
{
command = commandsByOS.get(KEY_OS_DEFAULT);
}
}
// check
if (command == null)
{
throw new RuntimeException(
"No command found for OS " + serverOs + " or '" + KEY_OS_DEFAULT + "': \n" +
" commands: " + commandsByOS);
}
this.command = command;
}
/**
* Supply a choice of commands to execute based on a mapping from the <i>os.name</i> system
* property to the command to execute. The {@link #KEY_OS_DEFAULT *} key can be used
* to get a command where there is not direct match to the operating system key.
*
* @param commandsByOS a map of command string keyed by operating system names
* @deprecated Use {@link #setCommandsAndArguments(Map)}
*/
public void setCommandMap(Map<String, String> commandsByOS)
{
// This is deprecated, so issue a warning
logger.warn(
"The bean RuntimeExec property 'commandMap' has been deprecated;" +
" use 'commandsAndArguments' instead. See https://issues.alfresco.com/jira/browse/ETHREEOH-579.");
Map<String, String[]> fixed = new LinkedHashMap<>();
for (Map.Entry<String, String> entry : commandsByOS.entrySet())
{
String os = entry.getKey();
String unparsedCmd = entry.getValue();
StringTokenizer tokenizer = new StringTokenizer(unparsedCmd);
String[] cmd = new String[tokenizer.countTokens()];
for (int i = 0; i < cmd.length; i++)
{
cmd[i] = tokenizer.nextToken();
}
fixed.put(os, cmd);
}
setCommandsAndArguments(fixed);
}
/**
* Set the default command-line properties to use when executing the command.
* These are properties that substitute variables defined in the command string itself.
* Properties supplied during execution will overwrite the default properties.
* <p>
* <code>null</code> properties will be treated as an empty string for substitution
* purposes.
*
* @param defaultProperties property values
*/
public void setDefaultProperties(Map<String, String> defaultProperties)
{
this.defaultProperties = defaultProperties;
}
/**
* Set additional runtime properties (environment properties) that will used
* by the executing process.
* <p>
* Any keys or properties that start and end with <b>${...}</b> will be removed on the assumption
* that these are unset properties. <tt>null</tt> values are translated to empty strings.
* All keys and values are trimmed of leading and trailing whitespace.
*
* @param processProperties Runtime process properties
* @see Runtime#exec(String, String[], java.io.File)
*/
public void setProcessProperties(Map<String, String> processProperties)
{
ArrayList<String> processPropList = new ArrayList<>(processProperties.size());
boolean hasPath = false;
String systemPath = System.getenv("PATH");
for (Map.Entry<String, String> entry : processProperties.entrySet())
{
String key = entry.getKey();
String value = entry.getValue();
if (key == null)
{
continue;
}
if (value == null)
{
value = "";
}
key = key.trim();
value = value.trim();
if (key.startsWith(VAR_OPEN) && key.endsWith(VAR_CLOSE))
{
continue;
}
if (value.startsWith(VAR_OPEN) && value.endsWith(VAR_CLOSE))
{
continue;
}
// If a path is specified, prepend it to the existing path
if (key.equals("PATH"))
{
if (systemPath != null && systemPath.length() > 0)
{
processPropList.add(key + "=" + value + File.pathSeparator + systemPath);
}
else
{
processPropList.add(key + "=" + value);
}
hasPath = true;
}
else
{
processPropList.add(key + "=" + value);
}
}
// If a path was not specified, inherit the current one
if (!hasPath && systemPath != null && systemPath.length() > 0)
{
processPropList.add("PATH=" + systemPath);
}
this.processProperties = processPropList.toArray(new String[0]);
}
/**
* Adds a property to existed processProperties.
* Property should not be null or empty.
* If property with the same value already exists then no change is made.
* If property exists with a different value then old value is replaced with the new one.
*
* @param name - property name
* @param value - property value
*/
public void setProcessProperty(String name, String value)
{
boolean set = false;
if (name == null || value == null)
{
return;
}
name = name.trim();
value = value.trim();
if (name.isEmpty() || value.isEmpty())
{
return;
}
String property = name + "=" + value;
for (String prop : this.processProperties)
{
if (prop.equals(property))
{
set = true;
break;
}
if (prop.startsWith(name))
{
String oldValue = prop.split("=")[1];
prop.replace(oldValue, value);
set = true;
}
}
if (!set)
{
String[] existedProperties = this.processProperties;
int epl = existedProperties.length;
String[] newProperties = Arrays.copyOf(existedProperties, epl + 1);
newProperties[epl] = property;
this.processProperties = newProperties;
}
}
/**
* Set the runtime location from which the command is executed.
* <p>
* If the value is an unsubsititued variable (<b>${...}</b>) then it is ignored.
* If the location is not visible at the time of setting, a warning is issued only.
*
* @param processDirectory the runtime location from which to execute the command
*/
public void setProcessDirectory(String processDirectory)
{
if (processDirectory.startsWith(VAR_OPEN) && processDirectory.endsWith(VAR_CLOSE))
{
this.processDirectory = null;
}
else
{
this.processDirectory = new File(processDirectory);
if (!this.processDirectory.exists())
{
logger.warn(
"The runtime process directory is not visible when setting property " +
"'processDirectory': \n{}", this);
}
}
}
/**
* A comma or space separated list of values that, if returned by the executed command,
* indicate an error value. This defaults to <b>"1, 2"</b>.
*
* @param errCodesStr the error codes for the execution
*/
public void setErrorCodes(String errCodesStr)
{
errCodes.clear();
StringTokenizer tokenizer = new StringTokenizer(errCodesStr, " ,");
while (tokenizer.hasMoreElements())
{
String errCodeStr = tokenizer.nextToken();
// attempt to convert it to an integer
try
{
int errCode = Integer.parseInt(errCodeStr);
this.errCodes.add(errCode);
}
catch (NumberFormatException e)
{
throw new RuntimeException(
"Property 'errorCodes' must be comma-separated list of integers: " + errCodesStr);
}
}
}
/**
* Executes the command using the default properties
*
* @see #execute(Map)
*/
public ExecutionResult execute()
{
return execute(defaultProperties);
}
/**
* Executes the statement that this instance was constructed with.
*
* @param properties the properties that the command might be executed with.
* <code>null</code> properties will be treated as an empty string for substitution
* purposes.
* @return Returns the full execution results
*/
public ExecutionResult execute(Map<String, String> properties)
{
return execute(properties, -1);
}
/**
* Executes the statement that this instance was constructed with an optional
* timeout after which the command is asked to
*
* @param properties the properties that the command might be executed with.
* <code>null</code> properties will be treated as an empty string for substitution
* purposes.
* @param timeoutMs a timeout after which {@link Process#destroy()} is called.
* ignored if less than or equal to zero. Note this method does not guarantee
* to terminate the process (it is not a kill -9).
* @return Returns the full execution results
*/
public ExecutionResult execute(Map<String, String> properties, final long timeoutMs)
{
int defaultFailureExitValue = errCodes.size() > 0 ? ((Integer) errCodes.toArray()[0]) : 1;
// check that the command has been set
if (command == null)
{
throw new RuntimeException("Runtime command has not been set: \n" + this);
}
// create the properties
Runtime runtime = Runtime.getRuntime();
Process process;
String[] commandToExecute = null;
try
{
// execute the command with full property replacement
commandToExecute = getCommand(properties);
final Process thisProcess = runtime.exec(commandToExecute, processProperties,
processDirectory);
process = thisProcess;
if (timeoutMs > 0)
{
final String[] command = commandToExecute;
timer.schedule(new TimerTask()
{
@Override
public void run()
{
// Only try to kill the process if it is still running
try
{
thisProcess.exitValue();
}
catch (IllegalThreadStateException stillRunning)
{
logger.debug(
"Process has taken too long ({} seconds). Killing process {}",
timeoutMs / 1000, Arrays.deepToString(command));
}
}
}, timeoutMs);
}
}
catch (IOException e)
{
// The process could not be executed here, so just drop out with an appropriate error state
String execOut = "";
String execErr = e.getMessage();
ExecutionResult result = new ExecutionResult(null, commandToExecute, errCodes,
defaultFailureExitValue, execOut, execErr);
logFullEnvironmentDump(result);
return result;
}
// create the stream gobblers
InputStreamReaderThread stdOutGobbler = new InputStreamReaderThread(
process.getInputStream(), charset);
InputStreamReaderThread stdErrGobbler = new InputStreamReaderThread(
process.getErrorStream(), charset);
// start gobbling
stdOutGobbler.start();
stdErrGobbler.start();
// wait for the process to finish
int exitValue = 0;
try
{
if (waitForCompletion)
{
exitValue = process.waitFor();
}
}
catch (InterruptedException e)
{
// process was interrupted - generate an error message
stdErrGobbler.addToBuffer(e.toString());
exitValue = defaultFailureExitValue;
}
if (waitForCompletion)
{
// ensure that the stream gobblers get to finish
stdOutGobbler.waitForCompletion();
stdErrGobbler.waitForCompletion();
}
// get the stream values
String execOut = stdOutGobbler.getBuffer();
String execErr = stdErrGobbler.getBuffer();
// construct the return value
ExecutionResult result = new ExecutionResult(process, commandToExecute, errCodes, exitValue,
execOut, execErr);
// done
logFullEnvironmentDump(result);
return result;
}
/**
* Dump the full environment in debug mode
*/
private void logFullEnvironmentDump(ExecutionResult result)
{
if (logger.isTraceEnabled())
{
StringBuilder sb = new StringBuilder();
sb.append(result);
// Environment variables modified by Alfresco
if (processProperties != null && processProperties.length > 0)
{
sb.append("\n modified environment: ");
for (String property : processProperties)
{
sb.append("\n ");
sb.append(property);
}
}
// Dump the full environment
sb.append("\n existing environment: ");
Map<String, String> envVariables = System.getenv();
for (Map.Entry<String, String> entry : envVariables.entrySet())
{
String name = entry.getKey();
String value = entry.getValue();
sb.append("\n ");
sb.append(name).append("=").append(value);
}
logger.trace(sb.toString());
}
logger.debug("Result: " + result.toString());
// close output stream (connected to input stream of native subprocess)
}
/**
* @return Returns the command that will be executed if no additional properties
* were to be supplied
*/
public String[] getCommand()
{
return getCommand(defaultProperties);
}
/**
* Get the command that will be executed post substitution.
* <p>
* <code>null</code> properties will be treated as an empty string for substitution
* purposes.
*
* @param properties the properties that the command might be executed with
* @return Returns the command that will be executed should the additional properties
* be supplied
*/
public String[] getCommand(Map<String, String> properties)
{
Map<String, String> execProperties;
if (properties == defaultProperties)
{
// we are just using the default properties
execProperties = defaultProperties;
}
else
{
execProperties = new HashMap<>(defaultProperties);
// overlay the supplied properties
execProperties.putAll(properties);
}
// Perform the substitution for each element of the command
ArrayList<String> adjustedCommandElements = new ArrayList<>(20);
for (String s : command)
{
StringBuilder sb = new StringBuilder(s);
for (Map.Entry<String, String> entry : execProperties.entrySet())
{
String key = entry.getKey();
String value = entry.getValue();
// ignore null
if (value == null)
{
value = "";
}
// progressively replace the property in the command
key = (VAR_OPEN + key + VAR_CLOSE);
int index = sb.indexOf(key);
while (index > -1)
{
// replace
sb.replace(index, index + key.length(), value);
// get the next one
index = sb.indexOf(key, index + 1);
}
}
String adjustedValue = sb.toString();
// Now SPLIT: it
if (adjustedValue.startsWith(DIRECTIVE_SPLIT))
{
String unsplitAdjustedValue = sb.substring(DIRECTIVE_SPLIT.length());
// There may be quoted arguments here (see ALF-7482)
ExecParameterTokenizer quoteAwareTokenizer = new ExecParameterTokenizer(
unsplitAdjustedValue);
List<String> tokens = quoteAwareTokenizer.getAllTokens();
adjustedCommandElements.addAll(tokens);
}
else
{
adjustedCommandElements.add(adjustedValue);
}
}
// done
return adjustedCommandElements.toArray(new String[0]);
}
/**
* Object to carry the results of an execution to the caller.
*
* @author Derek Hulley
*/
public static class ExecutionResult
{
private final Process process;
private final String[] command;
private final Set<Integer> errCodes;
private final int exitValue;
private final String stdOut;
private final String stdErr;
/**
* @param process the process attached to Java - <tt>null</tt> is allowed
*/
private ExecutionResult(
final Process process,
final String[] command,
final Set<Integer> errCodes,
final int exitValue,
final String stdOut,
final String stdErr)
{
this.process = process;
this.command = command;
this.errCodes = errCodes;
this.exitValue = exitValue;
this.stdOut = stdOut;
this.stdErr = stdErr;
}
@Override
public String toString()
{
String out = stdOut.length() > 250 ? stdOut.substring(0, 250) : stdOut;
String err = stdErr.length() > 250 ? stdErr.substring(0, 250) : stdErr;
StringBuilder sb = new StringBuilder(128);
sb.append("Execution result: \n")
.append(" os: ").append(System.getProperty(KEY_OS_NAME)).append("\n")
.append(" command: ");
appendCommand(sb, command).append("\n")
.append(" succeeded: ").append(getSuccess()).append("\n")
.append(" exit code: ").append(exitValue).append("\n")
.append(" out: ").append(out).append("\n")
.append(" err: ").append(err);
return sb.toString();
}
/**
* Appends the command in a form that make running from the command line simpler.
* It is not a real attempt at making a command given all the operating system
* and shell options, but makes copy, paste and edit a bit simpler.
*/
private StringBuilder appendCommand(StringBuilder sb, String[] command)
{
boolean arg = false;
for (String element : command)
{
if (element == null)
{
continue;
}
if (arg)
{
sb.append(' ');
}
else
{
arg = true;
}
boolean escape = element.indexOf(' ') != -1 || element.indexOf('>') != -1;
if (escape)
{
sb.append("\"");
}
sb.append(element);
if (escape)
{
sb.append("\"");
}
}
return sb;
}
/**
* A helper method to force a kill of the process that generated this result. This is
* useful in cases where the process started is not expected to exit, or doesn't exit
* quickly. If the {@linkplain RuntimeExec#setWaitForCompletion(boolean) "wait for completion"}
* flag is <tt>false</tt> then the process may still be running when this result is returned.
*
* @return <tt>true</tt> if the process was killed, otherwise <tt>false</tt>
*/
public boolean killProcess()
{
if (process == null)
{
return true;
}
try
{
process.destroy();
return true;
}
catch (Throwable e)
{
logger.warn(e.getMessage());
return false;
}
}
/**
* @param exitValue the command exit value
* @return Returns true if the code is a listed failure code
* @see #setErrorCodes(String)
*/
private boolean isFailureCode(int exitValue)
{
return errCodes.contains(exitValue);
}
/**
* @return Returns true if the command was deemed to be successful according to the
* failure codes returned by the execution.
*/
public boolean getSuccess()
{
return !isFailureCode(exitValue);
}
public int getExitValue()
{
return exitValue;
}
public String getStdOut()
{
return stdOut;
}
public String getStdErr()
{
return stdErr;
}
}
/**
* Gobbles an <code>InputStream</code> and writes it into a
* <code>StringBuffer</code>
* <p>
* The reading of the input stream is buffered.
*/
public static class InputStreamReaderThread extends Thread
{
private final InputStream is;
private final Charset charset;
private final StringBuffer buffer; // we require the synchronization
private boolean completed;
/**
* @param is an input stream to read - it will be wrapped in a buffer
* for reading
*/
public InputStreamReaderThread(InputStream is, Charset charset)
{
super();
setDaemon(true); // must not hold up the VM if it is terminating
this.is = is;
this.charset = charset;
this.buffer = new StringBuffer(BUFFER_SIZE);
this.completed = false;
}
public synchronized void run()
{
completed = false;
byte[] bytes = new byte[BUFFER_SIZE];
try (InputStream tempIs = new BufferedInputStream(is, BUFFER_SIZE))
{
int count = -2;
while (count != -1)
{
// do we have something previously read?
if (count > 0)
{
String toWrite = new String(bytes, 0, count, charset.name());
buffer.append(toWrite);
}
// read the next set of bytes
count = tempIs.read(bytes);
}
// done
}
catch (IOException e)
{
throw new RuntimeException("Unable to read stream", e);
}
finally
{
// The thread has finished consuming the stream
completed = true;
// Notify waiters
this.notifyAll(); // Note: Method is synchronized
}
}
/**
* Waits for the run to complete.
* <p>
* <b>Remember to <code>start</code> the thread first
*/
public synchronized void waitForCompletion()
{
while (!completed)
{
try
{
// release our lock and wait a bit
this.wait(1000L); // 200 ms
}
catch (InterruptedException ignore)
{
}
}
}
/**
* @param msg the message to add to the buffer
*/
public void addToBuffer(String msg)
{
buffer.append(msg);
}
public boolean isComplete()
{
return completed;
}
/**
* @return Returns the current state of the buffer
*/
public String getBuffer()
{
return buffer.toString();
}
}
}

View File

@@ -0,0 +1,133 @@
package org.alfresco.transformer.executors;
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
import org.alfresco.transform.exceptions.TransformException;
import java.io.File;
import java.util.Map;
import static org.alfresco.transform.common.Mimetype.MIMETYPE_METADATA_EMBED;
import static org.alfresco.transform.common.Mimetype.MIMETYPE_METADATA_EXTRACT;
import static org.alfresco.transformer.util.RequestParamMap.TRANSFORM_NAME_PARAMETER;
import static org.springframework.http.HttpStatus.BAD_REQUEST;
import static org.springframework.http.HttpStatus.INTERNAL_SERVER_ERROR;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
*
* Basic interface for executing transforms and metadata extract or embed actions.
*
* @author adavis
*/
@Deprecated
public interface Transformer
{
/**
* @return A unique transformer id,
*
*/
String getTransformerId();
default void transform(String sourceMimetype, String targetMimetype, Map<String, String> transformOptions,
File sourceFile, File targetFile) throws TransformException {
final String transformName = transformOptions.remove(TRANSFORM_NAME_PARAMETER);
transformExtractOrEmbed(transformName, sourceMimetype, targetMimetype, transformOptions, sourceFile, targetFile);
}
default void transformExtractOrEmbed(String transformName, String sourceMimetype, String targetMimetype,
Map<String, String> transformOptions,
File sourceFile, File targetFile) throws TransformException
{
try
{
if (MIMETYPE_METADATA_EXTRACT.equals(targetMimetype))
{
extractMetadata(transformName, sourceMimetype, targetMimetype, transformOptions, sourceFile, targetFile);
}
else if (MIMETYPE_METADATA_EMBED.equals(targetMimetype))
{
embedMetadata(transformName, sourceMimetype, targetMimetype, transformOptions, sourceFile, targetFile);
}
else
{
transform(transformName, sourceMimetype, targetMimetype, transformOptions, sourceFile, targetFile);
}
}
catch (TransformException e)
{
throw e;
}
catch (IllegalArgumentException e)
{
throw new TransformException(BAD_REQUEST, getMessage(e), e);
}
catch (Exception e)
{
throw new TransformException(INTERNAL_SERVER_ERROR, getMessage(e), e);
}
if (!targetFile.exists())
{
throw new TransformException(INTERNAL_SERVER_ERROR,
"Transformer failed to create an output file. Target file does not exist.");
}
if (sourceFile.length() > 0 && targetFile.length() == 0)
{
throw new TransformException(INTERNAL_SERVER_ERROR,
"Transformer failed to create an output file. Target file is empty but source file was not empty.");
}
}
private static String getMessage(Exception e)
{
return e.getMessage() == null ? e.getClass().getSimpleName() : e.getMessage();
}
default void transform(String transformName, String sourceMimetype, String targetMimetype,
Map<String, String> transformOptions,
File sourceFile, File targetFile) throws Exception
{
}
default void extractMetadata(String transformName, String sourceMimetype, String targetMimetype,
Map<String, String> transformOptions,
File sourceFile, File targetFile) throws Exception
{
}
/**
* @deprecated The content repository has no non test embed metadata implementations.
* This code exists in case there are custom implementations, that need to be converted to T-Engines.
* It is simply a copy and paste from the content repository and has received limited testing.
*/
default void embedMetadata(String transformName, String sourceMimetype, String targetMimetype,
Map<String, String> transformOptions,
File sourceFile, File targetFile) throws Exception
{
}
}

View File

@@ -0,0 +1,282 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transformer.fs;
import static org.springframework.http.HttpHeaders.CONTENT_DISPOSITION;
import static org.springframework.http.HttpStatus.BAD_REQUEST;
import static org.springframework.http.HttpStatus.INSUFFICIENT_STORAGE;
import static org.springframework.http.HttpStatus.INTERNAL_SERVER_ERROR;
import static org.springframework.util.StringUtils.getFilename;
import static org.springframework.util.StringUtils.getFilenameExtension;
import java.io.File;
import java.io.IOException;
import java.net.MalformedURLException;
import java.nio.file.Files;
import java.nio.file.StandardCopyOption;
import java.util.Arrays;
import javax.servlet.http.HttpServletRequest;
import org.alfresco.transform.exceptions.TransformException;
import org.alfresco.transformer.logging.LogEntry;
import org.springframework.core.io.Resource;
import org.springframework.core.io.UrlResource;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.util.UriUtils;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
*/
@Deprecated
public class FileManager
{
public static final String SOURCE_FILE = "sourceFile";
public static final String TARGET_FILE = "targetFile";
private static final String FILENAME = "filename=";
/**
* Returns a File to be used to store the result of a transformation.
*
* @param request
* @param filename The targetFilename supplied in the request. Only the filename if a path is used as part of the
* temporary filename.
* @return a temporary File.
* @throws TransformException if there was no target filename.
*/
public static File createTargetFile(HttpServletRequest request, String filename)
{
File file = buildFile(filename);
request.setAttribute(TARGET_FILE, file);
return file;
}
public static File buildFile(String filename)
{
filename = checkFilename(false, filename);
LogEntry.setTarget(filename);
return TempFileProvider.createTempFile("target_", "_" + filename);
}
public static void deleteFile(final File file) throws Exception
{
if (!file.delete())
{
throw new Exception("Failed to delete file");
}
}
/**
* Checks the filename is okay to uses in a temporary file name.
*
* @param filename or path to be checked.
* @return the filename part of the supplied filename if it was a path.
* @throws TransformException if there was no target filename.
*/
private static String checkFilename(boolean source, String filename)
{
filename = getFilename(filename);
if (filename == null || filename.isEmpty())
{
String sourceOrTarget = source ? "source" : "target";
HttpStatus statusCode = source ? BAD_REQUEST : INTERNAL_SERVER_ERROR;
throw new TransformException(statusCode, "The " + sourceOrTarget + " filename was not supplied");
}
return filename;
}
private static void save(MultipartFile multipartFile, File file)
{
try
{
Files.copy(multipartFile.getInputStream(), file.toPath(),
StandardCopyOption.REPLACE_EXISTING);
}
catch (IOException e)
{
throw new TransformException(INSUFFICIENT_STORAGE, "Failed to store the source file", e);
}
}
public static void save(Resource body, File file)
{
try
{
Files.copy(body.getInputStream(), file.toPath(), StandardCopyOption.REPLACE_EXISTING);
}
catch (IOException e)
{
throw new TransformException(INSUFFICIENT_STORAGE, "Failed to store the source file", e);
}
}
private static Resource load(File file)
{
try
{
Resource resource = new UrlResource(file.toURI());
if (resource.exists() || resource.isReadable())
{
return resource;
}
else
{
throw new TransformException(INTERNAL_SERVER_ERROR,
"Could not read the target file: " + file.getPath());
}
}
catch (MalformedURLException e)
{
throw new TransformException(INTERNAL_SERVER_ERROR,
"The target filename was malformed: " + file.getPath(), e);
}
}
public static String getFilenameFromContentDisposition(HttpHeaders headers)
{
String filename = "";
String contentDisposition = headers.getFirst(CONTENT_DISPOSITION);
if (contentDisposition != null)
{
String[] strings = contentDisposition.split("; *");
filename = Arrays.stream(strings)
.filter(s -> s.startsWith(FILENAME))
.findFirst()
.map(s -> s.substring(FILENAME.length()))
.orElse("");
}
return filename;
}
/**
* Returns the file name for the target file
*
* @param fileName Desired file name
* @param targetExtension File extension
* @return Target file name
*/
public static String createTargetFileName(final String fileName, final String targetExtension)
{
final String sourceFilename = getFilename(fileName);
if (sourceFilename == null || sourceFilename.isEmpty())
{
return null;
}
final String ext = getFilenameExtension(sourceFilename);
if (ext == null || ext.isEmpty())
{
return sourceFilename + '.' + targetExtension;
}
return sourceFilename.substring(0, sourceFilename.length() - ext.length() - 1) +
'.' + targetExtension;
}
/**
* Returns a File that holds the source content for a transformation.
*
* @param request
* @param multipartFile from the request
* @return a temporary File.
* @throws TransformException if there was no source filename.
*/
public static File createSourceFile(HttpServletRequest request, MultipartFile multipartFile)
{
String filename = multipartFile.getOriginalFilename();
long size = multipartFile.getSize();
filename = checkFilename(true, filename);
File file = TempFileProvider.createTempFile("source_", "_" + filename);
request.setAttribute(SOURCE_FILE, file);
save(multipartFile, file);
LogEntry.setSource(filename, size);
return file;
}
public static void deleteFile(HttpServletRequest request, String attributeName)
{
File file = (File) request.getAttribute(attributeName);
if (file != null)
{
file.delete();
}
}
public static ResponseEntity<Resource> createAttachment(String targetFilename, File
targetFile)
{
Resource targetResource = load(targetFile);
targetFilename = UriUtils.encodePath(getFilename(targetFilename), "UTF-8");
return ResponseEntity.ok().header(CONTENT_DISPOSITION,
"attachment; filename*=UTF-8''" + targetFilename).body(targetResource);
}
/**
* TempFileProvider - Duplicated and adapted from alfresco-core.
*/
public static class TempFileProvider
{
public static File createTempFile(final String prefix, final String suffix)
{
final File directory = getTempDir();
try
{
return File.createTempFile(prefix, suffix, directory);
}
catch (IOException e)
{
throw new RuntimeException(
"Failed to created temp file: \n prefix: " + prefix +
"\n suffix: " + suffix + "\n directory: " + directory, e);
}
}
private static File getTempDir()
{
final String dirName = "Alfresco";
final String systemTempDirPath = System.getProperty("java.io.tmpdir");
if (systemTempDirPath == null)
{
throw new RuntimeException("System property not available: java.io.tmpdir");
}
final File systemTempDir = new File(systemTempDirPath);
final File tempDir = new File(systemTempDir, dirName);
if (!tempDir.exists() && !tempDir.mkdirs() && !tempDir.exists())
{
throw new RuntimeException("Failed to create temp directory: " + tempDir);
}
return tempDir;
}
}
}

View File

@@ -0,0 +1,330 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transformer.logging;
import static java.lang.Math.max;
import static org.springframework.http.HttpStatus.OK;
import java.text.SimpleDateFormat;
import java.util.Collection;
import java.util.Date;
import java.util.Deque;
import java.util.concurrent.ConcurrentLinkedDeque;
import java.util.concurrent.atomic.AtomicInteger;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
*
* Provides setter and getter methods to allow the current Thread to set various log properties and for these
* values to be retrieved. The {@link #complete()} method should be called at the end of a request to flush the
* current entry to an internal log Collection of the latest entries. The {@link #getLog()} method is used to obtain
* access to this collection.
*/
@Deprecated
public final class LogEntry
{
private static final Logger logger = LoggerFactory.getLogger(LogEntry.class);
// TODO allow ProbeTestTransform to find out if there are any transforms running longer than the max time.
private static final AtomicInteger count = new AtomicInteger(0);
private static final Deque<LogEntry> log = new ConcurrentLinkedDeque<>();
private static final int MAX_LOG_SIZE = 10;
private static final SimpleDateFormat HH_MM_SS = new SimpleDateFormat("HH:mm:ss");
private static final ThreadLocal<LogEntry> currentLogEntry = ThreadLocal.withInitial(() -> {
LogEntry logEntry = new LogEntry();
if (log.size() >= MAX_LOG_SIZE)
{
log.removeLast();
}
log.addFirst(logEntry);
return logEntry;
});
private final int id = count.incrementAndGet();
private final long start = System.currentTimeMillis();
private int statusCode;
private long durationStreamIn;
private long durationTransform = -1;
private long durationStreamOut = -1;
private long durationDelay = -1;
private String source;
private long sourceSize;
private String target;
private long targetSize = -1;
private String options;
private String message;
@Override
public String toString()
{
StringBuilder sb = new StringBuilder();
append(sb, Integer.toString(getId()));
append(sb, HH_MM_SS.format(getDate()));
append(sb, Integer.toString(getStatusCode()));
append(sb, getDuration());
append(sb, getSource());
append(sb, getSourceSize());
append(sb, getTarget());
append(sb, getTargetSize());
append(sb, getOptions());
sb.append(getMessage());
return sb.toString();
}
private void append(StringBuilder sb, String value)
{
if (value != null && !value.isEmpty() && !"0bytes".equals(value))
{
sb.append(value);
sb.append(' ');
}
}
public static Collection<LogEntry> getLog()
{
return log;
}
public static void start()
{
currentLogEntry.get();
}
public static void setSource(String source, long sourceSize)
{
LogEntry logEntry = currentLogEntry.get();
logEntry.source = getExtension(source);
logEntry.sourceSize = sourceSize;
logEntry.durationStreamIn = System.currentTimeMillis() - logEntry.start;
}
public static void setTarget(String target)
{
currentLogEntry.get().target = getExtension(target);
}
private static String getExtension(String filename)
{
int i = filename.lastIndexOf('.');
if (i != -1)
{
filename = filename.substring(i + 1);
}
return filename;
}
public static void setTargetSize(long targetSize)
{
currentLogEntry.get().targetSize = targetSize;
}
public static void setOptions(String options)
{
currentLogEntry.get().options = options;
}
public static long setStatusCodeAndMessage(int statusCode, String message)
{
LogEntry logEntry = currentLogEntry.get();
logEntry.statusCode = statusCode;
logEntry.message = message;
logEntry.durationTransform = System.currentTimeMillis() - logEntry.start - logEntry.durationStreamIn;
return logEntry.durationTransform;
}
// In order to test connection timeouts, a testDelay may be added as a request parameter.
// This method waits for this period to end. It is in this class as all the times are recorded here.
public static long addDelay(Long testDelay)
{
long durationDelay = 0;
if (testDelay != null && testDelay > 0)
{
durationDelay = currentLogEntry.get().addDelayInternal(testDelay);
}
return durationDelay;
}
private long addDelayInternal(Long testDelay)
{
long durationDelay = Math.max(testDelay - System.currentTimeMillis() + start, -1);
if (durationDelay > 0)
{
try
{
Thread.sleep(durationDelay);
}
catch (InterruptedException ignore)
{
Thread.currentThread().interrupt();
}
this.durationDelay = durationDelay;
return durationDelay;
}
else
{
this.durationDelay = -1;
return 0;
}
}
public static void complete()
{
LogEntry logEntry = currentLogEntry.get();
if (logEntry.statusCode == OK.value())
{
logEntry.durationStreamOut = System.currentTimeMillis() - logEntry.start -
logEntry.durationStreamIn - max(logEntry.durationTransform,
0) - max(logEntry.durationDelay, 0);
}
currentLogEntry.remove();
if (logger.isDebugEnabled())
{
logger.debug(logEntry.toString());
}
}
public int getId()
{
return id;
}
public Date getDate()
{
return new Date(start);
}
public int getStatusCode()
{
return statusCode;
}
public String getDuration()
{
long duration = durationStreamIn + max(durationTransform, 0) + max(durationDelay, 0) + max(
durationStreamOut, 0);
return duration <= 5
? ""
: time(duration) +
" (" +
(time(durationStreamIn) + ' ' +
time(durationTransform) + ' ' +
(durationDelay > 0
? time(durationDelay) + ' ' + (durationStreamOut < 0 ? "-" : time(
durationStreamOut))
: time(durationStreamOut))).trim() +
")";
}
public String getSource()
{
return source;
}
public String getSourceSize()
{
return size(sourceSize);
}
public String getTarget()
{
return target;
}
public String getTargetSize()
{
return size(targetSize);
}
public String getOptions()
{
return options;
}
public String getMessage()
{
return message;
}
private String time(long ms)
{
return ms == -1 ? "" : size(ms, "1ms",
new String[]{"ms", "s", "min", "hr"},
new long[]{1000, 60 * 1000, 60 * 60 * 1000, Long.MAX_VALUE});
}
private String size(long size)
{
// TODO fix numeric overflow in TB expression
return size == -1 ? "" : size(size, "1 byte",
new String[]{"bytes", " KB", " MB", " GB", " TB"},
new long[]{1024, 1024 * 1024, 1024 * 1024 * 1024, 1024 * 1024 * 1024 * 1024, Long.MAX_VALUE});
}
private String size(long size, String singleValue, String[] units, long[] dividers)
{
if (size == 1)
{
return singleValue;
}
long divider = 1;
for (int i = 0; i < units.length - 1; i++)
{
long nextDivider = dividers[i];
if (size < nextDivider)
{
return unitFormat(size, divider, units[i]);
}
divider = nextDivider;
}
return unitFormat(size, divider, units[units.length - 1]);
}
private String unitFormat(long size, long divider, String unit)
{
size = size * 10 / divider;
int decimalPoint = (int) size % 10;
StringBuilder sb = new StringBuilder();
sb.append(size / 10);
if (decimalPoint != 0)
{
sb.append(".");
sb.append(decimalPoint);
}
sb.append(unit);
return sb.toString();
}
}

View File

@@ -0,0 +1,39 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transformer.logging;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
*/
@Deprecated
public interface StandardMessages
{
String LICENCE =
"If the Alfresco software was purchased under a paid Alfresco license, the terms of the paid license agreement \n" +
"will prevail. Otherwise, the software is provided under terms of the GNU LGPL v3 license. \n" +
"See the license at http://www.gnu.org/licenses/lgpl-3.0.txt. or in /LICENSE.txt \n\n";
}

View File

@@ -0,0 +1,111 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transformer.messaging;
import javax.jms.ConnectionFactory;
import javax.jms.Queue;
import org.alfresco.transform.messages.TransformRequestValidator;
import org.apache.activemq.command.ActiveMQQueue;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.jms.annotation.JmsListenerConfigurer;
import org.springframework.jms.config.DefaultJmsListenerContainerFactory;
import org.springframework.jms.config.JmsListenerEndpointRegistrar;
import org.springframework.jms.connection.JmsTransactionManager;
import org.springframework.lang.NonNull;
import org.springframework.messaging.handler.annotation.support.DefaultMessageHandlerMethodFactory;
import org.springframework.transaction.PlatformTransactionManager;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
*
* JMS and messaging configuration for the T-Engines. Contains the basic config in order to have the
* T-Engine able to read from queues and send a reply back.
*
* @author Lucian Tuca
* created on 18/12/2018
*/
@Deprecated
@Configuration
@ConditionalOnProperty(name = "activemq.url")
public class MessagingConfig implements JmsListenerConfigurer
{
private static final Logger logger = LoggerFactory.getLogger(MessagingConfig.class);
@Override
public void configureJmsListeners(@NonNull JmsListenerEndpointRegistrar registrar)
{
registrar.setMessageHandlerMethodFactory(methodFactory());
}
@Bean
@ConditionalOnProperty(name = "activemq.url")
public DefaultMessageHandlerMethodFactory methodFactory()
{
DefaultMessageHandlerMethodFactory factory = new DefaultMessageHandlerMethodFactory();
factory.setValidator(new TransformRequestValidator());
return factory;
}
@Bean
@ConditionalOnProperty(name = "activemq.url")
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory(
final ConnectionFactory connectionFactory,
final TransformMessageConverter transformMessageConverter)
{
final DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory);
factory.setMessageConverter(transformMessageConverter);
factory.setErrorHandler(t -> logger.error("JMS error: " + t.getMessage(), t));
factory.setTransactionManager(transactionManager(connectionFactory));
return factory;
}
@Bean
@ConditionalOnProperty(name = "activemq.url")
public PlatformTransactionManager transactionManager(final ConnectionFactory connectionFactory)
{
final JmsTransactionManager transactionManager = new JmsTransactionManager();
transactionManager.setConnectionFactory(connectionFactory);
return transactionManager;
}
@Bean
@ConditionalOnProperty(name = "activemq.url")
public Queue engineRequestQueue(
@Value("${queue.engineRequestQueue}") String engineRequestQueueValue)
{
return new ActiveMQQueue(engineRequestQueueValue);
}
}

View File

@@ -0,0 +1,70 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transformer.messaging;
import javax.annotation.PostConstruct;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Configuration;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
*
* Prints JMS status information at application startup.
*
* @author Cezar Leahu
*/
@Deprecated
@Configuration
public class MessagingInfo
{
private static final Logger logger = LoggerFactory.getLogger(MessagingInfo.class);
@Value("${activemq.url:}")
private String activemqUrl;
@PostConstruct
public void init()
{
// For backwards-compatibility, we continue to rely on setting ACTIVEMQ_URL environment variable (see application.yaml)
// The MessagingConfig class uses on ConditionalOnProperty (ie. activemq.url is set and not false)
// Note: as per application.yaml the broker url is appended with "?jms.watchTopicAdvisories=false". If this needs to be fully
// overridden then it would require explicitly setting both "spring.activemq.broker-url" *and* "activemq.url" (latter to non-false value).
if ((activemqUrl != null) && (! activemqUrl.equals("false")))
{
logger.info("JMS client is ENABLED - ACTIVEMQ_URL ='{}'", activemqUrl);
}
else
{
logger.info("JMS client is DISABLED - ACTIVEMQ_URL is not set");
}
}
}

View File

@@ -0,0 +1,100 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transformer.messaging;
import com.fasterxml.jackson.databind.JavaType;
import com.fasterxml.jackson.databind.type.TypeFactory;
import com.google.common.collect.ImmutableMap;
import org.alfresco.transform.client.model.TransformReply;
import org.alfresco.transform.client.model.TransformRequest;
import org.springframework.jms.support.converter.MappingJackson2MessageConverter;
import org.springframework.jms.support.converter.MessageConversionException;
import org.springframework.jms.support.converter.MessageConverter;
import org.springframework.jms.support.converter.MessageType;
import org.springframework.lang.NonNull;
import org.springframework.stereotype.Service;
import javax.jms.JMSException;
import javax.jms.Message;
import javax.jms.Session;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
*
* TODO: Duplicated from the Router
* Custom wrapper over MappingJackson2MessageConverter for T-Request/T-Reply objects.
*
* @author Cezar Leahu
*/
@Deprecated
@Service
public class TransformMessageConverter implements MessageConverter
{
private static final MappingJackson2MessageConverter converter;
private static final JavaType TRANSFORM_REQUEST_TYPE =
TypeFactory.defaultInstance().constructType(TransformRequest.class);
static
{
converter = new MappingJackson2MessageConverter()
{
@Override
@NonNull
protected JavaType getJavaTypeForMessage(final Message message) throws JMSException
{
if (message.getStringProperty("_type") == null)
{
return TRANSFORM_REQUEST_TYPE;
}
return super.getJavaTypeForMessage(message);
}
};
converter.setTargetType(MessageType.BYTES);
converter.setTypeIdPropertyName("_type");
converter.setTypeIdMappings(ImmutableMap.of(
TransformRequest.class.getName(), TransformRequest.class,
TransformReply.class.getName(), TransformReply.class)
);
}
@Override
@NonNull
public Message toMessage(
@NonNull final Object object,
@NonNull final Session session) throws JMSException, MessageConversionException
{
return converter.toMessage(object, session);
}
@Override
@NonNull
public Object fromMessage(@NonNull final Message message) throws JMSException
{
return converter.fromMessage(message);
}
}

View File

@@ -0,0 +1,81 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transformer.messaging;
import javax.jms.Destination;
import org.alfresco.transform.client.model.TransformReply;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.jms.core.JmsTemplate;
import org.springframework.stereotype.Component;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
*
* TODO: Duplicated from the Router
* TransformReplySender Bean
* <p/>
* JMS message sender/publisher
*
* @author Cezar Leahu
*/
@Deprecated
@Component
public class TransformReplySender
{
private static final Logger logger = LoggerFactory.getLogger(TransformReplySender.class);
@Autowired
private JmsTemplate jmsTemplate;
public void send(final Destination destination, final TransformReply reply)
{
send(destination, reply, reply.getRequestId());
}
public void send(final Destination destination, final TransformReply reply, final String correlationId)
{
if (destination != null)
{
try
{
//jmsTemplate.setSessionTransacted(true); // do we need this?
jmsTemplate.convertAndSend(destination, reply, m -> {
m.setJMSCorrelationID(correlationId);
return m;
});
logger.trace("Sent: {} - with correlation ID {}", reply, correlationId);
}
catch (Exception e)
{
logger.error("Failed to send T-Reply " + reply + " - for correlation ID " + correlationId, e);
}
}
}
}

View File

@@ -0,0 +1,592 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005-2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transformer.metadataExtractors;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.slf4j.Logger;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.io.Serializable;
import java.lang.reflect.Array;
import java.util.Arrays;
import java.util.Collection;
import java.util.Collections;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Properties;
import java.util.Set;
import java.util.StringTokenizer;
import java.util.TreeMap;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
*
* Helper methods for metadata extract and embed.
* <p>
* <i>Much of the code is based on AbstractMappingMetadataExtracter from the
* content repository. The code has been simplified to only set up mapping one way.</i>
* <p>
* If a transform specifies that it can convert from {@code "<MIMETYPE>"} to {@code "alfresco-metadata-extract"}
* (specified in the {@code engine_config.json}), it is indicating that it can extract metadata from {@code <MIMETYPE>}.
*
* The transform results in a Map of extracted properties encoded as json being returned to the content repository.
* <ul>
* <li>The content repository will use a transform in preference to any metadata extractors it might have defined
* locally for the same MIMETYPE.</li>
* <li>The T-Engine's Controller class will call a method in a class that extends {@link AbstractMetadataExtractor}
* based on the source and target mediatypes in the normal way.</li>
* <li>The method extracts ALL available metadata is extracted from the document and then calls
* {@link #mapMetadataAndWrite(File, Map, Map)}.</li>
* <li>Selected values from the available metadata are mapped into content repository property names and values,
* depending on what is defined in a {@code "<classname>_metadata_extract.properties"} file.</li>
* <li>The selected values are set back to the content repository as a JSON representation of a Map, where the values
* are applied to the source node.</li>
* </ul>
* To support the same functionality as metadata extractors configured inside the content repository,
* extra key value pairs may be returned from {@link #extractMetadata}. These are:
* <ul>
* <li>{@code "sys:overwritePolicy"} which can specify the
* {@code org.alfresco.repo.content.metadata.MetadataExtracter.OverwritePolicy} name. Defaults to "PRAGMATIC".</li>
* <li>{@code "sys:enableStringTagging"} if {@code "true"} finds or creates tags for each string mapped to
* {@code cm:taggable}. Defaults to {@code "false"} to ignore mapping strings to tags.</li>
* <li>{@code "sys:carryAspectProperties"} </li>
* <li>{@code "sys:stringTaggingSeparators"} </li>
* </ul>
*
* If a transform specifies that it can convert from {@code "<MIMETYPE>"} to {@code "alfresco-metadata-embed"}, it is
* indicating that it can embed metadata in {@code <MIMETYPE>}.
*
* The transform results in a new version of supplied source file that contains the metadata supplied in the transform
* options.
*
* @author Jesper Steen Møller
* @author Derek Hulley
* @author adavis
*/
@Deprecated
public abstract class AbstractMetadataExtractor
{
private static final String EXTRACT = "extract";
private static final String EMBED = "embed";
private static final String METADATA = "metadata";
private static final String EXTRACT_MAPPING = "extractMapping";
private static final String NAMESPACE_PROPERTY_PREFIX = "namespace.prefix.";
private static final char NAMESPACE_PREFIX = ':';
private static final char NAMESPACE_BEGIN = '{';
private static final char NAMESPACE_END = '}';
private static final List<String> SYS_PROPERTIES = Arrays.asList(
"sys:overwritePolicy",
"sys:enableStringTagging",
"sys:carryAspectProperties",
"sys:stringTaggingSeparators");
private static final ObjectMapper jsonObjectMapper = new ObjectMapper();
protected final Logger logger;
private Map<String, Set<String>> defaultExtractMapping;
private ThreadLocal<Map<String, Set<String>>> extractMapping = new ThreadLocal<>();
private Map<String, Set<String>> embedMapping;
public AbstractMetadataExtractor(Logger logger)
{
this.logger = logger;
defaultExtractMapping = Collections.emptyMap();
embedMapping = Collections.emptyMap();
try
{
defaultExtractMapping = buildExtractMapping();
embedMapping = buildEmbedMapping();
}
catch (Exception e)
{
logger.error("Failed to read config", e);
}
}
public abstract Map<String, Serializable> extractMetadata(String sourceMimetype, Map<String, String> transformOptions,
File sourceFile) throws Exception;
public void embedMetadata(String sourceMimetype, String targetMimetype, Map<String, String> transformOptions,
File sourceFile, File targetFile) throws Exception
{
// Default nothing, as embedding is not supported in most cases
}
protected Map<String, Serializable> getMetadata(Map<String, String> transformOptions)
{
String metadataAsJson = transformOptions.get(METADATA);
if (metadataAsJson == null)
{
throw new IllegalArgumentException("No metadata in embed request");
}
try
{
TypeReference<HashMap<String, Serializable>> typeRef = new TypeReference<>() {};
HashMap<String, Serializable> systemProperties = jsonObjectMapper.readValue(metadataAsJson, typeRef);
Map<String, Serializable> rawProperties = mapSystemToRaw(systemProperties);
return rawProperties;
}
catch (JsonProcessingException e)
{
throw new IllegalArgumentException("Failed to read metadata from request", e);
}
}
private Map<String, Serializable> mapSystemToRaw(Map<String, Serializable> systemMetadata)
{
Map<String, Serializable> metadataProperties = new HashMap<>(systemMetadata.size() * 2 + 1);
for (Map.Entry<String, Serializable> entry : systemMetadata.entrySet())
{
String modelProperty = entry.getKey();
// Check if there is a mapping for this
if (!embedMapping.containsKey(modelProperty))
{
// No mapping - ignore
continue;
}
Serializable documentValue = entry.getValue();
Set<String> metadataKeys = embedMapping.get(modelProperty);
for (String metadataKey : metadataKeys)
{
metadataProperties.put(metadataKey, documentValue);
}
}
// Done
if (logger.isDebugEnabled())
{
logger.debug(
"Converted system model values to metadata values: \n" +
" System Properties: " + systemMetadata + "\n" +
" Metadata Properties: " + metadataProperties);
}
return metadataProperties;
}
protected Map<String, Set<String>> getExtractMapping()
{
return Collections.unmodifiableMap(extractMapping.get());
}
public Map<String, Set<String>> getEmbedMapping()
{
return Collections.unmodifiableMap(embedMapping);
}
/**
* Based on AbstractMappingMetadataExtracter#getDefaultMapping.
*
* This method provides a <i>mapping</i> of where to store the values extracted from the documents. The list of
* properties need <b>not</b> include all metadata values extracted from the document. This mapping should be
* defined in a file based on the class name: {@code "<classname>_metadata_extract.properties"}
* @return Returns a static mapping. It may not be null.
*/
private Map<String, Set<String>> buildExtractMapping()
{
String filename = getPropertiesFilename(EXTRACT);
Properties properties = readProperties(filename);
if (properties == null)
{
logger.error("Failed to read "+filename);
}
Map<String, String> namespacesByPrefix = getNamespaces(properties);
return buildExtractMapping(properties, namespacesByPrefix);
}
private Map<String, Set<String>> buildExtractMapping(Properties properties, Map<String, String> namespacesByPrefix)
{
// Create the mapping
Map<String, Set<String>> convertedMapping = new HashMap<>(17);
for (Map.Entry<Object, Object> entry : properties.entrySet())
{
String documentProperty = (String) entry.getKey();
String qnamesStr = (String) entry.getValue();
if (documentProperty.startsWith(NAMESPACE_PROPERTY_PREFIX))
{
continue;
}
// Create the entry
Set<String> qnames = new HashSet<>(3);
convertedMapping.put(documentProperty, qnames);
// The to value can be a list of QNames
StringTokenizer tokenizer = new StringTokenizer(qnamesStr, ",");
while (tokenizer.hasMoreTokens())
{
String qnameStr = tokenizer.nextToken().trim();
qnameStr = getQNameString(namespacesByPrefix, entry, qnameStr, EXTRACT);
qnames.add(qnameStr);
}
if (logger.isTraceEnabled())
{
logger.trace("Added mapping from " + documentProperty + " to " + qnames);
}
}
return convertedMapping;
}
/**
* Based on AbstractMappingMetadataExtracter#getDefaultEmbedMapping.
*
* This method provides a <i>mapping</i> of model properties that should be embedded in the content. The list of
* properties need <b>not</b> include all properties. This mapping should be defined in a file based on the class
* name: {@code "<classname>_metadata_embed.properties"}
* <p>
* If no {@code "<classname>_metadata_embed.properties"} file is found, a reverse of the
* {@code "<classname>_metadata_extract.properties"} will be assumed. A last win approach will be used for handling
* duplicates.
* @return Returns a static mapping. It may not be null.
*/
private Map<String, Set<String>> buildEmbedMapping()
{
String filename = getPropertiesFilename(EMBED);
Properties properties = readProperties(filename);
Map<String, Set<String>> embedMapping;
if (properties != null)
{
Map<String, String> namespacesByPrefix = getNamespaces(properties);
embedMapping = buildEmbedMapping(properties, namespacesByPrefix);
}
else
{
if (logger.isDebugEnabled())
{
logger.debug("No " + filename + ", assuming reverse of extract mapping");
}
embedMapping = buildEmbedMappingByReversingExtract();
}
return embedMapping;
}
private Map<String, Set<String>> buildEmbedMapping(Properties properties, Map<String, String> namespacesByPrefix)
{
Map<String, Set<String>> convertedMapping = new HashMap<>(17);
for (Map.Entry<Object, Object> entry : properties.entrySet())
{
String modelProperty = (String) entry.getKey();
String metadataKeysString = (String) entry.getValue();
if (modelProperty.startsWith(NAMESPACE_PROPERTY_PREFIX))
{
continue;
}
modelProperty = getQNameString(namespacesByPrefix, entry, modelProperty, EMBED);
String[] metadataKeysArray = metadataKeysString.split(",");
Set<String> metadataKeys = new HashSet<String>(metadataKeysArray.length);
for (String metadataKey : metadataKeysArray) {
metadataKeys.add(metadataKey.trim());
}
// Create the entry
convertedMapping.put(modelProperty, metadataKeys);
if (logger.isTraceEnabled())
{
logger.trace("Added mapping from " + modelProperty + " to " + metadataKeysString);
}
}
return convertedMapping;
}
private Map<String, Set<String>> buildEmbedMappingByReversingExtract()
{
Map<String, Set<String>> extractMapping = buildExtractMapping();
Map<String, Set<String>> embedMapping;
embedMapping = new HashMap<>(extractMapping.size());
for (String metadataKey : extractMapping.keySet())
{
if (extractMapping.get(metadataKey) != null && extractMapping.get(metadataKey).size() > 0)
{
String modelProperty = extractMapping.get(metadataKey).iterator().next();
Set<String> metadataKeys = embedMapping.get(modelProperty);
if (metadataKeys == null)
{
metadataKeys = new HashSet<String>(1);
embedMapping.put(modelProperty, metadataKeys);
}
metadataKeys.add(metadataKey);
if (logger.isTraceEnabled())
{
logger.trace("Added mapping from " + modelProperty + " to " + metadataKeys.toString());
}
}
}
return embedMapping;
}
private String getPropertiesFilename(String suffix)
{
String className = this.getClass().getName();
String shortClassName = className.split("\\.")[className.split("\\.").length - 1];
shortClassName = shortClassName.replace('$', '-');
return shortClassName + "_metadata_" + suffix + ".properties";
}
private Properties readProperties(String filename)
{
Properties properties = null;
try
{
InputStream inputStream = AbstractMetadataExtractor.class.getClassLoader().getResourceAsStream(filename);
if (inputStream != null)
{
properties = new Properties();
properties.load(inputStream);
}
}
catch (IOException ignore)
{
}
return properties;
}
private Map<String, String> getNamespaces(Properties properties)
{
Map<String, String> namespacesByPrefix = new HashMap<String, String>(5);
for (Map.Entry<Object, Object> entry : properties.entrySet())
{
String propertyName = (String) entry.getKey();
if (propertyName.startsWith(NAMESPACE_PROPERTY_PREFIX))
{
String prefix = propertyName.substring(17);
String namespace = (String) entry.getValue();
namespacesByPrefix.put(prefix, namespace);
}
}
return namespacesByPrefix;
}
private String getQNameString(Map<String, String> namespacesByPrefix, Map.Entry<Object, Object> entry, String qnameStr, String type)
{
// Check if we need to resolve a namespace reference
int index = qnameStr.indexOf(NAMESPACE_PREFIX);
if (index > -1 && qnameStr.charAt(0) != NAMESPACE_BEGIN)
{
String prefix = qnameStr.substring(0, index);
String suffix = qnameStr.substring(index + 1);
// It is prefixed
String uri = namespacesByPrefix.get(prefix);
if (uri == null)
{
throw new IllegalArgumentException("No prefix mapping for " + type + " property mapping: \n" +
" Extractor: " + this + "\n" +
" Mapping: " + entry);
}
qnameStr = NAMESPACE_BEGIN + uri + NAMESPACE_END + suffix;
}
return qnameStr;
}
/**
* Adds a value to the map, conserving null values. Values are converted to null if:
* <ul>
* <li>it is an empty string value after trimming</li>
* <li>it is an empty collection</li>
* <li>it is an empty array</li>
* </ul>
* String values are trimmed before being put into the map.
* Otherwise, it is up to the extracter to ensure that the value is a <tt>Serializable</tt>.
* It is not appropriate to implicitly convert values in order to make them <tt>Serializable</tt>
* - the best conversion method will depend on the value's specific meaning.
*
* @param key the destination key
* @param value the serializable value
* @param destination the map to put values into
* @return Returns <tt>true</tt> if set, otherwise <tt>false</tt>
*/
// Copied from the content repository's AbstractMappingMetadataExtracter.
protected boolean putRawValue(String key, Serializable value, Map<String, Serializable> destination)
{
if (value == null)
{
// Just keep this
}
else if (value instanceof String)
{
String valueStr = ((String) value).trim();
if (valueStr.length() == 0)
{
value = null;
}
else
{
if (valueStr.indexOf("\u0000") != -1)
{
valueStr = valueStr.replaceAll("\u0000", "");
}
// Keep the trimmed value
value = valueStr;
}
}
else if (value instanceof Collection)
{
Collection<?> valueCollection = (Collection<?>) value;
if (valueCollection.isEmpty())
{
value = null;
}
}
else if (value.getClass().isArray())
{
if (Array.getLength(value) == 0)
{
value = null;
}
}
// It passed all the tests
destination.put(key, value);
return true;
}
/**
* The {@code transformOptions} may contain a replacement set of mappings. These will be used in place of the
* default mappings from read from file if supplied.
*/
public void extractMetadata(String sourceMimetype, Map<String, String> transformOptions, File sourceFile,
File targetFile) throws Exception
{
Map<String, Set<String>> mapping = getExtractMappingFromOptions(transformOptions, defaultExtractMapping);
// Use a ThreadLocal to avoid changing method signatures of methods that currently call getExtractMapping.
try
{
extractMapping.set(mapping);
Map<String, Serializable> metadata = extractMetadata(sourceMimetype, transformOptions, sourceFile);
mapMetadataAndWrite(targetFile, metadata, mapping);
}
finally
{
extractMapping.set(null);
}
}
private Map<String, Set<String>> getExtractMappingFromOptions(Map<String, String> transformOptions, Map<String,
Set<String>> defaultExtractMapping)
{
String extractMappingOption = transformOptions.get(EXTRACT_MAPPING);
if (extractMappingOption != null)
{
try
{
TypeReference<HashMap<String, Set<String>>> typeRef = new TypeReference<>() {};
return jsonObjectMapper.readValue(extractMappingOption, typeRef);
}
catch (JsonProcessingException e)
{
throw new IllegalArgumentException("Failed to read "+ EXTRACT_MAPPING +" from request", e);
}
}
return defaultExtractMapping;
}
/**
* @deprecated use {@link #extractMetadata(String, Map, File, File)} rather than calling this method.
* By default call the overloaded method with the default {@code extractMapping}.
*/
@Deprecated
public void mapMetadataAndWrite(File targetFile, Map<String, Serializable> metadata) throws IOException
{
mapMetadataAndWrite(targetFile, metadata, defaultExtractMapping);
}
public void mapMetadataAndWrite(File targetFile, Map<String, Serializable> metadata,
Map<String, Set<String>> extractMapping) throws IOException
{
if (logger.isDebugEnabled())
{
logger.debug("Raw metadata:");
metadata.forEach((k,v) -> logger.debug(" "+k+"="+v));
}
metadata = mapRawToSystem(metadata, extractMapping);
writeMetadata(targetFile, metadata);
}
/**
* Based on AbstractMappingMetadataExtracter#mapRawToSystem.
*
* @param rawMetadata Metadata keyed by document properties
* @param extractMapping Mapping between document ans system properties
* @return Returns the metadata keyed by the system properties
*/
private Map<String, Serializable> mapRawToSystem(Map<String, Serializable> rawMetadata,
Map<String, Set<String>> extractMapping)
{
boolean debugEnabled = logger.isDebugEnabled();
if (debugEnabled)
{
logger.debug("Returned metadata:");
}
Map<String, Serializable> systemProperties = new HashMap<String, Serializable>(rawMetadata.size() * 2 + 1);
for (Map.Entry<String, Serializable> entry : rawMetadata.entrySet())
{
String documentKey = entry.getKey();
Serializable documentValue = entry.getValue();
if (SYS_PROPERTIES.contains(documentKey))
{
systemProperties.put(documentKey, documentValue);
if (debugEnabled)
{
logger.debug(" " + documentKey + "=" + documentValue);
}
continue;
}
// Check if there is a mapping for this
if (!extractMapping.containsKey(documentKey))
{
// No mapping - ignore
continue;
}
Set<String> systemQNames = extractMapping.get(documentKey);
for (String systemQName : systemQNames)
{
if (debugEnabled)
{
logger.debug(" "+systemQName+"="+documentValue+" ("+documentKey+")");
}
systemProperties.put(systemQName, documentValue);
}
}
return new TreeMap<String, Serializable>(systemProperties);
}
private void writeMetadata(File targetFile, Map<String, Serializable> results)
throws IOException
{
jsonObjectMapper.writeValue(targetFile, results);
}
}

View File

@@ -0,0 +1,80 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transformer.model;
import java.util.Objects;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
*
* TODO: Copied from org.alfresco.store.entity (alfresco-shared-file-store). To be discussed
*
* POJO that represents content reference ({@link java.util.UUID})
*/
@Deprecated
public class FileRefEntity
{
private String fileRef;
public FileRefEntity() {}
public FileRefEntity(String fileRef)
{
this.fileRef = fileRef;
}
public void setFileRef(String fileRef)
{
this.fileRef = fileRef;
}
public String getFileRef()
{
return fileRef;
}
@Override
public boolean equals(Object o)
{
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
FileRefEntity that = (FileRefEntity) o;
return Objects.equals(fileRef, that.fileRef);
}
@Override
public int hashCode()
{
return Objects.hash(fileRef);
}
@Override
public String toString()
{
return fileRef;
}
}

View File

@@ -0,0 +1,57 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transformer.model;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
*
* TODO: Copied from org.alfresco.store.entity (alfresco-shared-file-store). To be discussed
*
* POJO that describes the ContentRefEntry response, contains {@link FileRefEntity} according to API spec
*/
@Deprecated
public class FileRefResponse
{
private FileRefEntity entry;
public FileRefResponse() {}
public FileRefResponse(FileRefEntity entry)
{
this.entry = entry;
}
public FileRefEntity getEntry()
{
return entry;
}
public void setEntry(FileRefEntity entry)
{
this.entry = entry;
}
}

View File

@@ -0,0 +1,374 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transformer.probes;
import static org.alfresco.transformer.fs.FileManager.SOURCE_FILE;
import static org.alfresco.transformer.fs.FileManager.TARGET_FILE;
import static org.alfresco.transformer.fs.FileManager.TempFileProvider.createTempFile;
import static org.springframework.http.HttpStatus.INSUFFICIENT_STORAGE;
import static org.springframework.http.HttpStatus.INTERNAL_SERVER_ERROR;
import static org.springframework.http.HttpStatus.OK;
import static org.springframework.http.HttpStatus.TOO_MANY_REQUESTS;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.StandardCopyOption;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicLong;
import javax.servlet.http.HttpServletRequest;
import org.alfresco.transform.exceptions.TransformException;
import org.alfresco.transformer.AbstractTransformerController;
import org.alfresco.transformer.logging.LogEntry;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
*
* Provides the logic performing test transformations by the live and ready probes.
*
* <p><b>K8s probes</b>: A readiness probe indicates if the pod should accept request. <b>It does not indicate that a pod is
* ready after startup</b>. The liveness probe indicates when to kill the pod. <b>Both probes are called throughout the
* lifetime of the pod</b> and a <b>liveness probes can take place before a readiness probe.</b> The k8s
* <b>initialDelaySeconds field is not fully honoured</b> as it is multiplied by a random number, so is
* actually a maximum initial delay in seconds, but could be 0. </p>
*
* <p>Live and readiness probes might do test transforms. The first 6 requests result in a transformation
* of a small test file. The average time is remembered, but excludes the first one which is normally longer. This is
* used in future requests to discover if transformations are becoming slower. The request also returns a non 200 status
* code resulting in the k8s pod being terminated, after a predefined number of transformations have been performed or
* if any transformation takes a long time. These are controlled by environment variables:</p>
* <ul>
* <li>livenessPercent - The percentage slower the small test transform must be to indicate there is a problem.</li>
* <li>livenessTransformPeriodSeconds - As liveness probes should be frequent, not every request should result in
* a test transformation. This value defines the gap between transformations.</li>
* <li>maxTransforms - the maximum number of transformation to be performed before a restart.</li>
* <li>maxTransformSeconds - the maximum time for a transformation, including failed ones.</li>
* </ul>
*/
@Deprecated
public abstract class ProbeTestTransform
{
private final Logger logger = LoggerFactory.getLogger(ProbeTestTransform.class);
private static final int AVERAGE_OVER_TRANSFORMS = 5;
private final String sourceFilename;
private final String targetFilename;
private final long minExpectedLength;
private final long maxExpectedLength;
private int livenessPercent;
private long probeCount;
private int transCount;
private long normalTime;
private long maxTime = Long.MAX_VALUE;
private long nextTransformTime;
private final boolean livenessTransformEnabled;
private final long livenessTransformPeriod;
private final long maxTransformCount;
private long maxTransformTime;
private final AtomicBoolean initialised = new AtomicBoolean(false);
private final AtomicBoolean readySent = new AtomicBoolean(false);
private final AtomicLong transformCount = new AtomicLong(0);
private final AtomicBoolean die = new AtomicBoolean(false);
public int getLivenessPercent()
{
return livenessPercent;
}
public long getMaxTime()
{
return maxTime;
}
/**
* See Probes.md for more info.
*
* @param expectedLength was the length of the target file during testing
* @param plusOrMinus simply allows for some variation in the transformed size caused by new versions of dates
* @param livenessPercent indicates that for this type of transform a variation up to 2 and a half times is not
* unreasonable under load
* @param maxTransforms default values normally supplied by helm. Not identical so we can be sure which value is used.
* @param maxTransformSeconds default values normally supplied by helm. Not identical so we can be sure which value is used.
* @param livenessTransformPeriodSeconds default values normally supplied by helm. Not identical so we can be sure which value is used.
*/
public ProbeTestTransform(AbstractTransformerController controller,
String sourceFilename, String targetFilename, long expectedLength, long plusOrMinus,
int livenessPercent, long maxTransforms, long maxTransformSeconds,
long livenessTransformPeriodSeconds)
{
this.sourceFilename = sourceFilename;
this.targetFilename = targetFilename;
minExpectedLength = Math.max(0, expectedLength - plusOrMinus);
maxExpectedLength = expectedLength + plusOrMinus;
this.livenessPercent = (int) getPositiveLongEnv("livenessPercent", livenessPercent);
maxTransformCount = getPositiveLongEnv("maxTransforms", maxTransforms);
maxTransformTime = getPositiveLongEnv("maxTransformSeconds", maxTransformSeconds) * 1000;
livenessTransformPeriod = getPositiveLongEnv("livenessTransformPeriodSeconds",
livenessTransformPeriodSeconds) * 1000;
livenessTransformEnabled = getBooleanEnvVar("livenessTransformEnabled", false);
}
private boolean getBooleanEnvVar(final String name, final boolean defaultValue)
{
try
{
return Boolean.parseBoolean(System.getenv(name));
}
catch (Exception ignore)
{
}
return defaultValue;
}
private long getPositiveLongEnv(String name, long defaultValue)
{
long l = -1;
String env = System.getenv(name);
if (env != null)
{
try
{
l = Long.parseLong(env);
}
catch (NumberFormatException ignore)
{
}
}
if (l <= 0)
{
l = defaultValue;
}
logger.trace("Probe: {}={}", name, l);
return l;
}
// We don't want to be doing test transforms every few seconds, but do want frequent live probes.
public String doTransformOrNothing(HttpServletRequest request, boolean isLiveProbe)
{
// If not initialised OR it is a live probe and we are scheduled to to do a test transform.
probeCount++;
// TODO: update/fix/refactor liveness probes as part of ATS-138
if (isLiveProbe && !livenessTransformEnabled)
{
return doNothing(true);
}
return (isLiveProbe && livenessTransformPeriod > 0 &&
(transCount <= AVERAGE_OVER_TRANSFORMS || nextTransformTime < System.currentTimeMillis()))
|| !initialised.get()
? doTransform(request, isLiveProbe)
: doNothing(isLiveProbe);
}
private String doNothing(boolean isLiveProbe)
{
String probeMessage = getProbeMessage(isLiveProbe);
String message = "Success - No transform.";
LogEntry.setStatusCodeAndMessage(OK.value(), probeMessage + message);
if (!isLiveProbe && !readySent.getAndSet(true))
{
logger.trace("{}{}", probeMessage, message);
}
return message;
}
private String doTransform(HttpServletRequest request, boolean isLiveProbe)
{
checkMaxTransformTimeAndCount(isLiveProbe);
long start = System.currentTimeMillis();
if (nextTransformTime != 0)
{
do
{
nextTransformTime += livenessTransformPeriod;
}
while (nextTransformTime < start);
}
File sourceFile = getSourceFile(request, isLiveProbe);
File targetFile = getTargetFile(request);
executeTransformCommand(sourceFile, targetFile);
long time = System.currentTimeMillis() - start;
String message = "Transform " + time + "ms";
checkTargetFile(targetFile, isLiveProbe, message);
recordTransformTime(time);
calculateMaxTime(time, isLiveProbe);
if (time > maxTime)
{
throw new TransformException(INTERNAL_SERVER_ERROR,
getMessagePrefix(isLiveProbe) +
message + " which is more than " + livenessPercent +
"% slower than the normal value of " + normalTime + "ms");
}
// We don't care if the ready or live probe works out if we are 'ready' to take requests.
initialised.set(true);
checkMaxTransformTimeAndCount(isLiveProbe);
return getProbeMessage(isLiveProbe) + message;
}
private void checkMaxTransformTimeAndCount(boolean isLiveProbe)
{
if (die.get())
{
throw new TransformException(TOO_MANY_REQUESTS,
getMessagePrefix(isLiveProbe) + "Transformer requested to die. A transform took " +
"longer than " + (maxTransformTime / 1000) + " seconds");
}
if (maxTransformCount > 0 && transformCount.get() > maxTransformCount)
{
throw new TransformException(TOO_MANY_REQUESTS,
getMessagePrefix(isLiveProbe) + "Transformer requested to die. It has performed " +
"more than " + maxTransformCount + " transformations");
}
}
private File getSourceFile(HttpServletRequest request, boolean isLiveProbe)
{
incrementTransformerCount();
File sourceFile = createTempFile("source_", "_" + sourceFilename);
request.setAttribute(SOURCE_FILE, sourceFile);
try (InputStream inputStream = this.getClass().getResourceAsStream('/' + sourceFilename))
{
Files.copy(inputStream, sourceFile.toPath(), StandardCopyOption.REPLACE_EXISTING);
}
catch (IOException e)
{
throw new TransformException(INSUFFICIENT_STORAGE,
getMessagePrefix(isLiveProbe) + "Failed to store the source file", e);
}
long length = sourceFile.length();
LogEntry.setSource(sourceFilename, length);
return sourceFile;
}
private File getTargetFile(HttpServletRequest request)
{
File targetFile = createTempFile("target_", "_" + targetFilename);
request.setAttribute(TARGET_FILE, targetFile);
LogEntry.setTarget(targetFilename);
return targetFile;
}
public void recordTransformTime(long time)
{
if (maxTransformTime > 0 && time > maxTransformTime)
{
die.set(true);
}
}
public void calculateMaxTime(long time, boolean isLiveProbe)
{
if (transCount <= AVERAGE_OVER_TRANSFORMS)
{
// Take the average of the first few transforms as the normal time. The initial transform might be slower
// so is ignored. Later ones are not included in case we have a gradual performance problem.
String message = getMessagePrefix(isLiveProbe) + "Success - Transform " + time + "ms";
if (++transCount > 1)
{
normalTime = (normalTime * (transCount - 2) + time) / (transCount - 1);
maxTime = (normalTime * (livenessPercent + 100)) / 100;
if ((!isLiveProbe && !readySent.getAndSet(
true)) || transCount > AVERAGE_OVER_TRANSFORMS)
{
nextTransformTime = System.currentTimeMillis() + livenessTransformPeriod;
logger.trace("{} - {}ms+{}%={}ms", message, normalTime, livenessPercent,
maxTime);
}
}
else if (!isLiveProbe && !readySent.getAndSet(true))
{
logger.trace(message);
}
}
}
protected abstract void executeTransformCommand(File sourceFile, File targetFile);
private void checkTargetFile(File targetFile, boolean isLiveProbe, String message)
{
String probeMessage = getProbeMessage(isLiveProbe);
if (!targetFile.exists() || !targetFile.isFile())
{
throw new TransformException(INTERNAL_SERVER_ERROR,
probeMessage + "Target File \"" + targetFile.getAbsolutePath() + "\" did not exist");
}
long length = targetFile.length();
if (length < minExpectedLength || length > maxExpectedLength)
{
throw new TransformException(INTERNAL_SERVER_ERROR,
probeMessage + "Target File \"" + targetFile.getAbsolutePath() +
"\" was the wrong size (" + length + "). Needed to be between " +
minExpectedLength + " and " + maxExpectedLength);
}
LogEntry.setTargetSize(length);
LogEntry.setStatusCodeAndMessage(OK.value(), probeMessage + "Success - " + message);
}
private String getMessagePrefix(boolean isLiveProbe)
{
return Long.toString(probeCount) + ' ' + getProbeMessage(isLiveProbe);
}
private String getProbeMessage(boolean isLiveProbe)
{
return (isLiveProbe ? "Live Probe: " : "Ready Probe: ");
}
public void incrementTransformerCount()
{
transformCount.incrementAndGet();
}
public void setLivenessPercent(int livenessPercent)
{
this.livenessPercent = livenessPercent;
}
public long getNormalTime()
{
return normalTime;
}
}

View File

@@ -0,0 +1,37 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2020 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transformer.util;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
*
* Partially duplicated from *alfresco-data-model*.
*/
@Deprecated
public interface MimetypeMap extends org.alfresco.transform.common.Mimetype
{
}

View File

@@ -0,0 +1,48 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transformer.util;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
*
* Extends the list of transform options with historic request parameters or 'extra' parameters used in testing
* or communication in the all-in-one transformer.
*/
@Deprecated
public interface RequestParamMap extends org.alfresco.transform.client.util.RequestParamMap
{
// This property can be sent by acs repository's legacy transformers to force a transform,
// instead of letting this T-Engine determine it based on the request parameters.
// This allows clients to specify transform names as they appear in the engine config files, for example:
// imagemagick, libreoffice, PdfBox, TikaAuto, ....
// See ATS-731.
@Deprecated
String TRANSFORM_NAME_PROPERTY = "transformName";
String TRANSFORM_NAME_PARAMETER = "alfresco.transform-name-parameter";
String TEST_DELAY = "testDelay";
}

View File

@@ -0,0 +1,67 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transformer.util;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
*/
@Deprecated
public class Util
{
/**
* Safely converts a {@link String} to an {@link Integer}
*
* @param param String to be converted
* @return Null if param is null or converted value as {@link Integer}
*/
public static Integer stringToInteger(final String param)
{
return param == null ? null : Integer.parseInt(param);
}
/**
* Safely converts a {@link String} to a {@link Boolean}
*
* @param param String to be converted
* @return Null if param is null or converted value as {@link Boolean}
*/
public static Boolean stringToBoolean(final String param)
{
return param == null ? null : Boolean.parseBoolean(param);
}
/**
* Safely converts a {@link String} to a {@link Long}
*
* @param param String to be converted
* @return Null if param is null or converted value as {@link Boolean}
*/
public static Long stringToLong(final String param)
{
return param == null ? null : Long.parseLong(param);
}
}

View File

@@ -0,0 +1,57 @@
---
spring:
servlet:
multipart:
max-file-size: 8192MB
max-request-size: 8192MB
activemq:
broker-url: ${ACTIVEMQ_URL:nio://localhost:61616}?jms.watchTopicAdvisories=false
user: ${ACTIVEMQ_USER:admin}
password: ${ACTIVEMQ_PASSWORD:admin}
pool:
enabled: true
max-connections: 20
jackson:
default-property-inclusion: non_empty
activemq:
url: ${ACTIVEMQ_URL:false}
server:
port: ${SERVER_PORT:8090}
error:
include-message: ALWAYS
logging:
level:
# org.alfresco.util.exec.RuntimeExec: debug
org.alfresco.transformer.LibreOfficeController: debug
org.alfresco.transformer.JodConverterSharedInstance: debug
org.alfresco.transformer.AlfrescoPdfRendererController: debug
org.alfresco.transformer.ImageMagickController: debug
org.alfresco.transformer.TikaController: debug
org.alfresco.transformer.MiscellaneousTransformersController: debug
org.alfresco.transform.common.TransformerDebug: debug
fileStoreUrl: ${FILE_STORE_URL:http://localhost:8099/alfresco/api/-default-/private/sfs/versions/1/file}
jms-listener:
concurrency: ${JMS_LISTENER_CONCURRENCY:1-10}
management:
endpoints:
web:
exposure:
include:
- metrics
- prometheus
- health
metrics:
enable[http]: false
enable[logback]: false
enable[tomcat]: false
enable[jvm.classes]: false
container:
name: ${HOSTNAME:t-engine}

View File

@@ -0,0 +1,22 @@
<html xmlns:th="http://www.thymeleaf.org">
<body>
<div>
<div th:if="${message} == null">
<h2>Error Page</h2>
</div>
<div th:if="${message}">
<h2 th:text="${#strings.substring(message,0,#strings.indexOf(message,' - '))} + ' Error Page'"></h2>
<h3 th:text="${#strings.substring(message,#strings.indexOf(message,' - ')+3)}"></h3>
</div>
<p th:text="${status} + ' - ' + ${error}"></p>
</div>
<div>
<br/>
<a href="/">Test Transformation</a>
<a href="/log">Log entries</a>
</div>
</body>
</html>

View File

@@ -0,0 +1,43 @@
<html xmlns:th="http://www.thymeleaf.org">
<body>
<div>
<h2 th:text="${title}"></h2>
<div th:if="${log}">
<table>
<tr>
<th>Id</th>
<th>Time</th>
<th>Status Code</th>
<th>Duration (ms)</th>
<th>Source</th>
<th></th>
<th>Target</th>
<th></th>
<th>Options</th>
<th>Message</th>
</tr>
<tr th:each="entry : ${log}">
<td th:text="${entry.id}"></td>
<td th:text="${#dates.format(entry.date, 'HH:mm:ss')}"></td>
<td th:text="${entry.statusCode}"></td>
<td th:text="${entry.duration}"></td>
<td th:text="${entry.source}"></td>
<td th:text="${entry.sourceSize}"></td>
<td th:text="${entry.target}"></td>
<td th:text="${entry.targetSize}"></td>
<td th:text="${entry.options}"></td>
<td th:text="${entry.message}"></td>
</tr>
</table>
</div>
</div>
<div>
<br/>
<a href="/">Test Transformation</a>
<a href="/transform/config?configVersion=9999">Transformer Config</a>
</div>
</body>
</html>

View File

@@ -0,0 +1,180 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transformer;
import static org.alfresco.transform.common.RequestParamMap.DIRECT_ACCESS_URL;
import static org.alfresco.transform.common.RequestParamMap.ENDPOINT_TRANSFORM;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.springframework.http.HttpMethod.POST;
import static org.springframework.http.MediaType.MULTIPART_FORM_DATA;
import static org.springframework.test.util.AssertionErrors.assertTrue;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.web.client.TestRestTemplate;
import org.springframework.boot.web.server.LocalServerPort;
import org.springframework.http.HttpEntity;
import org.springframework.http.HttpHeaders;
import org.springframework.http.ResponseEntity;
import org.springframework.util.LinkedMultiValueMap;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
*
* Super class for testing controllers with a server. Includes tests for the AbstractTransformerController itself.
* Note: Currently uses json rather than HTML as json is returned by this spring boot test harness.
*/
@Deprecated
public abstract class AbstractHttpRequestTest
{
@LocalServerPort
private int port;
@Autowired
private TestRestTemplate restTemplate;
protected abstract String getTransformerName();
protected abstract String getSourceExtension();
@Test
public void testPageExists()
{
String result = restTemplate.getForObject("http://localhost:" + port + "/", String.class);
String title = getTransformerName() + ' ' + "Test Transformation";
assertTrue("\"" + title + "\" should be part of the page title", result.contains(title));
}
@Test
public void logPageExists()
{
String result = restTemplate.getForObject("http://localhost:" + port + "/log",
String.class);
String title = getTransformerName() + ' ' + "Log";
assertTrue("\"" + title + "\" should be part of the page title", result.contains(title));
}
@Test
public void errorPageExists()
{
String result = restTemplate.getForObject("http://localhost:" + port + "/error",
String.class);
String title = getTransformerName() + ' ' + "Error Page";
assertTrue("\"" + title + "\" should be part of the page title",
result.contains("Error Page"));
}
@Test
public void noFileError()
{
LinkedMultiValueMap<String, Object> parameters = new LinkedMultiValueMap<>();
parameters.add("targetExtension", ".tmp");
assertTransformError(false,
getTransformerName() + " - Required request part 'file' is not present",
parameters);
}
@Test
public void noTargetExtensionError()
{
assertMissingParameter("targetExtension");
}
private void assertMissingParameter(String name)
{
assertTransformError(true,
getTransformerName() + " - Request parameter '" + name + "' is missing", null);
}
protected void assertTransformError(boolean addFile,
String errorMessage,
LinkedMultiValueMap<String, Object> additionalParams)
{
LinkedMultiValueMap<String, Object> parameters = new LinkedMultiValueMap<>();
if (addFile)
{
parameters.add("file",
new org.springframework.core.io.ClassPathResource("quick." + getSourceExtension()));
}
if (additionalParams != null)
{
parameters.addAll(additionalParams);
}
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MULTIPART_FORM_DATA);
HttpEntity<LinkedMultiValueMap<String, Object>> entity = new HttpEntity<>(parameters,
headers);
sendTranformationRequest(entity, errorMessage);
}
@Test
public void httpTransformRequestDirectAccessUrlNotFoundTest()
{
String directUrl = "https://expired/direct/access/url";
LinkedMultiValueMap<String, Object> parameters = new LinkedMultiValueMap<>();
parameters.add("targetExtension", ".tmp");
parameters.add(DIRECT_ACCESS_URL, directUrl);
assertTransformError(false,
getTransformerName() + " - Direct Access Url not found.",
parameters);
}
protected void sendTranformationRequest(
final HttpEntity<LinkedMultiValueMap<String, Object>> entity, final String errorMessage)
{
final ResponseEntity<String> response = restTemplate.exchange(ENDPOINT_TRANSFORM, POST, entity,
String.class, "");
assertEquals(errorMessage, getErrorMessage(response.getBody()));
}
// Strip out just the error message from the returned json content body
// Had been expecting the Error page to be returned, but we end up with the json in this test harness.
// Is correct if run manually, so not worrying too much about this.
private String getErrorMessage(String content)
{
String message = "";
int i = content.indexOf("\"message\":\"");
if (i != -1)
{
int j = content.indexOf("\",\"path\":", i);
if (j != -1)
{
message = content.substring(i + 11, j);
}
}
return message;
}
}

View File

@@ -0,0 +1,137 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transformer;
import static java.text.MessageFormat.format;
import static org.alfresco.transformer.EngineClient.sendTRequest;
import static org.alfresco.transform.common.Mimetype.MIMETYPE_METADATA_EXTRACT;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.fail;
import static org.springframework.http.HttpStatus.OK;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.io.Serializable;
import java.util.HashMap;
import java.util.Map;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.springframework.core.io.Resource;
import org.springframework.http.ResponseEntity;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
*
* Super class of metadata integration tests. Sub classes should provide the following:
* <p>
* <ul>
* <li>A method providing a
* Stream of test files: {@code public static Stream<TestFileInfo> engineTransformations()}; </li>
* <li> Provide expected json files (&lt;sourceFilename>"_metadata.json") as resources on the classpath.</li>
* <li> Override the method {@code testTransformation(TestFileInfo testFileInfo)} such that it calls
* the super method as a {@code @ParameterizedTest} for example:</li> </ul>
* <pre>
* &#64;ParameterizedTest
*
* &#64;MethodSource("engineTransformations")
*
* &#64;Override
* public void testTransformation(TestFileInfo testFileInfo)
*
* {
* super.testTransformation(TestFileInfo testFileInfo)
* }
* </pre>
*
* @author adavis
* @author dedwards
*/
@Deprecated
public abstract class AbstractMetadataExtractsIT
{
private static final String ENGINE_URL = "http://localhost:8090";
// These are normally variable, hence the lowercase.
private static final String targetMimetype = MIMETYPE_METADATA_EXTRACT;
private static final String targetExtension = "json";
private final ObjectMapper jsonObjectMapper = new ObjectMapper();
public void testTransformation(TestFileInfo testFileInfo)
{
final String sourceMimetype = testFileInfo.getMimeType();
final String sourceFile = testFileInfo.getPath();
final String descriptor = format("Transform ({0}, {1} -> {2}, {3})",
sourceFile, sourceMimetype, targetMimetype, targetExtension);
try
{
final ResponseEntity<Resource> response = sendTRequest(ENGINE_URL, sourceFile,
sourceMimetype, targetMimetype, targetExtension);
assertEquals(OK, response.getStatusCode(), descriptor);
String metadataFilename = sourceFile + "_metadata.json";
Map<String, Serializable> actualMetadata = readMetadata(response.getBody().getInputStream());
File actualMetadataFile = new File(metadataFilename);
jsonObjectMapper.writerWithDefaultPrettyPrinter().writeValue(actualMetadataFile, actualMetadata);
Map<String, Serializable> expectedMetadata = readExpectedMetadata(metadataFilename, actualMetadataFile);
assertEquals(expectedMetadata, actualMetadata,
sourceFile+": The metadata did not match the expected value. It has been saved in "+actualMetadataFile.getAbsolutePath());
actualMetadataFile.delete();
}
catch (Exception e)
{
e.printStackTrace();
fail(descriptor + " exception: " + e.getMessage());
}
}
private Map<String, Serializable> readExpectedMetadata(String filename, File actualMetadataFile) throws IOException
{
try (InputStream inputStream = this.getClass().getClassLoader().getResourceAsStream(filename))
{
if (inputStream == null)
{
fail("The expected metadata file "+filename+" did not exist.\n"+
"The actual metadata has been saved in "+actualMetadataFile.getAbsoluteFile());
}
return readMetadata(inputStream);
}
}
private Map<String, Serializable> readMetadata(InputStream inputStream) throws IOException
{
TypeReference<HashMap<String, Serializable>> typeRef = new TypeReference<HashMap<String, Serializable>>() {};
return jsonObjectMapper.readValue(inputStream, typeRef);
}
}

View File

@@ -0,0 +1,77 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transformer;
import static org.junit.jupiter.api.Assertions.assertEquals;
import javax.jms.Queue;
import org.alfresco.transform.client.model.TransformReply;
import org.alfresco.transform.client.model.TransformRequest;
import org.apache.activemq.command.ActiveMQQueue;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.jms.core.JmsTemplate;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
*
* @author Lucian Tuca
* created on 15/01/2019
*/
@Deprecated
@SpringBootTest(properties = {"activemq.url=nio://localhost:61616"})
public abstract class AbstractQueueTransformServiceIT
{
@Autowired
private Queue engineRequestQueue;
@Autowired
private JmsTemplate jmsTemplate;
private final ActiveMQQueue testingQueue = new ActiveMQQueue(
"org.alfresco.transform.engine.IT");
@Test
public void queueTransformServiceIT()
{
TransformRequest request = buildRequest();
jmsTemplate.convertAndSend(engineRequestQueue, request, m -> {
m.setJMSCorrelationID(request.getRequestId());
m.setJMSReplyTo(testingQueue);
return m;
});
this.jmsTemplate.setReceiveTimeout(1_000);
TransformReply reply = (TransformReply) this.jmsTemplate.receiveAndConvert(testingQueue);
assertEquals(request.getRequestId(), reply.getRequestId());
}
protected abstract TransformRequest buildRequest();
}

View File

@@ -0,0 +1,615 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transformer;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import org.alfresco.transform.client.model.InternalContext;
import org.alfresco.transform.client.model.TransformReply;
import org.alfresco.transform.client.model.TransformRequest;
import org.alfresco.transform.config.SupportedSourceAndTarget;
import org.alfresco.transform.config.TransformConfig;
import org.alfresco.transform.config.TransformOption;
import org.alfresco.transform.config.TransformOptionGroup;
import org.alfresco.transform.config.TransformOptionValue;
import org.alfresco.transform.config.Transformer;
import org.alfresco.transform.messages.TransformStack;
import org.alfresco.transform.registry.TransformServiceRegistry;
import org.alfresco.transformer.clients.AlfrescoSharedFileStoreClient;
import org.alfresco.transformer.model.FileRefEntity;
import org.alfresco.transformer.model.FileRefResponse;
import org.alfresco.transformer.probes.ProbeTestTransform;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.test.mock.mockito.MockBean;
import org.springframework.boot.test.mock.mockito.SpyBean;
import org.springframework.core.io.ClassPathResource;
import org.springframework.mock.web.MockMultipartFile;
import org.springframework.test.util.ReflectionTestUtils;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.ResultActions;
import org.springframework.test.web.servlet.request.MockHttpServletRequestBuilder;
import org.springframework.test.web.servlet.request.MockMvcRequestBuilders;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.net.URL;
import java.nio.channels.FileChannel;
import java.nio.file.Files;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.UUID;
import static java.nio.file.StandardCopyOption.REPLACE_EXISTING;
import static org.alfresco.transform.common.RequestParamMap.DIRECT_ACCESS_URL;
import static org.alfresco.transform.common.RequestParamMap.ENDPOINT_TRANSFORM;
import static org.alfresco.transform.common.RequestParamMap.ENDPOINT_TRANSFORM_CONFIG;
import static org.alfresco.transform.common.RequestParamMap.ENDPOINT_TRANSFORM_CONFIG_LATEST;
import static org.hamcrest.Matchers.containsString;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.when;
import static org.springframework.http.HttpHeaders.ACCEPT;
import static org.springframework.http.HttpHeaders.CONTENT_TYPE;
import static org.springframework.http.HttpStatus.BAD_REQUEST;
import static org.springframework.http.HttpStatus.CREATED;
import static org.springframework.http.HttpStatus.INTERNAL_SERVER_ERROR;
import static org.springframework.http.HttpStatus.OK;
import static org.springframework.http.MediaType.APPLICATION_JSON_VALUE;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.content;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.header;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
*
* Super class for testing controllers without a server. Includes tests for the AbstractTransformerController itself.
*/
@Deprecated
public abstract class AbstractTransformerControllerTest
{
@TempDir // added as part of ATS-702 to allow test resources to be read from the imported jar files to prevent test resource duplication
public File tempDir;
@Autowired
protected MockMvc mockMvc;
@Autowired
protected ObjectMapper objectMapper;
@MockBean
protected AlfrescoSharedFileStoreClient alfrescoSharedFileStoreClient;
@SpyBean
protected TransformServiceRegistry transformRegistry;
@Value("${transform.core.version}")
private String coreVersion;
protected String sourceExtension;
protected String targetExtension;
protected String sourceMimetype;
protected String targetMimetype;
protected HashMap<String, String> options = new HashMap<>();
protected MockMultipartFile sourceFile;
protected String expectedOptions;
protected String expectedSourceSuffix;
protected Long expectedTimeout = 0L;
protected byte[] expectedSourceFileBytes;
/**
* The expected result. Taken resting target quick file's bytes.
*
* Note: These checks generally don't work on Windows (Mac and Linux are okay). Possibly to do with byte order
* loading.
*/
protected byte[] expectedTargetFileBytes;
// Called by sub class
protected abstract void mockTransformCommand(String sourceExtension,
String targetExtension, String sourceMimetype,
boolean readTargetFileBytes) throws IOException;
protected abstract AbstractTransformerController getController();
protected abstract void updateTransformRequestWithSpecificOptions(
TransformRequest transformRequest);
/**
* This method ends up being the core of the mock.
* It copies content from an existing file in the resources folder to the desired location
* in order to simulate a successful transformation.
*
* @param actualTargetExtension Requested extension.
* @param testFile The test file (transformed) - basically the result.
* @param targetFile The location where the content from the testFile should be copied
* @throws IOException in case of any errors.
*/
void generateTargetFileFromResourceFile(String actualTargetExtension, File testFile,
File targetFile) throws IOException
{
if (testFile != null)
{
try (var inputStream = new FileInputStream(testFile);
var outputStream = new FileOutputStream(targetFile))
{
FileChannel source = inputStream.getChannel();
FileChannel target = outputStream.getChannel();
target.transferFrom(source, 0, source.size());
} catch (Exception e)
{
throw e;
}
}
else
{
testFile = getTestFile("quick." + actualTargetExtension, false);
if (testFile != null)
{
try (var inputStream = new FileInputStream(testFile);
var outputStream = new FileOutputStream(targetFile))
{
FileChannel source = inputStream.getChannel();
FileChannel target = outputStream.getChannel();
target.transferFrom(source, 0, source.size());
} catch (Exception e)
{
throw e;
}
}
}
}
protected byte[] readTestFile(String extension) throws IOException
{
return Files.readAllBytes(getTestFile("quick." + extension, true).toPath());
}
protected File getTestFile(String testFilename, boolean required) throws IOException
{
File testFile = null;
ClassLoader classLoader = getClass().getClassLoader();
URL testFileUrl = classLoader.getResource(testFilename);
if (required && testFileUrl == null)
{
throw new IOException("The test file " + testFilename +
" does not exist in the resources directory");
}
// added as part of ATS-702 to allow test resources to be read from the imported jar files to prevent test resource duplication
if (testFileUrl!=null)
{
// Each use of the tempDir should result in a unique directory being used
testFile = new File(tempDir, testFilename);
Files.copy(classLoader.getResourceAsStream(testFilename), testFile.toPath(),REPLACE_EXISTING);
}
return testFileUrl == null ? null : testFile;
}
protected MockHttpServletRequestBuilder mockMvcRequest(String url, MockMultipartFile sourceFile, String... params)
{
if (sourceFile == null)
{
return mockMvcRequestWithoutMockMultipartFile(url, params);
}
else
{
return mockMvcRequestWithMockMultipartFile(url, sourceFile, params);
}
}
private MockHttpServletRequestBuilder mockMvcRequestWithoutMockMultipartFile(String url,
String... params)
{
MockHttpServletRequestBuilder builder = MockMvcRequestBuilders.multipart(ENDPOINT_TRANSFORM);
if (params.length % 2 != 0)
{
throw new IllegalArgumentException("each param should have a name and value.");
}
for (int i = 0; i < params.length; i += 2)
{
builder = builder.param(params[i], params[i + 1]);
}
return builder;
}
private MockHttpServletRequestBuilder mockMvcRequestWithMockMultipartFile(String url, MockMultipartFile sourceFile,
String... params)
{
MockHttpServletRequestBuilder builder = MockMvcRequestBuilders.multipart(ENDPOINT_TRANSFORM).file(
sourceFile);
if (params.length % 2 != 0)
{
throw new IllegalArgumentException("each param should have a name and value.");
}
for (int i = 0; i < params.length; i += 2)
{
builder = builder.param(params[i], params[i + 1]);
}
return builder;
}
protected TransformRequest createTransformRequest(String sourceFileRef, File sourceFile)
{
TransformRequest transformRequest = new TransformRequest();
transformRequest.setRequestId("1");
transformRequest.setSchema(1);
transformRequest.setClientData("Alfresco Digital Business Platform");
transformRequest.setTransformRequestOptions(options);
transformRequest.setSourceReference(sourceFileRef);
transformRequest.setSourceExtension(sourceExtension);
transformRequest.setSourceMediaType(sourceMimetype);
transformRequest.setSourceSize(sourceFile.length());
transformRequest.setTargetExtension(targetExtension);
transformRequest.setTargetMediaType(targetMimetype);
transformRequest.setInternalContext(InternalContext.initialise(null));
transformRequest.getInternalContext().getMultiStep().setInitialRequestId("123");
transformRequest.getInternalContext().getMultiStep().setInitialSourceMediaType(sourceMimetype);
TransformStack.setInitialTransformRequestOptions(transformRequest.getInternalContext(), options);
TransformStack.setInitialSourceReference(transformRequest.getInternalContext(), sourceFileRef);
TransformStack.addTransformLevel(transformRequest.getInternalContext(),
TransformStack.levelBuilder(TransformStack.PIPELINE_FLAG)
.withStep("transformerName", sourceMimetype, targetMimetype));
return transformRequest;
}
@Test
public void simpleTransformTest() throws Exception
{
mockMvc.perform(
mockMvcRequest(ENDPOINT_TRANSFORM, sourceFile, "targetExtension", targetExtension))
.andExpect(status().is(OK.value()))
.andExpect(content().bytes(expectedTargetFileBytes))
.andExpect(header().string("Content-Disposition",
"attachment; filename*= UTF-8''quick." + targetExtension));
}
@Test
public void testDelayTest() throws Exception
{
long start = System.currentTimeMillis();
mockMvc.perform(mockMvcRequest(ENDPOINT_TRANSFORM, sourceFile, "targetExtension", targetExtension,
"testDelay", "400"))
.andExpect(status().is(OK.value()))
.andExpect(content().bytes(expectedTargetFileBytes))
.andExpect(header().string("Content-Disposition",
"attachment; filename*= UTF-8''quick." + targetExtension));
long ms = System.currentTimeMillis() - start;
System.out.println("Transform incluing test delay was " + ms);
assertTrue(ms >= 400, "Delay sending the result back was too small " + ms);
assertTrue(ms <= 500,"Delay sending the result back was too big " + ms);
}
@Test
public void noTargetFileTest() throws Exception
{
mockMvc.perform(mockMvcRequest(ENDPOINT_TRANSFORM, sourceFile, "targetExtension", "xxx"))
.andExpect(status().is(INTERNAL_SERVER_ERROR.value()));
}
@Test
// Looks dangerous but is okay as we only use the final filename
public void dotDotSourceFilenameTest() throws Exception
{
sourceFile = new MockMultipartFile("file", "../quick." + sourceExtension, sourceMimetype,
expectedSourceFileBytes);
mockMvc.perform(
mockMvcRequest(ENDPOINT_TRANSFORM, sourceFile, "targetExtension", targetExtension))
.andExpect(status().is(OK.value()))
.andExpect(content().bytes(expectedTargetFileBytes))
.andExpect(header().string("Content-Disposition",
"attachment; filename*= UTF-8''quick." + targetExtension));
}
@Test
// Is okay, as the target filename is built up from the whole source filename and the targetExtension
public void noExtensionSourceFilenameTest() throws Exception
{
sourceFile = new MockMultipartFile("file", "../quick", sourceMimetype,
expectedSourceFileBytes);
mockMvc.perform(
mockMvcRequest(ENDPOINT_TRANSFORM, sourceFile, "targetExtension", targetExtension))
.andExpect(status().is(OK.value()))
.andExpect(content().bytes(expectedTargetFileBytes))
.andExpect(header().string("Content-Disposition",
"attachment; filename*= UTF-8''quick." + targetExtension));
}
@Test
// Invalid file name that ends in /
public void badSourceFilenameTest() throws Exception
{
sourceFile = new MockMultipartFile("file", "abc/", sourceMimetype, expectedSourceFileBytes);
mockMvc.perform(
mockMvcRequest(ENDPOINT_TRANSFORM, sourceFile, "targetExtension", targetExtension))
.andExpect(status().is(BAD_REQUEST.value()))
.andExpect(status().reason(containsString("The source filename was not supplied")));
}
@Test
public void blankSourceFilenameTest() throws Exception
{
sourceFile = new MockMultipartFile("file", "", sourceMimetype, expectedSourceFileBytes);
mockMvc.perform(
mockMvcRequest(ENDPOINT_TRANSFORM, sourceFile, "targetExtension", targetExtension))
.andExpect(status().is(BAD_REQUEST.value()));
}
@Test
public void noTargetExtensionTest() throws Exception
{
mockMvc.perform(mockMvcRequest(ENDPOINT_TRANSFORM, sourceFile))
.andExpect(status().is(BAD_REQUEST.value()))
.andExpect(status().reason(
containsString("Request parameter 'targetExtension' is missing")));
}
@Test
public void calculateMaxTime() throws Exception
{
ProbeTestTransform probeTestTransform = getController().getProbeTestTransform();
probeTestTransform.setLivenessPercent(110);
long[][] values = new long[][]{
{5000, 0, Long.MAX_VALUE}, // 1st transform is ignored
{1000, 1000, 2100}, // 1000 + 1000*1.1
{3000, 2000, 4200}, // 2000 + 2000*1.1
{2000, 2000, 4200},
{6000, 3000, 6300},
{8000, 4000, 8400},
{4444, 4000, 8400}, // no longer in the first few, so normal and max times don't change
{5555, 4000, 8400}
};
for (long[] v : values)
{
long time = v[0];
long expectedNormalTime = v[1];
long expectedMaxTime = v[2];
probeTestTransform.calculateMaxTime(time, true);
assertEquals(expectedNormalTime, probeTestTransform.getNormalTime());
assertEquals(expectedMaxTime, probeTestTransform.getMaxTime());
}
}
@Test
public void testEmptyPojoTransform() throws Exception
{
// Transformation Request POJO
TransformRequest transformRequest = new TransformRequest();
// Serialize and call the transformer
String tr = objectMapper.writeValueAsString(transformRequest);
String transformationReplyAsString = mockMvc
.perform(MockMvcRequestBuilders
.post(ENDPOINT_TRANSFORM)
.header(ACCEPT, APPLICATION_JSON_VALUE)
.header(CONTENT_TYPE, APPLICATION_JSON_VALUE)
.content(tr))
.andExpect(status().is(BAD_REQUEST.value()))
.andReturn().getResponse().getContentAsString();
TransformReply transformReply = objectMapper.readValue(transformationReplyAsString,
TransformReply.class);
// Assert the reply
assertEquals(BAD_REQUEST.value(), transformReply.getStatus());
}
/**
*
* @return transformer specific engine config name
*/
public String getEngineConfigName()
{
return "engine_config.json";
}
@Test
public void testGetTransformConfigInfo() throws Exception
{
TransformConfig expectedTransformConfig = objectMapper
.readValue(getTestFile(getEngineConfigName(), true),
TransformConfig.class);
expectedTransformConfig.getTransformers().forEach(transformer -> {
transformer.setCoreVersion(coreVersion);
transformer.getTransformOptions().add(DIRECT_ACCESS_URL);
});
expectedTransformConfig.getTransformOptions().put(DIRECT_ACCESS_URL, Set.of(new TransformOptionValue(false, DIRECT_ACCESS_URL)));
ReflectionTestUtils.setField(transformRegistry, "engineConfig",
new ClassPathResource(getEngineConfigName()));
String response = mockMvc
.perform(MockMvcRequestBuilders.get(ENDPOINT_TRANSFORM_CONFIG_LATEST))
.andExpect(status().is(OK.value()))
.andExpect(header().string(CONTENT_TYPE, APPLICATION_JSON_VALUE))
.andReturn().getResponse().getContentAsString();
TransformConfig transformConfig = objectMapper.readValue(response, TransformConfig.class);
assertEquals(expectedTransformConfig, transformConfig);
}
@Test
// Test for case when T-Router or Repository is a version that does not expect it
public void testGetTransformConfigInfoExcludingCoreVersion() throws Exception
{
TransformConfig expectedTransformConfig = objectMapper
.readValue(getTestFile(getEngineConfigName(), true),
TransformConfig.class);
ReflectionTestUtils.setField(transformRegistry, "engineConfig",
new ClassPathResource(getEngineConfigName()));
String response = mockMvc
.perform(MockMvcRequestBuilders.get(ENDPOINT_TRANSFORM_CONFIG))
.andExpect(status().is(OK.value()))
.andExpect(header().string(CONTENT_TYPE, APPLICATION_JSON_VALUE))
.andReturn().getResponse().getContentAsString();
TransformConfig transformConfig = objectMapper.readValue(response, TransformConfig.class);
assertEquals(expectedTransformConfig, transformConfig);
}
@Test
public void testGetInfoFromConfigWithDuplicates() throws Exception
{
TransformConfig expectedResult = buildCompleteTransformConfig();
ReflectionTestUtils.setField(transformRegistry, "engineConfig",
new ClassPathResource("engine_config_with_duplicates.json"));
String response = mockMvc
.perform(MockMvcRequestBuilders.get(ENDPOINT_TRANSFORM_CONFIG))
.andExpect(status().is(OK.value()))
.andExpect(header().string(CONTENT_TYPE, APPLICATION_JSON_VALUE))
.andReturn().getResponse().getContentAsString();
TransformConfig transformConfig = objectMapper.readValue(response, TransformConfig.class);
assertNotNull(transformConfig);
assertEquals(expectedResult, transformConfig);
assertEquals(3, transformConfig.getTransformOptions().get("engineXOptions").size());
assertEquals(1,
transformConfig.getTransformers().get(0).getSupportedSourceAndTargetList().size());
assertEquals(1,
transformConfig.getTransformers().get(0).getTransformOptions().size());
}
@Test
public void testGetInfoFromConfigWithEmptyTransformOptions() throws Exception
{
Transformer transformer = buildTransformer("application/pdf", "image/png");
TransformConfig expectedResult = new TransformConfig();
expectedResult.setTransformers(ImmutableList.of(transformer));
ReflectionTestUtils.setField(transformRegistry, "engineConfig",
new ClassPathResource("engine_config_incomplete.json"));
String response = mockMvc
.perform(MockMvcRequestBuilders.get(ENDPOINT_TRANSFORM_CONFIG))
.andExpect(status().is(OK.value()))
.andExpect(header().string(CONTENT_TYPE, APPLICATION_JSON_VALUE))
.andReturn().getResponse().getContentAsString();
TransformConfig transformConfig = objectMapper.readValue(response, TransformConfig.class);
assertNotNull(transformConfig);
assertEquals(expectedResult, transformConfig);
}
@Test
public void testGetInfoFromConfigWithNoTransformOptions() throws Exception
{
Transformer transformer = buildTransformer("application/pdf", "image/png");
transformer.setTransformerName("engineX");
TransformConfig expectedResult = new TransformConfig();
expectedResult.setTransformers(ImmutableList.of(transformer));
ReflectionTestUtils.setField(transformRegistry, "engineConfig",
new ClassPathResource("engine_config_no_transform_options.json"));
String response = mockMvc
.perform(MockMvcRequestBuilders.get(ENDPOINT_TRANSFORM_CONFIG))
.andExpect(status().is(OK.value()))
.andExpect(header().string(CONTENT_TYPE, APPLICATION_JSON_VALUE))
.andReturn().getResponse().getContentAsString();
TransformConfig transformConfig = objectMapper.readValue(response, TransformConfig.class);
assertNotNull(transformConfig);
assertEquals(expectedResult, transformConfig);
}
private TransformConfig buildCompleteTransformConfig()
{
TransformConfig expectedResult = new TransformConfig();
Set<TransformOption> transformOptionGroup = ImmutableSet.of(
new TransformOptionValue(false, "cropGravity"));
Set<TransformOption> transformOptions = ImmutableSet.of(
new TransformOptionValue(false, "page"),
new TransformOptionValue(false, "width"),
new TransformOptionGroup(false, transformOptionGroup));
Map<String, Set<TransformOption>> transformOptionsMap = ImmutableMap.of("engineXOptions",
transformOptions);
Transformer transformer = buildTransformer("application/pdf", "image/png", "engineXOptions",
"engineX");
List<Transformer> transformers = ImmutableList.of(transformer);
expectedResult.setTransformOptions(transformOptionsMap);
expectedResult.setTransformers(transformers);
return expectedResult;
}
private Transformer buildTransformer(String sourceMediaType, String targetMediaType,
String transformOptions, String transformerName)
{
Transformer transformer = buildTransformer(sourceMediaType, targetMediaType);
transformer.setTransformerName(transformerName);
transformer.setTransformOptions(ImmutableSet.of(transformOptions));
return transformer;
}
private Transformer buildTransformer(String sourceMediaType, String targetMediaType)
{
Set<SupportedSourceAndTarget> supportedSourceAndTargetList = ImmutableSet.of(
SupportedSourceAndTarget.builder()
.withSourceMediaType(sourceMediaType)
.withTargetMediaType(targetMediaType)
.build());
Transformer transformer = new Transformer();
transformer.setSupportedSourceAndTargetList(supportedSourceAndTargetList);
return transformer;
}
}

View File

@@ -0,0 +1,72 @@
/*
* Copyright 2015-2022 Alfresco Software, Ltd. All rights reserved.
*
* License rights for this program may be obtained from Alfresco Software, Ltd.
* pursuant to a written agreement and any use of this program without such an
* agreement is prohibited.
*/
package org.alfresco.transformer;
import static java.util.Collections.emptyMap;
import static org.alfresco.transform.common.RequestParamMap.ENDPOINT_TRANSFORM;
import static org.springframework.http.MediaType.MULTIPART_FORM_DATA;
import java.util.Map;
import org.springframework.core.io.ClassPathResource;
import org.springframework.core.io.Resource;
import org.springframework.http.HttpEntity;
import org.springframework.http.HttpHeaders;
import org.springframework.http.ResponseEntity;
import org.springframework.util.LinkedMultiValueMap;
import org.springframework.util.MultiValueMap;
import org.springframework.web.client.RestTemplate;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
*
* @author Cezar Leahu
*/
@Deprecated
public class EngineClient
{
private static final RestTemplate REST_TEMPLATE = new RestTemplate();
public static ResponseEntity<Resource> sendTRequest(
final String engineUrl, final String sourceFile,
final String sourceMimetype, final String targetMimetype, final String targetExtension)
{
return sendTRequest(engineUrl, sourceFile, sourceMimetype, targetMimetype, targetExtension,
emptyMap());
}
public static ResponseEntity<Resource> sendTRequest(
final String engineUrl, final String sourceFile,
final String sourceMimetype, final String targetMimetype, final String targetExtension,
final Map<String, String> transformOptions)
{
final HttpHeaders headers = new HttpHeaders();
headers.setContentType(MULTIPART_FORM_DATA);
//headers.setAccept(ImmutableList.of(MULTIPART_FORM_DATA));
final MultiValueMap<String, Object> body = new LinkedMultiValueMap<>();
body.add("file", new ClassPathResource(sourceFile));
if (sourceMimetype != null && !sourceMimetype.trim().isEmpty())
{
body.add("sourceMimetype", sourceMimetype);
}
if (targetMimetype != null && !targetMimetype.trim().isEmpty())
{
body.add("targetMimetype", targetMimetype);
}
if (targetExtension != null && !targetExtension.trim().isEmpty())
{
body.add("targetExtension", targetExtension);
}
transformOptions.forEach(body::add);
final HttpEntity<MultiValueMap<String, Object>> entity = new HttpEntity<>(body, headers);
return REST_TEMPLATE.postForEntity(engineUrl + ENDPOINT_TRANSFORM, entity, Resource.class);
}
}

View File

@@ -0,0 +1,244 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transformer;
import static org.mockito.Mockito.doReturn;
import static org.mockito.Mockito.doThrow;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.verifyNoMoreInteractions;
import static org.springframework.http.HttpStatus.BAD_REQUEST;
import static org.springframework.http.HttpStatus.CREATED;
import static org.springframework.http.HttpStatus.INTERNAL_SERVER_ERROR;
import javax.jms.Destination;
import javax.jms.JMSException;
import javax.jms.Message;
import org.alfresco.transform.client.model.TransformReply;
import org.alfresco.transform.client.model.TransformRequest;
import org.alfresco.transformer.messaging.TransformMessageConverter;
import org.alfresco.transformer.messaging.TransformReplySender;
import org.apache.activemq.command.ActiveMQObjectMessage;
import org.apache.activemq.command.ActiveMQQueue;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.MockitoAnnotations;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.jms.support.converter.MessageConversionException;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
*/
@Deprecated
public class QueueTransformServiceTest
{
@Mock
private TransformController transformController;
@Mock
private TransformMessageConverter transformMessageConverter;
@Mock
private TransformReplySender transformReplySender;
@InjectMocks
private QueueTransformService queueTransformService;
@BeforeEach
public void setup()
{
MockitoAnnotations.initMocks(this);
}
@Test
public void testWhenReceiveNullMessageThenStopFlow()
{
queueTransformService.receive(null);
verifyNoMoreInteractions(transformController);
verifyNoMoreInteractions(transformMessageConverter);
verifyNoMoreInteractions(transformReplySender);
}
@Test
public void testWhenReceiveMessageWithNoReplyToQueueThenStopFlow()
{
queueTransformService.receive(new ActiveMQObjectMessage());
verifyNoMoreInteractions(transformController);
verifyNoMoreInteractions(transformMessageConverter);
verifyNoMoreInteractions(transformReplySender);
}
@Test
public void testConvertMessageReturnsNullThenReplyWithInternalServerError() throws JMSException
{
ActiveMQObjectMessage msg = new ActiveMQObjectMessage();
msg.setCorrelationId("1234");
ActiveMQQueue destination = new ActiveMQQueue();
msg.setJMSReplyTo(destination);
TransformReply reply = TransformReply
.builder()
.withStatus(INTERNAL_SERVER_ERROR.value())
.withErrorDetails(
"JMS exception during T-Request deserialization of message with correlationID "
+ msg.getCorrelationId() + ": null")
.build();
doReturn(null).when(transformMessageConverter).fromMessage(msg);
queueTransformService.receive(msg);
verify(transformMessageConverter).fromMessage(msg);
verify(transformReplySender).send(destination, reply, msg.getCorrelationId());
verifyNoMoreInteractions(transformController);
}
@Test
public void testConvertMessageThrowsMessageConversionExceptionThenReplyWithBadRequest()
throws JMSException
{
ActiveMQObjectMessage msg = new ActiveMQObjectMessage();
msg.setCorrelationId("1234");
ActiveMQQueue destination = new ActiveMQQueue();
msg.setJMSReplyTo(destination);
TransformReply reply = TransformReply
.builder()
.withStatus(BAD_REQUEST.value())
.withErrorDetails(
"Message conversion exception during T-Request deserialization of message with correlationID"
+ msg.getCorrelationId() + ": null")
.build();
doThrow(MessageConversionException.class).when(transformMessageConverter).fromMessage(msg);
queueTransformService.receive(msg);
verify(transformMessageConverter).fromMessage(msg);
verify(transformReplySender).send(destination, reply, msg.getCorrelationId());
verifyNoMoreInteractions(transformController);
}
@Test
public void testConvertMessageThrowsJMSExceptionThenReplyWithInternalServerError()
throws JMSException
{
ActiveMQObjectMessage msg = new ActiveMQObjectMessage();
msg.setCorrelationId("1234");
ActiveMQQueue destination = new ActiveMQQueue();
msg.setJMSReplyTo(destination);
TransformReply reply = TransformReply
.builder()
.withStatus(INTERNAL_SERVER_ERROR.value())
.withErrorDetails(
"JMSException during T-Request deserialization of message with correlationID " +
msg.getCorrelationId() + ": null")
.build();
doThrow(JMSException.class).when(transformMessageConverter).fromMessage(msg);
queueTransformService.receive(msg);
verify(transformMessageConverter).fromMessage(msg);
verify(transformReplySender).send(destination, reply, msg.getCorrelationId());
verifyNoMoreInteractions(transformController);
}
@Test
public void testWhenReceiveValidTransformRequestThenReplyWithSuccess() throws JMSException
{
ActiveMQObjectMessage msg = new ActiveMQObjectMessage();
ActiveMQQueue destination = new ActiveMQQueue();
msg.setJMSReplyTo(destination);
TransformRequest request = new TransformRequest();
TransformReply reply = TransformReply
.builder()
.withStatus(CREATED.value())
.build();
doReturn(request).when(transformMessageConverter).fromMessage(msg);
doReturn(new ResponseEntity<>(reply, HttpStatus.valueOf(reply.getStatus())))
.when(transformController).transform(request, null);
queueTransformService.receive(msg);
verify(transformMessageConverter).fromMessage(msg);
verify(transformController).transform(request, null);
verify(transformReplySender).send(destination, reply);
}
@Test
public void testWhenJMSExceptionOnMessageIsThrownThenStopFlow() throws JMSException
{
Message msg = mock(Message.class);
doThrow(JMSException.class).when(msg).getJMSReplyTo();
queueTransformService.receive(msg);
verifyNoMoreInteractions(transformController);
verifyNoMoreInteractions(transformMessageConverter);
verifyNoMoreInteractions(transformReplySender);
}
@Test
public void testWhenExceptionOnCorrelationIdIsThrownThenContinueFlowWithNullCorrelationId()
throws JMSException
{
Message msg = mock(Message.class);
Destination destination = mock(Destination.class);
doThrow(JMSException.class).when(msg).getJMSCorrelationID();
doReturn(destination).when(msg).getJMSReplyTo();
TransformRequest request = new TransformRequest();
TransformReply reply = TransformReply
.builder()
.withStatus(CREATED.value())
.build();
doReturn(request).when(transformMessageConverter).fromMessage(msg);
doReturn(new ResponseEntity<>(reply, HttpStatus.valueOf(reply.getStatus())))
.when(transformController).transform(request, null);
queueTransformService.receive(msg);
verify(transformMessageConverter).fromMessage(msg);
verify(transformController).transform(request, null);
verify(transformReplySender).send(destination, reply);
}
}

View File

@@ -0,0 +1,76 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transformer;
import java.util.Objects;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
*
* Source & Target media type pair
*
* @author Cezar Leahu
*/
@Deprecated
public class SourceTarget
{
public final String source;
public final String target;
private SourceTarget(final String source, final String target)
{
this.source = source;
this.target = target;
}
@Override
public boolean equals(Object o)
{
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
SourceTarget that = (SourceTarget) o;
return Objects.equals(source, that.source) &&
Objects.equals(target, that.target);
}
@Override
public int hashCode()
{
return Objects.hash(source, target);
}
@Override
public String toString()
{
return source + '|' + target;
}
public static SourceTarget of(final String source, final String target)
{
return new SourceTarget(source, target);
}
}

View File

@@ -0,0 +1,87 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2005 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transformer;
/**
* @deprecated will be removed in a future release. Replaced by alfresco-base-t-engine.
* @author Cezar Leahu
*/
@Deprecated
public class TestFileInfo
{
private final String mimeType;
private final String extension;
private final String path;
private final boolean exactMimeType;
public TestFileInfo(final String mimeType, final String extension, final String path,
final boolean exactMimeType)
{
this.mimeType = mimeType;
this.extension = extension;
this.path = path;
this.exactMimeType = exactMimeType;
}
public String getMimeType()
{
return mimeType;
}
public String getExtension()
{
return extension;
}
public String getPath()
{
return path;
}
public boolean isExactMimeType()
{
return exactMimeType;
}
public static TestFileInfo testFile(final String mimeType, final String extension,
final String path, final boolean exactMimeType)
{
return new TestFileInfo(mimeType, extension, path, exactMimeType);
}
public static TestFileInfo testFile(final String mimeType, final String extension,
final String path)
{
return new TestFileInfo(mimeType, extension, path, false);
}
@Override
public String toString()
{
return path;
}
}

View File

@@ -0,0 +1,22 @@
{
"transformOptions": {
"engineXOptions": [
{"value": {"name": "page"}},
{"value": {"name": "width"}},
{"group": {"transformOptions": [
{"value": {"name": "cropGravity"}}
]}}
]
},
"transformers": [
{
"transformerName": "engineX",
"supportedSourceAndTargetList": [
{"sourceMediaType": "application/pdf", "targetMediaType": "image/png" }
],
"transformOptions": [
"engineXOptions"
]
}
]
}

View File

@@ -0,0 +1,10 @@
{
"transformOptions": {},
"transformers": [
{
"supportedSourceAndTargetList": [
{"sourceMediaType": "application/pdf", "targetMediaType": "image/png" }
]
}
]
}

View File

@@ -0,0 +1,10 @@
{
"transformers": [
{
"transformerName": "engineX",
"supportedSourceAndTargetList": [
{"sourceMediaType": "application/pdf", "targetMediaType": "image/png" }
]
}
]
}

View File

@@ -0,0 +1,26 @@
{
"transformOptions": {
"engineXOptions": [
{"value": {"name": "page"}},
{"value": {"name": "page"}},
{"value": {"name": "width"}},
{"group": {"transformOptions": [
{"value": {"name": "cropGravity"}}
]}}
]
},
"transformers": [
{
"transformerName": "engineX",
"supportedSourceAndTargetList": [
{"sourceMediaType": "application/pdf", "targetMediaType": "image/png" },
{"sourceMediaType": "application/pdf", "targetMediaType": "image/png" },
{"sourceMediaType": "application/pdf", "targetMediaType": "image/png" }
],
"transformOptions": [
"engineXOptions",
"engineXOptions"
]
}
]
}