Alan Davis babe26b0ba
HXENG-64 refactor ATS (#657)
Refactor to clean up packages in the t-model and to introduce a simpler to implement t-engine base.

The new t-engines (tika, imagemagick, libreoffice, pdfrenderer, misc, aio, aspose) and t-router may be used in combination with older components as the API between the content Repo and between components has not changed. As far as possible the same artifacts are created (the -boot projects no longer exist). They may be used with older ACS repo versions.

The main changes to look for are:
* The introduction of TransformEngine and CustomTransformer interfaces to be implemented.
* The removal in t-engines and t-router of the Controller, Application, test template page, Controller tests and application config, as this is all now done by the t-engine base package.
* The t-router now extends the t-engine base, which also reduced the amount of duplicate code.
* The t-engine base provides the test page, which includes drop downs of known transform options. The t-router is able to use pipeline and failover transformers. This was not possible to do previously as the router had no test UI.
* Resources including licenses are automatically included in the all-in-one t-engine, from the individual t-engines. They just need to be added as dependencies in the pom. 
* The ugly code in the all-in-one t-engine and misc t-engine to pick transformers has gone, as they are now just selected by the transformRegistry.
* The way t-engines respond to http or message queue transform requests has been combined (eliminates the similar but different code that existed before).
* The t-engine base now uses InputStream and OutputStream rather than Files by default. As a result it will be simpler to avoid writing content to a temporary location.
* A number of the Tika and Misc CustomTransforms no longer use Files.
* The original t-engine base still exists so customers can continue to create custom t-engines the way they have done previously. the project has just been moved into a folder called deprecated.
* The folder structure has changed. The long "alfresco-transform-..." names have given way to shorter easier to read and type names.
* The t-engine project structure now has a single project rather than two. 
* The previous config values still exist, but there are now a new set for config values for in files with names that don't misleadingly imply they only contain pipeline of routing information. 
* The concept of 'routing' has much less emphasis in class names as the code just uses the transformRegistry. 
* TransformerConfig may now be read as json or yaml. The restrictions about what could be specified in yaml has gone.
* T-engines and t-router may use transform config from files. Previously it was just the t-router.
* The POC code to do with graphs of possible routes has been removed.
* All master branch changes have been merged in.
* The concept of a single transform request which results in multiple responses (e.g. images from a video) has been added to the core processing of requests in the t-engine base.
* Many SonarCloud linter fixes.
2022-09-14 13:40:19 +01:00

5.1 KiB

Common base code for T-Engines

This project provides a common base for T-Engines and supersedes the original base.

This project provides a base Spring Boot application (as a jar) to which transform specific code may be added. It includes actions such as communication between components and logging.

For more details on build a custom T-Engine and T-Config, please refer to the docs in ACS Packaging, including:

Overview

A T-Engine project which extends this base is expected to provide the following:

  • An implementation of the TransformEngine interface to describe the T-Engine.
  • Implementations of the CustomTransformer interface with the actual transform code.
  • An application-default.yaml file to define a unique name for the message queue to the T-Engine.

The TransformEngine and CustomTransformer implementations should have an @Component annotation and be in or below theorg.alfresco.transform package, so that they will be discovered by the base T-Engine.

The TransformEngine.getTransformConfig() method typically reads a json file. The names in the config should match the names returned by the CustomTransformer implementations.

Example TransformEngine

The TransformEngineName is important if the config from multiple T-Engines is being combined as they are sorted by name.

package org.alfresco.transform.example;

import com.google.common.collect.ImmutableMap;
import org.alfresco.transform.base.TransformEngine;
import org.alfresco.transform.base.probes.ProbeTransform;
import org.alfresco.transform.config.reader.TransformConfigResourceReader;
import org.alfresco.transform.config.TransformConfig;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

@Component
public class HelloTransformEngine implements TransformEngine
{
    @Autowired
    private TransformConfigResourceReader transformConfigResourceReader;

    @Override
    public String getTransformEngineName()
    {
        return "0200_hello";
    }

    @Override
    public String getStartupMessage()
    {
        return "Startup "+getTransformEngineName()+"\nNo 3rd party licenses";
    }

    @Override
    public TransformConfig getTransformConfig()
    {
        return transformConfigResourceReader.read("classpath:hello_engine_config.json");
    }

    @Override
    public ProbeTransform getProbeTransform()
    {
        return new ProbeTransform("probe.txt", "text/plain", "text/plain",
            ImmutableMap.of("sourceEncoding", "UTF-8", "language", "English"),
            11, 10, 150, 1024, 1, 60 * 2);
    }
}

Example CustomTransformer

package org.alfresco.transform.example;

import org.alfresco.transform.base.CustomTransformer;
import org.alfresco.transform.base.TransformManager;
import org.springframework.stereotype.Component;

import java.io.InputStream;
import java.io.OutputStream;
import java.util.Map;

@Component
public class HelloTransformer implements CustomTransformer
{
    @Override
    public String getTransformerName()
    {
        return "hello";
    }

    @Override
    public void transform(String sourceMimetype, InputStream inputStream, String targetMimetype,
            OutputStream outputStream, Map<String, String> transformOptions, TransformManager transformManager)
            throws Exception
    {
        String name = new String(inputStream.readAllBytes(), transformOptions.get("sourceEncoding"));
        String greeting = String.format(getGreeting(transformOptions.get("language")), name);
        byte[] bytes = greeting.getBytes(transformOptions.get("sourceEncoding"));
        outputStream.write(bytes, 0, bytes.length);
    }

    private String getGreeting(String language)
    {
        return "Hello %s";
    }
}

Example T-Config resources/hello_engine_config.json

{
  "transformOptions": {
    "helloOptions": [
      {"value": {"name": "language"}},
      {"value": {"name": "sourceEncoding"}}
    ]
  },
  "transformers": [
    {
      "transformerName": "hello",
      "supportedSourceAndTargetList": [
        {"sourceMediaType": "text/plain", "targetMediaType": "text/plain" }
      ],
      "transformOptions": [
        "helloOptions"
      ]
    }
  ]
}

Example properties resources/application-default.yaml

As can be seen the following defines a default which can be overridden by an environment variable.

queue:
  engineRequestQueue: ${TRANSFORM_ENGINE_REQUEST_QUEUE:org.alfresco.transform.engine.libreoffice.acs}

Example ProbeTransform test file resources/probe.txt

Jane