inteligr8/alfresco-transform-core
1
0
Fork 0
mirror of https://github.com/Alfresco/alfresco-transform-core.git synced 2025-07-31 17:38:33 +00:00
Code Issues Packages Projects Releases Wiki Activity

HXENG-64 refactor ATS (#657)

Browse Source 
Refactor to clean up packages in the t-model and to introduce a simpler to implement t-engine base.

The new t-engines (tika, imagemagick, libreoffice, pdfrenderer, misc, aio, aspose) and t-router may be used in combination with older components as the API between the content Repo and between components has not changed. As far as possible the same artifacts are created (the -boot projects no longer exist). They may be used with older ACS repo versions.

The main changes to look for are:
* The introduction of TransformEngine and CustomTransformer interfaces to be implemented.
* The removal in t-engines and t-router of the Controller, Application, test template page, Controller tests and application config, as this is all now done by the t-engine base package.
* The t-router now extends the t-engine base, which also reduced the amount of duplicate code.
* The t-engine base provides the test page, which includes drop downs of known transform options. The t-router is able to use pipeline and failover transformers. This was not possible to do previously as the router had no test UI.
* Resources including licenses are automatically included in the all-in-one t-engine, from the individual t-engines. They just need to be added as dependencies in the pom. 
* The ugly code in the all-in-one t-engine and misc t-engine to pick transformers has gone, as they are now just selected by the transformRegistry.
* The way t-engines respond to http or message queue transform requests has been combined (eliminates the similar but different code that existed before).
* The t-engine base now uses InputStream and OutputStream rather than Files by default. As a result it will be simpler to avoid writing content to a temporary location.
* A number of the Tika and Misc CustomTransforms no longer use Files.
* The original t-engine base still exists so customers can continue to create custom t-engines the way they have done previously. the project has just been moved into a folder called deprecated.
* The folder structure has changed. The long "alfresco-transform-..." names have given way to shorter easier to read and type names.
* The t-engine project structure now has a single project rather than two. 
* The previous config values still exist, but there are now a new set for config values for in files with names that don't misleadingly imply they only contain pipeline of routing information. 
* The concept of 'routing' has much less emphasis in class names as the code just uses the transformRegistry. 
* TransformerConfig may now be read as json or yaml. The restrictions about what could be specified in yaml has gone.
* T-engines and t-router may use transform config from files. Previously it was just the t-router.
* The POC code to do with graphs of possible routes has been removed.
* All master branch changes have been merged in.
* The concept of a single transform request which results in multiple responses (e.g. images from a video) has been added to the core processing of requests in the t-engine base.
* Many SonarCloud linter fixes.
This commit is contained in:
Alan Davis
2022-09-14 13:40:19 +01:00
committed by GitHub
parent ea83ef9ebc
commit babe26b0ba
652 changed files with 19479 additions and 18195 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
engines/example/.maven-dockerignore Normal file
View File

@@ -0,0 +1 @@
target/docker/

26
engines/example/Dockerfile Normal file
View File

@@ -0,0 +1,26 @@
FROM alfresco/alfresco-base-java:jre17-rockylinux8-202207110835
ARG EXIFTOOL_VERSION=12.25
ARG EXIFTOOL_FOLDER=Image-ExifTool-${EXIFTOOL_VERSION}
ARG EXIFTOOL_URL=https://nexus.alfresco.com/nexus/service/local/repositories/thirdparty/content/org/exiftool/image-exiftool/${EXIFTOOL_VERSION}/image-exiftool-${EXIFTOOL_VERSION}.tgz
ENV JAVA_OPTS=""
ARG GROUPNAME=Alfresco
ARG GROUPID=1000
ARG EXAMPLEUSERNAME=example
ARG USERID=33009
COPY target/${env.project_artifactId}-${env.project_version}.jar /usr/bin
RUN ln /usr/bin/${env.project_artifactId}-${env.project_version}.jar /usr/bin/${env.project_artifactId}.jar
RUN groupadd -g ${GROUPID} ${GROUPNAME} && \
useradd -u ${USERID} -G ${GROUPNAME} ${EXAMPLEUSERNAME} && \
chgrp -R ${GROUPNAME} /usr/bin/${env.project_artifactId}.jar
EXPOSE 8090
USER ${EXAMPLEUSERNAME}
ENTRYPOINT java $JAVA_OPTS -jar /usr/bin/${env.project_artifactId}.jar

300
engines/example/pom.xml Normal file
View File

@@ -0,0 +1,300 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<artifactId>alfresco-transform-example</artifactId>
<name>- Example</name>
<parent>
<groupId>org.alfresco</groupId>
<artifactId>alfresco-transform-core</artifactId>
<version>3.0.0-HXP-A10-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>
<properties>
<image.name>alfresco/alfresco-transform-example</image.name>
<image.registry>quay.io</image.registry>
<env.project_artifactId>${project.artifactId}</env.project_artifactId>
</properties>
<dependencies>
<dependency>
<groupId>org.alfresco</groupId>
<artifactId>alfresco-base-t-engine</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.alfresco</groupId>
<artifactId>alfresco-base-t-engine</artifactId>
<version>${project.version}</version>
<classifier>tests</classifier>
<type>test-jar</type>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-thymeleaf</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>com.vaadin.external.google</groupId>
<artifactId>android-json</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.dom4j</groupId>
<artifactId>dom4j</artifactId>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-junit-jupiter</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<mainClass>org.alfresco.transform.base.Application</mainClass>
</configuration>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>license-maven-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>test-jar</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
<profiles>
<profile>
<id>docker-it-setup</id>
<!-- raises an ActiveMq container for the Integration Tests -->
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
</plugin>
<plugin>
<groupId>io.fabric8</groupId>
<artifactId>docker-maven-plugin</artifactId>
<executions>
<execution>
<id>integration-tests</id>
<goals>
<goal>start</goal>
<goal>stop</goal>
</goals>
<configuration>
<images>
<image>
<name>alfresco/alfresco-activemq:5.17.1-jre11-rockylinux8</name>
<run>
<hostname>activemq</hostname>
<ports>
<port>8161:8161</port>
<port>5672:5672</port>
<port>61616:61616</port>
</ports>
<wait>
<log>Apache ActiveMQ 5.17.1 .* started</log>
<time>20000</time>
<kill>500</kill>
<shutdown>100</shutdown>
<exec>
<preStop>kill 1</preStop>
<preStop>kill -9 1</preStop>
</exec>
</wait>
</run>
</image>
<image>
<name>${image.name}:${image.tag}</name>
<run>
<ports>
<port>8090:8090</port>
</ports>
<wait>
<http>
<url>http://localhost:8090/transform/config</url>
<method>GET</method>
<status>200...299</status>
</http>
<time>300000</time>
<kill>500</kill>
<shutdown>100</shutdown>
<exec>
<preStop>kill 1</preStop>
<preStop>kill -9 1</preStop>
</exec>
</wait>
</run>
</image>
</images>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
<profile>
<id>local</id>
<build>
<plugins>
<plugin>
<groupId>io.fabric8</groupId>
<artifactId>docker-maven-plugin</artifactId>
<executions>
<execution>
<id>build-image</id>
<phase>package</phase>
<goals>
<goal>build</goal>
</goals>
<configuration>
<images>
<image>
<name>${image.name}:${image.tag}</name>
<build>
<contextDir>${project.basedir}</contextDir>
<buildOptions>
<squash>true</squash>
</buildOptions>
</build>
</image>
</images>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
<profile>
<id>internal</id>
<build>
<plugins>
<plugin>
<groupId>io.fabric8</groupId>
<artifactId>docker-maven-plugin</artifactId>
<configuration>
<images>
<!-- QuayIO image -->
<image>
<name>${image.name}:${image.tag}</name>
<registry>${image.registry}</registry>
<build>
<contextDir>${project.basedir}</contextDir>
<buildOptions>
<squash>true</squash>
</buildOptions>
</build>
</image>
<!-- DockerHub image -->
<image>
<name>${image.name}:${image.tag}</name>
<build>
<contextDir>${project.basedir}</contextDir>
<buildOptions>
<squash>true</squash>
</buildOptions>
</build>
</image>
</images>
</configuration>
<executions>
<execution>
<id>build-image</id>
<phase>package</phase>
<goals>
<goal>build</goal>
</goals>
</execution>
<execution>
<id>push-image</id>
<phase>install</phase>
<goals>
<goal>push</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
<profile>
<id>release</id>
<build>
<plugins>
<plugin>
<groupId>io.fabric8</groupId>
<artifactId>docker-maven-plugin</artifactId>
<configuration>
<images>
<!-- QuayIO image -->
<image>
<name>${image.name}:${project.version}</name>
<registry>${image.registry}</registry>
<build>
<contextDir>${project.basedir}</contextDir>
<buildOptions>
<squash>true</squash>
</buildOptions>
</build>
</image>
<!-- DockerHub image -->
<image>
<name>${image.name}:${project.version}</name>
<build>
<contextDir>${project.basedir}</contextDir>
<buildOptions>
<squash>true</squash>
</buildOptions>
</build>
</image>
</images>
</configuration>
<executions>
<execution>
<goals>
<goal>build</goal>
<goal>push</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>

68
engines/example/src/main/java/org/alfresco/transform/example/HelloTransformEngine.java Normal file
View File

@@ -0,0 +1,68 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2022 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.example;
import com.google.common.collect.ImmutableMap;
import org.alfresco.transform.base.TransformEngine;
import org.alfresco.transform.base.probes.ProbeTransform;
import org.alfresco.transform.config.reader.TransformConfigResourceReader;
import org.alfresco.transform.config.TransformConfig;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
@Component
public class HelloTransformEngine implements TransformEngine
{
@Autowired
private TransformConfigResourceReader transformConfigResourceReader;
@Override
public String getTransformEngineName()
{
return "0200 Hello";
}
@Override
public String getStartupMessage()
{
return "Startup "+getTransformEngineName()+"\nNo 3rd party licenses";
}
@Override
public TransformConfig getTransformConfig()
{
return transformConfigResourceReader.read("classpath:hello_engine_config.json");
}
@Override
public ProbeTransform getProbeTransform()
{
return new ProbeTransform("probe.txt", "text/plain", "text/plain",
ImmutableMap.of("sourceEncoding", "UTF-8", "language", "English"),
11, 10, 150, 1024, 1, 60 * 2);
}
}

61
engines/example/src/main/java/org/alfresco/transform/example/HelloTransformer.java Normal file
View File

@@ -0,0 +1,61 @@
/*
* #%L
* Alfresco Transform Core
* %%
* Copyright (C) 2022 - 2022 Alfresco Software Limited
* %%
* This file is part of the Alfresco software.
* -
* If the software was purchased under a paid Alfresco license, the terms of
* the paid license agreement will prevail. Otherwise, the software is
* provided under the following open source license terms:
* -
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
* -
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
* -
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
* #L%
*/
package org.alfresco.transform.example;
import org.alfresco.transform.base.CustomTransformer;
import org.alfresco.transform.base.TransformManager;
import org.springframework.stereotype.Component;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.Map;
@Component
public class HelloTransformer implements CustomTransformer
{
@Override
public String getTransformerName()
{
return "hello";
}
@Override
public void transform(String sourceMimetype, InputStream inputStream, String targetMimetype,
OutputStream outputStream, Map<String, String> transformOptions, TransformManager transformManager)
throws Exception
{
String name = new String(inputStream.readAllBytes(), transformOptions.get("sourceEncoding"));
String greeting = String.format(getGreeting(transformOptions.get("language")), name);
byte[] bytes = greeting.getBytes(transformOptions.get("sourceEncoding"));
outputStream.write(bytes, 0, bytes.length);
}
private String getGreeting(String language)
{
return "Hello %s";
}
}

2
engines/example/src/main/resources/application-default.yaml Normal file
View File

@@ -0,0 +1,2 @@
queue:
engineRequestQueue: ${TRANSFORM_ENGINE_REQUEST_QUEUE:org.alfresco.transform.engine.example.acs}

19
engines/example/src/main/resources/hello_engine_config.json Normal file
View File

@@ -0,0 +1,19 @@
{
"transformOptions": {
"helloOptions": [
{"value": {"name": "language"}},
{"value": {"name": "sourceEncoding"}}
]
},
"transformers": [
{
"transformerName": "hello",
"supportedSourceAndTargetList": [
{"sourceMediaType": "text/plain", "targetMediaType": "text/plain" }
],
"transformOptions": [
"helloOptions"
]
}
]
}

1
engines/example/src/main/resources/probe.txt Normal file
View File

@@ -0,0 +1 @@
Jane