mirror of
https://github.com/Alfresco/alfresco-transform-core.git
synced 2025-08-14 17:58:27 +00:00
Batch changes from of review comments - there are more
This commit is contained in:
@@ -53,7 +53,7 @@ src/main/java/org/alfresco/transformer/Application.java
|
||||
~~~
|
||||
|
||||
* *TransformerName*Controller.java - A [Spring Boot](https://projects.spring.io/spring-boot/) Controller that
|
||||
extends TransformController to handel requests. It implements a few methods including *transformImpl*
|
||||
extends TransformController to handle requests. It implements a few methods including *transformImpl*
|
||||
which is intended to perform the actual transform. Generally the transform is done in a sub class of
|
||||
*JavaExecutor*, when a Java library is being used or *AbstractCommandExecutor*, when an external process is used.
|
||||
Both are sub interfaces of *Transformer*.
|
||||
|
@@ -1,4 +1,4 @@
|
||||
## T-Engines
|
||||
# T-Engines
|
||||
|
||||
The t-engines provide the basic transform operations. The Transform Service
|
||||
provides a common base for the communication with other components. It is
|
||||
@@ -13,7 +13,7 @@ A t-engine groups together one of more Transformers. Each Transformer
|
||||
transformations from one MIME Type to another with a common set of
|
||||
t-options.
|
||||
|
||||
~~~
|
||||
~~~yaml
|
||||
0010 my-t-engine
|
||||
Transformer 1
|
||||
mimetype A -> mimetype B
|
||||
@@ -32,7 +32,7 @@ t-options.
|
||||
...
|
||||
~~~
|
||||
|
||||
### Endpoints
|
||||
## Endpoints
|
||||
|
||||
* `POST /transform` to perform a transform. There are two forms:
|
||||
* For asynchronous transforms: Perform a transform using a
|
||||
@@ -57,4 +57,4 @@ t-options.
|
||||
* `GET /version` provides a String message to be included in client debug
|
||||
messages.
|
||||
* `GET /ready` used by Kubernetes as a ready probe.
|
||||
* `GET /live` used by Kubernetes as a ready probe.
|
||||
* `GET /live` used by Kubernetes as a ready probe.
|
||||
|
@@ -1,4 +1,4 @@
|
||||
## T-Engine configuration
|
||||
# T-Engine configuration
|
||||
|
||||
Each t-engine provides an endpoint that returns t-config that defines what
|
||||
it supports. The t-router and t-engines may also have external t-config files.
|
||||
@@ -22,7 +22,7 @@ files) are specified in Spring Boot properties or such as
|
||||
The following is a simple t-config file from an example Hello World
|
||||
t-engine.
|
||||
|
||||
~~~
|
||||
~~~json
|
||||
{
|
||||
"transformOptions":
|
||||
{
|
||||
@@ -69,7 +69,7 @@ t-engine.
|
||||
from text to HTML and we have limited the source file size, to avoid
|
||||
transforming files that clearly don't contain names.
|
||||
|
||||
### Transform pipelines
|
||||
## Transform pipelines
|
||||
|
||||
Transforms may be combined in a pipeline to form a new transformer, where
|
||||
the output from one becomes the input to the next and so on. The t-config
|
||||
@@ -86,7 +86,7 @@ message in the body. This is then transformed back into a text file. This
|
||||
example contains just one pipeline transformer, but many may be defined
|
||||
in the same file.
|
||||
|
||||
~~~
|
||||
~~~json
|
||||
{
|
||||
"transformers": [
|
||||
{
|
||||
@@ -127,14 +127,14 @@ in the same file.
|
||||
* **transformOptions** A list of references to options required by the
|
||||
pipeline transformer.
|
||||
|
||||
### Failover transforms
|
||||
## Failover transforms
|
||||
|
||||
A failover transform, simply provides a list of transforms to be attempted
|
||||
one after another until one succeeds. For example, you may have a fast
|
||||
transform that is able to handle a limited set of transforms and another
|
||||
that is slower but handles all cases.
|
||||
|
||||
~~~
|
||||
~~~json
|
||||
{
|
||||
"transformers": [
|
||||
{
|
||||
@@ -163,14 +163,15 @@ that is slower but handles all cases.
|
||||
* **transformOptions** A list of references to options required by the
|
||||
pipeline transformer.
|
||||
|
||||
### Overriding transforms
|
||||
## Overriding transforms
|
||||
|
||||
It is possible to override a previously defined transform definition. The
|
||||
following example removes most of the supported source to target media
|
||||
types from the standard `"libreoffice"` transform. It also changes the
|
||||
max size and priority of others. This is not something you would normally
|
||||
want to do.
|
||||
~~~
|
||||
|
||||
~~~json
|
||||
{
|
||||
"transformers": [
|
||||
{
|
||||
@@ -188,7 +189,7 @@ want to do.
|
||||
}
|
||||
~~~
|
||||
|
||||
### Removing a transformer
|
||||
## Removing a transformer
|
||||
|
||||
To discard a previous transformer definition include its name in the
|
||||
optional `"removeTransformers"` list. You might want to do this if you
|
||||
@@ -197,7 +198,7 @@ contains no alternatives), or you wish to temporarily remove it. The
|
||||
following example removes two transformers before processing any other
|
||||
configuration in the same T-Engine or pipeline file.
|
||||
|
||||
~~~
|
||||
~~~json
|
||||
{
|
||||
"removeTransformers" : [
|
||||
"libreoffice",
|
||||
@@ -207,7 +208,7 @@ configuration in the same T-Engine or pipeline file.
|
||||
}
|
||||
~~~
|
||||
|
||||
### Overriding the supportedSourceAndTargetList
|
||||
## Overriding the supportedSourceAndTargetList
|
||||
|
||||
Rather than totally override an existing transform definition, it is
|
||||
generally simpler to modify the `"supportedSourceAndTargetList"` by adding
|
||||
@@ -219,7 +220,8 @@ in the original, the same change is not needed in a second place. The
|
||||
following example adds one transform, removes two others and changes
|
||||
the `"priority"` and `"maxSourceSizeBytes"` of another. This is done before
|
||||
processing any other configuration in the same T-Engine or pipeline file.
|
||||
~~~
|
||||
|
||||
~~~json
|
||||
{
|
||||
"addSupported": [
|
||||
{
|
||||
@@ -255,7 +257,7 @@ processing any other configuration in the same T-Engine or pipeline file.
|
||||
}
|
||||
~~~
|
||||
|
||||
### Default maxSourceSizeBytes and priority values
|
||||
## Default maxSourceSizeBytes and priority values
|
||||
|
||||
When defining `"supportedSourceAndTargetList"` elements the `"priority"`
|
||||
and `"maxSourceSizeBytes"` are optional and normally have the default
|
||||
@@ -283,7 +285,7 @@ be used in preference.
|
||||
|
||||
Defaults values are only applied after all t-config has been read.
|
||||
|
||||
~~~
|
||||
~~~json
|
||||
{
|
||||
"supportedDefaults": [
|
||||
{
|
||||
@@ -305,4 +307,4 @@ Defaults values are only applied after all t-config has been read.
|
||||
]
|
||||
...
|
||||
}
|
||||
~~~
|
||||
~~~
|
||||
|
@@ -1,4 +1,4 @@
|
||||
## Transform specific code
|
||||
# Transform specific code
|
||||
|
||||
To create a new t-engine an author uses a base t-engine (a Spring Boot
|
||||
application) and implements the following interfaces. An implementation of
|
||||
@@ -11,7 +11,7 @@ one format to another without really worrying about all the plumbing.
|
||||
Typically, the transform specific code uses a 3rd party library or an
|
||||
external executable which needs to be added to the Docker image.
|
||||
|
||||
~~~
|
||||
~~~java
|
||||
package org.alfresco.transform;
|
||||
|
||||
import org.alfresco.transform.config.TransformConfig;
|
||||
@@ -49,7 +49,7 @@ public interface TransformEngine
|
||||
|
||||
implementations of the following interface provide the actual transform code.
|
||||
|
||||
~~~
|
||||
~~~java
|
||||
package org.alfresco.transform;
|
||||
|
||||
import java.io.InputStream;
|
||||
@@ -81,7 +81,7 @@ The implementation of the following interface is provided by the t-base,
|
||||
allows the `CustomTransformer` to interact with the base t-engine. The
|
||||
creation of Files is discouraged as it is better not to leave files on disk.
|
||||
|
||||
~~~
|
||||
~~~java
|
||||
package org.alfresco.transform.base;
|
||||
|
||||
import java.io.File;
|
||||
@@ -137,4 +137,4 @@ public interface TransformManager
|
||||
* @throws IOException if there was a problem sending the response.
|
||||
OutputStream respondWithFragment(Integer index);
|
||||
}
|
||||
~~~
|
||||
~~~
|
||||
|
@@ -1,4 +1,4 @@
|
||||
## Transformer selection strategy
|
||||
# Transformer selection strategy
|
||||
|
||||
The TransformRegistry uses t-config to choose which Transformer will be
|
||||
used. A transformer definition contains a supported list of source and
|
||||
@@ -6,12 +6,13 @@ target Media Types. This is used for the most basic selection. It is further
|
||||
refined by checking that the definition also supports transform options (the
|
||||
parameters) that have been supplied in a transform request.
|
||||
|
||||
~~~
|
||||
~~~text
|
||||
Transformer 1 defines options: Op1, Op2
|
||||
Transformer 2 defines options: Op1, Op2, Op3, Op4
|
||||
|
||||
Transform request provides values for options: Op2, Op3
|
||||
~~~
|
||||
|
||||
If we assume both transformers support the required source and target Media
|
||||
Types, Transformer 2 will be selected because it knows about all the supplied
|
||||
options. The definition may also specify that some options are required or
|
||||
@@ -24,4 +25,4 @@ the transforms consuming too many resources.
|
||||
|
||||
The configuration may also specify a priority which will be used in
|
||||
Transformer selection if there are a number of possible transformers. The
|
||||
highest priority is the one with the lowest number.
|
||||
highest priority is the one with the lowest number.
|
||||
|
@@ -1,10 +1,10 @@
|
||||
## TransformerDebug
|
||||
# TransformerDebug
|
||||
|
||||
In addition to any normal logging, the t-engines, t-router and t-client also
|
||||
use the `TransformerDebug` class to provide request based logging. The
|
||||
following is an example from Alfresco after the upload of a `docx` file.
|
||||
|
||||
~~~
|
||||
~~~text
|
||||
163 docx json AGM 2016 - Masters report.docx 14.8 KB -- metadataExtract -- TransformService
|
||||
163 workspace://SpacesStore/0db3a665-328d-4437-85ed-56b753cf19c8 1563306426
|
||||
163 docx json 14.8 KB -- metadataExtract -- PoiMetadataExtractor
|
||||
@@ -43,4 +43,4 @@ to `pdf` and then another pipeline to convert to `png`. The last step
|
||||
options.
|
||||
|
||||
If requested, log information is passed back in the TransformReply's
|
||||
clientData.
|
||||
clientData.
|
||||
|
@@ -35,7 +35,8 @@ implementations.
|
||||
|
||||
The `TransformEngineName` is important if the config from multiple T-Engines is being
|
||||
combined as they are sorted by name.
|
||||
```
|
||||
|
||||
```java
|
||||
package org.alfresco.transform.example;
|
||||
|
||||
import com.google.common.collect.ImmutableMap;
|
||||
@@ -81,7 +82,8 @@ public class HelloTransformEngine implements TransformEngine
|
||||
```
|
||||
|
||||
**Example CustomTransformer**
|
||||
```
|
||||
|
||||
```java
|
||||
package org.alfresco.transform.example;
|
||||
|
||||
import org.alfresco.transform.base.CustomTransformer;
|
||||
@@ -120,6 +122,7 @@ public class HelloTransformer implements CustomTransformer
|
||||
```
|
||||
|
||||
**Example T-Config** `resources/hello_engine_config.json`
|
||||
|
||||
```json
|
||||
{
|
||||
"transformOptions": {
|
||||
@@ -145,12 +148,14 @@ public class HelloTransformer implements CustomTransformer
|
||||
**Example properties** `resources/application-default.yaml`
|
||||
|
||||
As can be seen the following defines a default which can be overridden by an environment variable.
|
||||
|
||||
```yaml
|
||||
queue:
|
||||
engineRequestQueue: ${TRANSFORM_ENGINE_REQUEST_QUEUE:org.alfresco.transform.engine.libreoffice.acs}
|
||||
```
|
||||
|
||||
**Example ProbeTransform test file** `resources/probe.txt`
|
||||
```json
|
||||
|
||||
```text
|
||||
Jane
|
||||
```
|
@@ -46,8 +46,6 @@ import org.springframework.scheduling.annotation.EnableScheduling;
|
||||
@EnableRetry
|
||||
public class Application
|
||||
{
|
||||
private static final Logger logger = LoggerFactory.getLogger(Application.class);
|
||||
|
||||
@Value("${container.name}")
|
||||
private String containerName;
|
||||
|
||||
|
@@ -97,13 +97,19 @@ import static org.springframework.http.MediaType.MULTIPART_FORM_DATA_VALUE;
|
||||
@Controller
|
||||
public class TransformController
|
||||
{
|
||||
|
||||
private static final Logger logger = LoggerFactory.getLogger(TransformController.class);
|
||||
|
||||
private static final String MODEL_TITLE = "title";
|
||||
private static final String MODEL_PROXY_PATH_PREFIX = "proxyPathPrefix";
|
||||
private static final String MODEL_MESSAGE = "message";
|
||||
|
||||
@Autowired(required = false)
|
||||
private List<TransformEngine> transformEngines;
|
||||
@Autowired
|
||||
private TransformServiceRegistry transformRegistry;
|
||||
@Autowired TransformHandler transformHandler;
|
||||
@Autowired
|
||||
TransformHandler transformHandler;
|
||||
@Autowired
|
||||
private String coreVersion;
|
||||
@Autowired
|
||||
@@ -119,11 +125,11 @@ public class TransformController
|
||||
if (transformEngines != null)
|
||||
{
|
||||
transformEngine = getTransformEngine();
|
||||
logger.info("TransformEngine: " + transformEngine.getTransformEngineName());
|
||||
logger.info("TransformEngine: {}", transformEngine.getTransformEngineName());
|
||||
transformEngines.stream()
|
||||
.filter(te -> te != transformEngine)
|
||||
.filter(transformEngineFromStream -> transformEngineFromStream != transformEngine)
|
||||
.sorted(Comparator.comparing(TransformEngine::getTransformEngineName))
|
||||
.map(transformEngine -> " "+transformEngine.getTransformEngineName()).forEach(logger::info);
|
||||
.map(sortedTransformEngine -> " "+sortedTransformEngine.getTransformEngineName()).forEach(logger::info);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -133,7 +139,7 @@ public class TransformController
|
||||
// CustomTransform code from many t-engines into a single t-engine. In this case, there should be a wrapper
|
||||
// TransformEngine (it has no TransformConfig of its own).
|
||||
return transformEngines.stream()
|
||||
.filter(transformEngine -> transformEngine.getTransformConfig() == null)
|
||||
.filter(transformEngineFromStream -> transformEngineFromStream.getTransformConfig() == null)
|
||||
.findFirst()
|
||||
.orElse(transformEngines.get(0));
|
||||
}
|
||||
@@ -171,8 +177,8 @@ public class TransformController
|
||||
@GetMapping(ENDPOINT_ROOT)
|
||||
public String test(Model model)
|
||||
{
|
||||
model.addAttribute("title", getSimpleTransformEngineName() + " Test Page");
|
||||
model.addAttribute("proxyPathPrefix", getPathPrefix());
|
||||
model.addAttribute(MODEL_TITLE, getSimpleTransformEngineName() + " Test Page");
|
||||
model.addAttribute(MODEL_PROXY_PATH_PREFIX, getPathPrefix());
|
||||
TransformConfig transformConfig = ((TransformRegistry) transformRegistry).getTransformConfig();
|
||||
transformConfig = setOrClearCoreVersion(transformConfig, 0);
|
||||
model.addAttribute("transformOptions", optionLister.getOptionNames(transformConfig.getTransformOptions()));
|
||||
@@ -185,8 +191,8 @@ public class TransformController
|
||||
@GetMapping(ENDPOINT_ERROR)
|
||||
public String error(Model model)
|
||||
{
|
||||
model.addAttribute("title", getSimpleTransformEngineName() + " Error Page");
|
||||
model.addAttribute("proxyPathPrefix", getPathPrefix());
|
||||
model.addAttribute(MODEL_TITLE, getSimpleTransformEngineName() + " Error Page");
|
||||
model.addAttribute(MODEL_PROXY_PATH_PREFIX, getPathPrefix());
|
||||
return "error"; // display error.html
|
||||
}
|
||||
|
||||
@@ -196,8 +202,8 @@ public class TransformController
|
||||
@GetMapping(ENDPOINT_LOG)
|
||||
String log(Model model)
|
||||
{
|
||||
model.addAttribute("title", getSimpleTransformEngineName() + " Log Entries");
|
||||
model.addAttribute("proxyPathPrefix", getPathPrefix());
|
||||
model.addAttribute(MODEL_TITLE, getSimpleTransformEngineName() + " Log Entries");
|
||||
model.addAttribute(MODEL_PROXY_PATH_PREFIX, getPathPrefix());
|
||||
Collection<LogEntry> log = LogEntry.getLog();
|
||||
if (!log.isEmpty())
|
||||
{
|
||||
@@ -342,9 +348,9 @@ public class TransformController
|
||||
response.sendError(e.getStatus().value(), message);
|
||||
|
||||
ModelAndView mav = new ModelAndView();
|
||||
mav.addObject("title", getSimpleTransformEngineName() + " Error Page");
|
||||
mav.addObject("proxyPathPrefix", getPathPrefix());
|
||||
mav.addObject("message", message);
|
||||
mav.addObject(MODEL_TITLE, getSimpleTransformEngineName() + " Error Page");
|
||||
mav.addObject(MODEL_PROXY_PATH_PREFIX, getPathPrefix());
|
||||
mav.addObject(MODEL_MESSAGE, message);
|
||||
mav.setViewName("error"); // display error.html
|
||||
return mav;
|
||||
}
|
||||
|
@@ -221,7 +221,7 @@ public class FileManager
|
||||
|
||||
final File systemTempDir = new File(systemTempDirPath);
|
||||
final File tempDir = new File(systemTempDir, dirName);
|
||||
if (!tempDir.exists() && !tempDir.mkdirs() && !tempDir.exists())
|
||||
if (!tempDir.exists() && !tempDir.mkdirs())
|
||||
{
|
||||
throw new RuntimeException("Failed to create temp directory: " + tempDir);
|
||||
}
|
||||
|
@@ -1,4 +1,9 @@
|
||||
<html xmlns:th="http://www.thymeleaf.org">
|
||||
<!DOCTYPE html>
|
||||
<html xmlns:th="http://www.thymeleaf.org" xml:lang="en">
|
||||
<head>
|
||||
<title th:text="${title}"/>
|
||||
</head>
|
||||
|
||||
<body>
|
||||
|
||||
<div>
|
||||
|
@@ -1,4 +1,9 @@
|
||||
<html xmlns:th="http://www.thymeleaf.org">
|
||||
<!DOCTYPE html>
|
||||
<html xmlns:th="http://www.thymeleaf.org" xml:lang="en">
|
||||
<head>
|
||||
<title th:text="${title}"/>
|
||||
</head>
|
||||
|
||||
<body>
|
||||
|
||||
<div>
|
||||
|
@@ -1,4 +1,9 @@
|
||||
<html xmlns:th="http://www.thymeleaf.org">
|
||||
<!DOCTYPE html>
|
||||
<html xmlns:th="http://www.thymeleaf.org" xml:lang="en">
|
||||
<head>
|
||||
<title th:text="${title}"/>
|
||||
</head>
|
||||
|
||||
<body>
|
||||
<div>
|
||||
<h2 th:text="${title}"></h2>
|
||||
|
Reference in New Issue
Block a user