Batch changes from of review comments - there are more

This commit is contained in:
alandavis
2022-09-07 15:12:19 +01:00
parent 1abbae8e5b
commit 6e85b860e9
13 changed files with 81 additions and 54 deletions

View File

@@ -53,7 +53,7 @@ src/main/java/org/alfresco/transformer/Application.java
~~~
* *TransformerName*Controller.java - A [Spring Boot](https://projects.spring.io/spring-boot/) Controller that
extends TransformController to handel requests. It implements a few methods including *transformImpl*
extends TransformController to handle requests. It implements a few methods including *transformImpl*
which is intended to perform the actual transform. Generally the transform is done in a sub class of
*JavaExecutor*, when a Java library is being used or *AbstractCommandExecutor*, when an external process is used.
Both are sub interfaces of *Transformer*.

View File

@@ -1,4 +1,4 @@
## T-Engines
# T-Engines
The t-engines provide the basic transform operations. The Transform Service
provides a common base for the communication with other components. It is
@@ -13,7 +13,7 @@ A t-engine groups together one of more Transformers. Each Transformer
transformations from one MIME Type to another with a common set of
t-options.
~~~
~~~yaml
0010 my-t-engine
Transformer 1
mimetype A -> mimetype B
@@ -32,7 +32,7 @@ t-options.
...
~~~
### Endpoints
## Endpoints
* `POST /transform` to perform a transform. There are two forms:
* For asynchronous transforms: Perform a transform using a

View File

@@ -1,4 +1,4 @@
## T-Engine configuration
# T-Engine configuration
Each t-engine provides an endpoint that returns t-config that defines what
it supports. The t-router and t-engines may also have external t-config files.
@@ -22,7 +22,7 @@ files) are specified in Spring Boot properties or such as
The following is a simple t-config file from an example Hello World
t-engine.
~~~
~~~json
{
"transformOptions":
{
@@ -69,7 +69,7 @@ t-engine.
from text to HTML and we have limited the source file size, to avoid
transforming files that clearly don't contain names.
### Transform pipelines
## Transform pipelines
Transforms may be combined in a pipeline to form a new transformer, where
the output from one becomes the input to the next and so on. The t-config
@@ -86,7 +86,7 @@ message in the body. This is then transformed back into a text file. This
example contains just one pipeline transformer, but many may be defined
in the same file.
~~~
~~~json
{
"transformers": [
{
@@ -127,14 +127,14 @@ in the same file.
* **transformOptions** A list of references to options required by the
pipeline transformer.
### Failover transforms
## Failover transforms
A failover transform, simply provides a list of transforms to be attempted
one after another until one succeeds. For example, you may have a fast
transform that is able to handle a limited set of transforms and another
that is slower but handles all cases.
~~~
~~~json
{
"transformers": [
{
@@ -163,14 +163,15 @@ that is slower but handles all cases.
* **transformOptions** A list of references to options required by the
pipeline transformer.
### Overriding transforms
## Overriding transforms
It is possible to override a previously defined transform definition. The
following example removes most of the supported source to target media
types from the standard `"libreoffice"` transform. It also changes the
max size and priority of others. This is not something you would normally
want to do.
~~~
~~~json
{
"transformers": [
{
@@ -188,7 +189,7 @@ want to do.
}
~~~
### Removing a transformer
## Removing a transformer
To discard a previous transformer definition include its name in the
optional `"removeTransformers"` list. You might want to do this if you
@@ -197,7 +198,7 @@ contains no alternatives), or you wish to temporarily remove it. The
following example removes two transformers before processing any other
configuration in the same T-Engine or pipeline file.
~~~
~~~json
{
"removeTransformers" : [
"libreoffice",
@@ -207,7 +208,7 @@ configuration in the same T-Engine or pipeline file.
}
~~~
### Overriding the supportedSourceAndTargetList
## Overriding the supportedSourceAndTargetList
Rather than totally override an existing transform definition, it is
generally simpler to modify the `"supportedSourceAndTargetList"` by adding
@@ -219,7 +220,8 @@ in the original, the same change is not needed in a second place. The
following example adds one transform, removes two others and changes
the `"priority"` and `"maxSourceSizeBytes"` of another. This is done before
processing any other configuration in the same T-Engine or pipeline file.
~~~
~~~json
{
"addSupported": [
{
@@ -255,7 +257,7 @@ processing any other configuration in the same T-Engine or pipeline file.
}
~~~
### Default maxSourceSizeBytes and priority values
## Default maxSourceSizeBytes and priority values
When defining `"supportedSourceAndTargetList"` elements the `"priority"`
and `"maxSourceSizeBytes"` are optional and normally have the default
@@ -283,7 +285,7 @@ be used in preference.
Defaults values are only applied after all t-config has been read.
~~~
~~~json
{
"supportedDefaults": [
{

View File

@@ -1,4 +1,4 @@
## Transform specific code
# Transform specific code
To create a new t-engine an author uses a base t-engine (a Spring Boot
application) and implements the following interfaces. An implementation of
@@ -11,7 +11,7 @@ one format to another without really worrying about all the plumbing.
Typically, the transform specific code uses a 3rd party library or an
external executable which needs to be added to the Docker image.
~~~
~~~java
package org.alfresco.transform;
import org.alfresco.transform.config.TransformConfig;
@@ -49,7 +49,7 @@ public interface TransformEngine
implementations of the following interface provide the actual transform code.
~~~
~~~java
package org.alfresco.transform;
import java.io.InputStream;
@@ -81,7 +81,7 @@ The implementation of the following interface is provided by the t-base,
allows the `CustomTransformer` to interact with the base t-engine. The
creation of Files is discouraged as it is better not to leave files on disk.
~~~
~~~java
package org.alfresco.transform.base;
import java.io.File;

View File

@@ -1,4 +1,4 @@
## Transformer selection strategy
# Transformer selection strategy
The TransformRegistry uses t-config to choose which Transformer will be
used. A transformer definition contains a supported list of source and
@@ -6,12 +6,13 @@ target Media Types. This is used for the most basic selection. It is further
refined by checking that the definition also supports transform options (the
parameters) that have been supplied in a transform request.
~~~
~~~text
Transformer 1 defines options: Op1, Op2
Transformer 2 defines options: Op1, Op2, Op3, Op4
Transform request provides values for options: Op2, Op3
~~~
If we assume both transformers support the required source and target Media
Types, Transformer 2 will be selected because it knows about all the supplied
options. The definition may also specify that some options are required or

View File

@@ -1,10 +1,10 @@
## TransformerDebug
# TransformerDebug
In addition to any normal logging, the t-engines, t-router and t-client also
use the `TransformerDebug` class to provide request based logging. The
following is an example from Alfresco after the upload of a `docx` file.
~~~
~~~text
163 docx json AGM 2016 - Masters report.docx 14.8 KB -- metadataExtract -- TransformService
163 workspace://SpacesStore/0db3a665-328d-4437-85ed-56b753cf19c8 1563306426
163 docx json 14.8 KB -- metadataExtract -- PoiMetadataExtractor

View File

@@ -35,7 +35,8 @@ implementations.
The `TransformEngineName` is important if the config from multiple T-Engines is being
combined as they are sorted by name.
```
```java
package org.alfresco.transform.example;
import com.google.common.collect.ImmutableMap;
@@ -81,7 +82,8 @@ public class HelloTransformEngine implements TransformEngine
```
**Example CustomTransformer**
```
```java
package org.alfresco.transform.example;
import org.alfresco.transform.base.CustomTransformer;
@@ -120,6 +122,7 @@ public class HelloTransformer implements CustomTransformer
```
**Example T-Config** `resources/hello_engine_config.json`
```json
{
"transformOptions": {
@@ -145,12 +148,14 @@ public class HelloTransformer implements CustomTransformer
**Example properties** `resources/application-default.yaml`
As can be seen the following defines a default which can be overridden by an environment variable.
```yaml
queue:
engineRequestQueue: ${TRANSFORM_ENGINE_REQUEST_QUEUE:org.alfresco.transform.engine.libreoffice.acs}
```
**Example ProbeTransform test file** `resources/probe.txt`
```json
```text
Jane
```

View File

@@ -46,8 +46,6 @@ import org.springframework.scheduling.annotation.EnableScheduling;
@EnableRetry
public class Application
{
private static final Logger logger = LoggerFactory.getLogger(Application.class);
@Value("${container.name}")
private String containerName;

View File

@@ -97,13 +97,19 @@ import static org.springframework.http.MediaType.MULTIPART_FORM_DATA_VALUE;
@Controller
public class TransformController
{
private static final Logger logger = LoggerFactory.getLogger(TransformController.class);
private static final String MODEL_TITLE = "title";
private static final String MODEL_PROXY_PATH_PREFIX = "proxyPathPrefix";
private static final String MODEL_MESSAGE = "message";
@Autowired(required = false)
private List<TransformEngine> transformEngines;
@Autowired
private TransformServiceRegistry transformRegistry;
@Autowired TransformHandler transformHandler;
@Autowired
TransformHandler transformHandler;
@Autowired
private String coreVersion;
@Autowired
@@ -119,11 +125,11 @@ public class TransformController
if (transformEngines != null)
{
transformEngine = getTransformEngine();
logger.info("TransformEngine: " + transformEngine.getTransformEngineName());
logger.info("TransformEngine: {}", transformEngine.getTransformEngineName());
transformEngines.stream()
.filter(te -> te != transformEngine)
.filter(transformEngineFromStream -> transformEngineFromStream != transformEngine)
.sorted(Comparator.comparing(TransformEngine::getTransformEngineName))
.map(transformEngine -> " "+transformEngine.getTransformEngineName()).forEach(logger::info);
.map(sortedTransformEngine -> " "+sortedTransformEngine.getTransformEngineName()).forEach(logger::info);
}
}
@@ -133,7 +139,7 @@ public class TransformController
// CustomTransform code from many t-engines into a single t-engine. In this case, there should be a wrapper
// TransformEngine (it has no TransformConfig of its own).
return transformEngines.stream()
.filter(transformEngine -> transformEngine.getTransformConfig() == null)
.filter(transformEngineFromStream -> transformEngineFromStream.getTransformConfig() == null)
.findFirst()
.orElse(transformEngines.get(0));
}
@@ -171,8 +177,8 @@ public class TransformController
@GetMapping(ENDPOINT_ROOT)
public String test(Model model)
{
model.addAttribute("title", getSimpleTransformEngineName() + " Test Page");
model.addAttribute("proxyPathPrefix", getPathPrefix());
model.addAttribute(MODEL_TITLE, getSimpleTransformEngineName() + " Test Page");
model.addAttribute(MODEL_PROXY_PATH_PREFIX, getPathPrefix());
TransformConfig transformConfig = ((TransformRegistry) transformRegistry).getTransformConfig();
transformConfig = setOrClearCoreVersion(transformConfig, 0);
model.addAttribute("transformOptions", optionLister.getOptionNames(transformConfig.getTransformOptions()));
@@ -185,8 +191,8 @@ public class TransformController
@GetMapping(ENDPOINT_ERROR)
public String error(Model model)
{
model.addAttribute("title", getSimpleTransformEngineName() + " Error Page");
model.addAttribute("proxyPathPrefix", getPathPrefix());
model.addAttribute(MODEL_TITLE, getSimpleTransformEngineName() + " Error Page");
model.addAttribute(MODEL_PROXY_PATH_PREFIX, getPathPrefix());
return "error"; // display error.html
}
@@ -196,8 +202,8 @@ public class TransformController
@GetMapping(ENDPOINT_LOG)
String log(Model model)
{
model.addAttribute("title", getSimpleTransformEngineName() + " Log Entries");
model.addAttribute("proxyPathPrefix", getPathPrefix());
model.addAttribute(MODEL_TITLE, getSimpleTransformEngineName() + " Log Entries");
model.addAttribute(MODEL_PROXY_PATH_PREFIX, getPathPrefix());
Collection<LogEntry> log = LogEntry.getLog();
if (!log.isEmpty())
{
@@ -342,9 +348,9 @@ public class TransformController
response.sendError(e.getStatus().value(), message);
ModelAndView mav = new ModelAndView();
mav.addObject("title", getSimpleTransformEngineName() + " Error Page");
mav.addObject("proxyPathPrefix", getPathPrefix());
mav.addObject("message", message);
mav.addObject(MODEL_TITLE, getSimpleTransformEngineName() + " Error Page");
mav.addObject(MODEL_PROXY_PATH_PREFIX, getPathPrefix());
mav.addObject(MODEL_MESSAGE, message);
mav.setViewName("error"); // display error.html
return mav;
}

View File

@@ -221,7 +221,7 @@ public class FileManager
final File systemTempDir = new File(systemTempDirPath);
final File tempDir = new File(systemTempDir, dirName);
if (!tempDir.exists() && !tempDir.mkdirs() && !tempDir.exists())
if (!tempDir.exists() && !tempDir.mkdirs())
{
throw new RuntimeException("Failed to create temp directory: " + tempDir);
}

View File

@@ -1,4 +1,9 @@
<html xmlns:th="http://www.thymeleaf.org">
<!DOCTYPE html>
<html xmlns:th="http://www.thymeleaf.org" xml:lang="en">
<head>
<title th:text="${title}"/>
</head>
<body>
<div>

View File

@@ -1,4 +1,9 @@
<html xmlns:th="http://www.thymeleaf.org">
<!DOCTYPE html>
<html xmlns:th="http://www.thymeleaf.org" xml:lang="en">
<head>
<title th:text="${title}"/>
</head>
<body>
<div>

View File

@@ -1,4 +1,9 @@
<html xmlns:th="http://www.thymeleaf.org">
<!DOCTYPE html>
<html xmlns:th="http://www.thymeleaf.org" xml:lang="en">
<head>
<title th:text="${title}"/>
</head>
<body>
<div>
<h2 th:text="${title}"></h2>