From 819988c518d3a54c2355a8131e4377bfec9faa00 Mon Sep 17 00:00:00 2001 From: Alan Davis Date: Fri, 13 Jan 2012 17:25:32 +0000 Subject: [PATCH] ALF-12273: Merge V4.0-BUG-FIX to HEAD 33119: Merge V3.4-BUG-FIX (3.4.8) to V4.0-BUG-FIX (4.0.1) 33099: ALF-10412 Nonreducing 100% CPU Uploading Large Files to Share Site Document Library ALF-10976 Excel files bigger than 2mb cause soffice.exe to take 100% of one CPU for more than 2 minutes in previews. - Polish TransformerDebug - Better config for txt and xlsx to swf 33095: ALF-10412 Nonreducing 100% CPU Uploading Large Files to Share Site Document Library ALF-10976 Excel files bigger than 2mb cause soffice.exe to take 100% of one CPU for more than 2 minutes in previews. - Improvements to TransformerDebug so that calls to getTransformers use trace rather than debug level logging allowing one to see the wood for the trees 33016: ALF-10412 Nonreducing 100% CPU Uploading Large Files to Share Site Document Library ALF-10976 Excel files bigger than 2mb cause soffice.exe to take 100% of one CPU for more than 2 minutes in previews. - fix build errors - may not get all of them as not tested on Linux 33005: ALF-10412 Nonreducing 100% CPU Uploading Large Files to Share Site Document Library ALF-10976 Excel files bigger than 2mb cause soffice.exe to take 100% of one CPU for more than 2 minutes in previews. - Disable transformers if the source txt or xlsx is too large - avoids transforms that don't finish txt limit is 5MB xlsx limit is 1MB - Added a general 2 minute timeout added (ignored by JOD transformers - which already have a 2 minute timeout and OpenOffice transformers - would require more work) - Previous commit already limited txt -> pdf -> png so that only the 1st pdf page was created when creating a doclib icon - Earlier commit already reduced the priority of the background Thread used for transformations so that user interaction did not suffer. 33004: ALF-10412 Nonreducing 100% CPU Uploading Large Files to Share Site Document Library ALF-10976 Excel files bigger than 2mb cause soffice.exe to take 100% of one CPU for more than 2 minutes in previews. - Added time, size and page limits to transformer configuration to allow one to avoid costly transformations and to stop them if they do occur. Limits avoid a transformer being selected if the source is too large, or make it throw and Exception or discard data after a given time, KBytes read or pages created. - Page limits currently only used by TextToPdfContentTransformer for thumbnail (icon) creation. - Time, Size and Page limits are currently ignored by JodContentTransformer and OpenOfficeContentTransformerWorker once running but the max source size limits may be used to avoid the selection of the transformer in the first place. - TransformerDebug added to be able to see what is going on. A real eye opener! log4j org.alfresco.repo.content.transform.TransformerDebug 32136: ALF-10412 Nonreducing 100% CPU Uploading Large Files to Share Site Document Library Reducing the priority of the async thread pool that is used to perform the transformations so that normal activity (and even garbage collection) is not interrupted by transformations. git-svn-id: https://svn.alfresco.com/repos/alfresco-enterprise/alfresco/HEAD/root@33223 c4b6b30b-aa2e-2d43-bbcb-ca4b014f7261 --- config/alfresco/content-services-context.xml | 93 ++- config/alfresco/repository.properties | 55 ++ .../Search/lucene/lucene-search-context.xml | 8 +- config/alfresco/swf-transform-context.xml | 9 + config/alfresco/thumbnail-service-context.xml | 102 +++- .../ImageTransformActionExecuter.java | 2 +- .../executer/TransformActionExecuter.java | 2 +- .../repo/content/AbstractContentReader.java | 174 +++++- .../AbstractContentReaderLimitTest.java | 344 +++++++++++ .../ContentMinimalContextTestSuite.java | 12 + .../repo/content/ContentServiceImpl.java | 77 ++- .../transform/AbstractContentTransformer.java | 16 +- .../AbstractContentTransformer2.java | 30 +- .../AbstractContentTransformerLimits.java | 380 ++++++++++++ .../AbstractContentTransformerLimitsTest.java | 319 ++++++++++ .../AbstractContentTransformerTest.java | 13 +- .../ArchiveContentTransformerTest.java | 8 +- ...naryPassThroughContentTransformerTest.java | 14 +- .../transform/ComplexContentTransformer.java | 25 +- .../ComplexContentTransformerTest.java | 11 +- .../content/transform/ContentTransformer.java | 12 +- .../transform/ContentTransformerRegistry.java | 28 +- .../ContentTransformerRegistryTest.java | 73 +-- .../transform/FailoverContentTransformer.java | 23 +- .../FailoverContentTransformerTest.java | 5 +- .../HtmlParserContentTransformerTest.java | 8 +- .../transform/MailContentTransformerTest.java | 10 +- .../MediaWikiContentTransformerTest.java | 10 +- .../OpenOfficeContentTransformerTest.java | 18 +- .../PdfBoxContentTransformerTest.java | 14 +- ...dfBoxPdfToImageContentTransformerTest.java | 7 +- .../transform/PoiContentTransformerTest.java | 32 +- .../transform/PoiHssfContentTransformer.java | 19 +- .../PoiHssfContentTransformerTest.java | 16 +- .../PoiOOXMLContentTransformerTest.java | 30 +- .../transform/ProxyContentTransformer.java | 17 +- ...ntimeExecutableContentTransformerTest.java | 3 + ...imeExecutableContentTransformerWorker.java | 3 +- .../StringExtractingContentTransformer.java | 2 +- ...tringExtractingContentTransformerTest.java | 8 +- .../TextMiningContentTransformerTest.java | 10 +- .../TextToPdfContentTransformer.java | 160 ++++- .../TextToPdfContentTransformerTest.java | 116 +++- .../TikaAutoContentTransformerTest.java | 50 +- .../TikaPoweredContentTransformer.java | 16 +- .../content/transform/TransformerDebug.java | 570 ++++++++++++++++++ .../ImageMagickContentTransformerTest.java | 13 +- .../ImageMagickContentTransformerWorker.java | 3 +- .../org/alfresco/repo/jscript/ScriptNode.java | 7 +- ...AbstractTransformationRenderingEngine.java | 98 ++- .../executer/ImageRenderingEngine.java | 33 +- .../executer/ReformatRenderingEngine.java | 17 +- .../repo/rule/RuleServiceCoverageTest.java | 6 +- .../ADMLuceneIndexerAndSearcherFactory.java | 13 + .../impl/lucene/ADMLuceneIndexerImpl.java | 122 ++-- .../search/impl/lucene/ADMLuceneTest.java | 8 +- .../AVMLuceneIndexerAndSearcherFactory.java | 13 + .../impl/lucene/AVMLuceneIndexerImpl.java | 120 ++-- .../CreateThumbnailActionExecuter.java | 62 +- .../repo/thumbnail/SimpleThumbnailer.java | 2 +- .../repo/thumbnail/ThumbnailRegistry.java | 38 +- .../ThumbnailRenditionConvertor.java | 10 +- .../thumbnail/ThumbnailServiceImplTest.java | 8 +- .../UpdateThumbnailActionExecuter.java | 60 +- .../cmr/repository/ContentService.java | 23 +- .../TransformationOptionLimits.java | 257 ++++++++ .../TransformationOptionLimitsTest.java | 316 ++++++++++ .../repository/TransformationOptionPair.java | 244 ++++++++ .../TransformationOptionPairTest.java | 332 ++++++++++ .../cmr/repository/TransformationOptions.java | 193 +++++- 70 files changed, 4514 insertions(+), 438 deletions(-) create mode 100644 source/java/org/alfresco/repo/content/AbstractContentReaderLimitTest.java create mode 100644 source/java/org/alfresco/repo/content/transform/AbstractContentTransformerLimits.java create mode 100644 source/java/org/alfresco/repo/content/transform/AbstractContentTransformerLimitsTest.java create mode 100644 source/java/org/alfresco/repo/content/transform/TransformerDebug.java create mode 100644 source/java/org/alfresco/service/cmr/repository/TransformationOptionLimits.java create mode 100644 source/java/org/alfresco/service/cmr/repository/TransformationOptionLimitsTest.java create mode 100644 source/java/org/alfresco/service/cmr/repository/TransformationOptionPair.java create mode 100644 source/java/org/alfresco/service/cmr/repository/TransformationOptionPairTest.java diff --git a/config/alfresco/content-services-context.xml b/config/alfresco/content-services-context.xml index 131a198fcf..f0a7701b2f 100644 --- a/config/alfresco/content-services-context.xml +++ b/config/alfresco/content-services-context.xml @@ -112,6 +112,9 @@ ${policy.content.update.ignoreEmpty} + + + @@ -253,6 +256,13 @@ + + + + + + + + + + + ${content.transformer.default.timeoutMs} + ${content.transformer.default.readLimitTimeMs} + ${content.transformer.default.maxSourceSizeKBytes} + ${content.transformer.default.readLimitKBytes} + ${content.transformer.default.pageLimit} + ${content.transformer.default.maxPages} - - - - + class="org.alfresco.repo.content.transform.PdfToImageContentTransformer" + parent="baseContentTransformer" /> + - - - - + class="org.alfresco.repo.content.transform.PdfBoxPdfToImageContentTransformer" + parent="baseContentTransformer" /> application/pdf + + ${content.transformer.PdfBox.TextToPdf.maxSourceSizeKBytes} text/plain - + + + + + + + + + 0 + + + + + + ${content.transformer.OpenOffice.mimeTypeLimits.txt.pdf.maxSourceSizeKBytes} + + + + + + + + + + ${content.transformer.OpenOffice.mimeTypeLimits.xlsx.pdf.maxSourceSizeKBytes} + + + + + + application/pdf + + + + + + + + 0 + + + + + + + + + + 0 + + + + + + pdf using PdfBox (txt=text/*) 10M takes about 12 seconds +content.transformer.PdfBox.TextToPdf.maxSourceSizeKBytes=10240 + +# pdf -> swf using Pdf2swf 1M takes about 30 seconds. +# Using a value of 1.25M (a bit larger that the txt or xlsx) used to create +# the pdf on the way to swf to avoid the second part of a transform failing +content.transformer.Pdf2swf.maxSourceSizeKBytes=1152 + +# txt -> pdf -> swf 1M (pdf is about the same size as the txt) +# Need this limit as transformer.PdfBox txt -> pdf is allowed up to 10M +# unlike transformer.OpenOffice and transformer.JodConverter which +# also only allow 1M text +content.transformer.complex.Text.Pdf2swf.maxSourceSizeKBytes=1024 + +# Limits for specific mimetype transformations (txt=text/plain only) 1M +# Has implication for Share preview which generally goes via pdf to get to swf +content.transformer.OpenOffice.mimeTypeLimits.txt.pdf.maxSourceSizeKBytes=1024 +content.transformer.OpenOffice.mimeTypeLimits.xlsx.pdf.maxSourceSizeKBytes=1024 + +content.transformer.JodConverter.mimeTypeLimits.txt.pdf.maxSourceSizeKBytes=1024 +content.transformer.JodConverter.mimeTypeLimits.xlsx.pdf.maxSourceSizeKBytes=1024 + # Property to enable upgrade from 2.1-A V2.1-A.fixes.to.schema=0 #V2.1-A.fixes.to.schema=82 diff --git a/config/alfresco/subsystems/Search/lucene/lucene-search-context.xml b/config/alfresco/subsystems/Search/lucene/lucene-search-context.xml index 9504149931..4606104698 100644 --- a/config/alfresco/subsystems/Search/lucene/lucene-search-context.xml +++ b/config/alfresco/subsystems/Search/lucene/lucene-search-context.xml @@ -282,9 +282,8 @@ - - - + + @@ -381,6 +380,9 @@ + + + diff --git a/config/alfresco/swf-transform-context.xml b/config/alfresco/swf-transform-context.xml index 33c567df3e..3e0e3fc2fa 100644 --- a/config/alfresco/swf-transform-context.xml +++ b/config/alfresco/swf-transform-context.xml @@ -23,6 +23,7 @@ + ${content.transformer.Pdf2swf.maxSourceSizeKBytes} application/pdf + application/pdf + + ${content.transformer.complex.Text.Pdf2swf.maxSourceSizeKBytes} diff --git a/config/alfresco/thumbnail-service-context.xml b/config/alfresco/thumbnail-service-context.xml index c8e4658b94..fe3c2044de 100644 --- a/config/alfresco/thumbnail-service-context.xml +++ b/config/alfresco/thumbnail-service-context.xml @@ -1,7 +1,10 @@ - - + @@ -93,6 +96,12 @@ + + + + + + @@ -114,27 +123,27 @@ - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + @@ -200,6 +209,13 @@ + + + ${system.thumbnail.generate} + + + + @@ -216,8 +232,48 @@ + + + ${system.thumbnail.generate} + + + + + + + + ${system.thumbnail.mimetype.maxSourceSizeKBytes.pdf} + + + ${system.thumbnail.mimetype.maxSourceSizeKBytes.txt} + + + ${system.thumbnail.mimetype.maxSourceSizeKBytes.docx} + + + ${system.thumbnail.mimetype.maxSourceSizeKBytes.xlsx} + + + ${system.thumbnail.mimetype.maxSourceSizeKBytes.pptx} + + + ${system.thumbnail.mimetype.maxSourceSizeKBytes.odt} + + + ${system.thumbnail.mimetype.maxSourceSizeKBytes.ods} + + + ${system.thumbnail.mimetype.maxSourceSizeKBytes.odp} + + + diff --git a/source/java/org/alfresco/repo/action/executer/ImageTransformActionExecuter.java b/source/java/org/alfresco/repo/action/executer/ImageTransformActionExecuter.java index ec01c1c60a..015db34dea 100644 --- a/source/java/org/alfresco/repo/action/executer/ImageTransformActionExecuter.java +++ b/source/java/org/alfresco/repo/action/executer/ImageTransformActionExecuter.java @@ -80,7 +80,7 @@ public class ImageTransformActionExecuter extends TransformActionExecuter imageOptions.setCommandOptions(convertCommand); // check if the transformer is going to work, i.e. is available - if (!this.imageMagickContentTransformer.isTransformable(contentReader.getMimetype(), contentWriter + if (!this.imageMagickContentTransformer.isTransformable(contentReader.getMimetype(), contentReader.getSize(), contentWriter .getMimetype(), imageOptions)) { throw new NoTransformerException(contentReader.getMimetype(), contentWriter.getMimetype()); diff --git a/source/java/org/alfresco/repo/action/executer/TransformActionExecuter.java b/source/java/org/alfresco/repo/action/executer/TransformActionExecuter.java index f024661b41..5e5cf635ad 100644 --- a/source/java/org/alfresco/repo/action/executer/TransformActionExecuter.java +++ b/source/java/org/alfresco/repo/action/executer/TransformActionExecuter.java @@ -172,7 +172,7 @@ public class TransformActionExecuter extends ActionExecuterAbstractBase throw new RuleServiceException(CONTENT_READER_NOT_FOUND_MESSAGE); } - if (null == contentService.getTransformer(contentReader.getMimetype(), mimeType)) + if (null == contentService.getTransformer(contentReader.getContentUrl(), contentReader.getMimetype(), contentReader.getSize(), mimeType, new TransformationOptions())) { throw new RuleServiceException(String.format(TRANSFORMER_NOT_EXISTS_MESSAGE_PATTERN, contentReader.getMimetype(), mimeType)); } diff --git a/source/java/org/alfresco/repo/content/AbstractContentReader.java b/source/java/org/alfresco/repo/content/AbstractContentReader.java index 1c72302849..5e4ae7432e 100644 --- a/source/java/org/alfresco/repo/content/AbstractContentReader.java +++ b/source/java/org/alfresco/repo/content/AbstractContentReader.java @@ -1,5 +1,5 @@ /* - * Copyright (C) 2005-2010 Alfresco Software Limited. + * Copyright (C) 2005-2011 Alfresco Software Limited. * * This file is part of Alfresco * @@ -32,13 +32,20 @@ import java.nio.channels.FileChannel; import java.nio.channels.ReadableByteChannel; import java.util.ArrayList; import java.util.List; +import java.util.Timer; +import java.util.TimerTask; +import java.util.concurrent.atomic.AtomicBoolean; import org.alfresco.error.AlfrescoRuntimeException; import org.alfresco.repo.content.filestore.FileContentWriter; +import org.alfresco.repo.content.transform.TransformerDebug; import org.alfresco.service.cmr.repository.ContentAccessor; import org.alfresco.service.cmr.repository.ContentIOException; import org.alfresco.service.cmr.repository.ContentReader; import org.alfresco.service.cmr.repository.ContentStreamListener; +import org.alfresco.service.cmr.repository.TransformationOptionLimits; +import org.alfresco.service.cmr.repository.TransformationOptionPair; +import org.alfresco.service.cmr.repository.TransformationOptionPair.Action; import org.alfresco.util.EqualsHelper; import org.alfresco.util.TempFileProvider; import org.apache.commons.logging.Log; @@ -46,6 +53,9 @@ import org.apache.commons.logging.LogFactory; import org.springframework.aop.framework.ProxyFactory; import org.springframework.util.FileCopyUtils; +import sun.nio.ch.ChannelInputStream; + + /** * Implements all the convenience methods of the interface. The only methods * that need to be implemented, i.e. provide low-level content access are: @@ -59,10 +69,20 @@ import org.springframework.util.FileCopyUtils; public abstract class AbstractContentReader extends AbstractContentAccessor implements ContentReader { private static final Log logger = LogFactory.getLog(AbstractContentReader.class); + private static final Timer timer = new Timer(); private List listeners; private ReadableByteChannel channel; + // Optional limits on reading + private TransformationOptionLimits limits; + + // Only needed if limits are set + private TransformerDebug transformerDebug; + + // For testing: Allows buffering to be turned off + private boolean useBufferedInputStream = true; + /** * @param contentUrl the content URL - this should be relative to the root of the store * and not absolute: to enable moving of the stores @@ -73,7 +93,37 @@ public abstract class AbstractContentReader extends AbstractContentAccessor impl listeners = new ArrayList(2); } + + public void setLimits(TransformationOptionLimits limits) + { + this.limits = limits; + } + + public TransformationOptionLimits getLimits() + { + return limits; + } + public void setTransformerDebug(TransformerDebug transformerDebug) + { + this.transformerDebug = transformerDebug; + } + + public TransformerDebug getTransformerDebug() + { + return transformerDebug; + } + + public void setUseBufferedInputStream(boolean useBufferedInputStream) + { + this.useBufferedInputStream = useBufferedInputStream; + } + + public boolean getUseBufferedInputStream() + { + return useBufferedInputStream; + } + /** * Adds the listener after checking that the output stream isn't already in * use. @@ -323,7 +373,26 @@ public abstract class AbstractContentReader extends AbstractContentAccessor impl try { ReadableByteChannel channel = getReadableChannel(); - InputStream is = new BufferedInputStream(Channels.newInputStream(channel)); + InputStream is = Channels.newInputStream(channel); + + // If we have a timeout or read limit, intercept the calls. + if (limits != null) + { + TransformationOptionPair time = limits.getTimePair(); + TransformationOptionPair kBytes = limits.getKBytesPair(); + long timeoutMs = time.getValue(); + long readLimitBytes = kBytes.getValue() * 1024; + + if (timeoutMs > 0 || readLimitBytes > 0) + { + Action timeoutAction = time.getAction(); + Action readLimitAction = kBytes.getAction(); + + is = new TimeSizeRestrictedInputStream(is, timeoutMs, timeoutAction, + readLimitBytes, readLimitAction, transformerDebug); + } + } + is = new BufferedInputStream(is); // done return is; } @@ -486,4 +555,105 @@ public abstract class AbstractContentReader extends AbstractContentAccessor impl " right: " + right); } } + + /** + * InputStream that wraps another InputStream to terminate early after a timeout + * or after reading a number of bytes. It terminates by either returning end of file + * (-1) or throwing an IOException. + + * @author Alan Davis + */ + private class TimeSizeRestrictedInputStream extends InputStream + { + private final AtomicBoolean timeoutFlag = new AtomicBoolean(false); + + private final InputStream is; + private final long timeoutMs; + private final long readLimitBytes; + private final Action timeoutAction; + private final Action readLimitAction; + private final TransformerDebug transformerDebug; + + private long readCount = 0; + + public TimeSizeRestrictedInputStream(InputStream is, + long timeoutMs, Action timeoutAction, + long readLimitBytes, Action readLimitAction, + TransformerDebug transformerDebug) + { + this.is = useBufferedInputStream ? new BufferedInputStream(is) : is; + this.timeoutMs = timeoutMs; + this.timeoutAction = timeoutAction; + this.readLimitBytes = readLimitBytes; + this.readLimitAction = readLimitAction; + this.transformerDebug = transformerDebug; + + if (timeoutMs > 0) + { + timer.schedule(new TimerTask() + { + @Override + public void run() + { + timeoutFlag.set(true); + } + + }, timeoutMs); + } + } + + @Override + public int read() throws IOException + { + // Throws exception or return true to indicate EOF + if (hitTimeout() || hitReadLimit()) + { + return -1; + } + + int n = is.read(); + if (n > 0) + { + readCount++; + } + return n; + } + + private boolean hitTimeout() throws IOException + { + if (timeoutMs > 0 && timeoutFlag.get()) + { + timeoutAction.throwIOExceptionIfRequired( + "Transformation has taken too long ("+ + (timeoutMs/1000)+" seconds)", transformerDebug); + return true; + } + return false; + } + + private boolean hitReadLimit() throws IOException + { + if (readLimitBytes > 0 && readCount >= readLimitBytes) + { + readLimitAction.throwIOExceptionIfRequired( + "Transformation has read too many bytes ("+ + (readLimitBytes/1024)+"K)", transformerDebug); + return true; + } + return false; + } + + @Override + public void close() throws IOException + { + try + { + is.close(); + } + finally + { + super.close(); + } + } + }; } diff --git a/source/java/org/alfresco/repo/content/AbstractContentReaderLimitTest.java b/source/java/org/alfresco/repo/content/AbstractContentReaderLimitTest.java new file mode 100644 index 0000000000..33a5cc21c1 --- /dev/null +++ b/source/java/org/alfresco/repo/content/AbstractContentReaderLimitTest.java @@ -0,0 +1,344 @@ +/* + * Copyright (C) 2005-2011 Alfresco Software Limited. + * + * This file is part of Alfresco + * + * Alfresco is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * Alfresco is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with Alfresco. If not, see . + */ +package org.alfresco.repo.content; + + +import static org.junit.Assert.assertTrue; + +import java.io.IOException; +import java.io.InputStream; +import java.nio.channels.Channels; +import java.nio.channels.ReadableByteChannel; + +import org.alfresco.repo.content.transform.TransformerDebug; +import org.alfresco.service.cmr.repository.ContentIOException; +import org.alfresco.service.cmr.repository.ContentReader; +import org.alfresco.service.cmr.repository.TransformationOptionLimits; +import org.junit.Before; +import org.junit.Test; +import org.springframework.context.ApplicationContext; + +/** + * Test class for time and byte limits on an AbstractContentReader + * + * @author Alan Davis + */ +public class AbstractContentReaderLimitTest +{ + private static final int K = 1024; + private static final int M = K*K; + + // Normal test file size 5K + private static final long SIZE = 5*K; + + // Normal test delay between returning bytes, giving a run time of about 5 seconds if not interrupted. + private static final long MS_PER_BYTE = 1; + + // Large test file size 100 MB with no delay takes about a second to read. + private static final long LARGE_SIZE = 100*M; + + // Top speed to read 1 MB via the DummyAbstractContentReader + // Assume about 10ms normally per MB, so use half that and a high + // MARGIN_OF_ERROR_PERCENTAGE_FAST. + private static final long MS_PER_MB = 5; + + // Margins of error when using a DummyAbstractContentReader + // with or without a delay. Used to make sure different runs + // don't result in failing tests but at the same time that + // they will if there is a real problem. + private static final int MARGIN_OF_ERROR_PERCENTAGE_SLOW = 50; + private static final int MARGIN_OF_ERROR_PERCENTAGE_FAST = 900; + + private DummyAbstractContentReader reader; + private TransformationOptionLimits limits; + private TransformerDebug transformerDebug; + private long minTime; + private long maxTime; + private long minLength; + private long maxLength; + + @Before + public void setUp() throws Exception + { + ApplicationContext ctx = ContentMinimalContextTestSuite.getContext(); + transformerDebug = (TransformerDebug) ctx.getBean("transformerDebug"); + + limits = new TransformationOptionLimits(); + reader = new DummyAbstractContentReader(SIZE, MS_PER_BYTE); + reader.setLimits(limits); + reader.setTransformerDebug(transformerDebug); + + // Without the following, the bytes from the DummyAbstractContentReader are read in 4K blocks + // so the test to do with timeouts and read limits will not work, as they expect to read 1K per + // second. Not an issue in the real world as read rates are much higher, so a buffer makes no + // difference to limit checking. It does make a vast difference to performance when the InputStream + // is wrapped in a InputStreamReader as is done by a number of transformers. + reader.setUseBufferedInputStream(false); + } + + @Test + public void noLimitTest() throws Exception + { + readAndCheck(); + } + + @Test(expected=ContentIOException.class) + public void maxKBytesTest() throws Exception + { + limits.setMaxSourceSizeKBytes(1); + readAndCheck(); + } + + @Test(expected=ContentIOException.class) + public void maxTimeTest() throws Exception + { + limits.setTimeoutMs(1000); + readAndCheck(); + } + + @Test(expected=ContentIOException.class) + public void maxTimeAndKBytesTest() throws Exception + { + limits.setTimeoutMs(1000); + limits.setMaxSourceSizeKBytes(1); + readAndCheck(); + } + + @Test + public void limitKBytesTest() throws Exception + { + limits.setReadLimitKBytes(1); + readAndCheck(); + } + + @Test + public void limitTimeTest() throws Exception + { + limits.setReadLimitTimeMs(1000); + readAndCheck(); + } + + @Test + public void limitTimeAndKBytesTest() throws Exception + { + limits.setReadLimitTimeMs(1000); + limits.setReadLimitKBytes(1); + readAndCheck(); + } + + @Test + public void fullSpeedReader() throws Exception + { + // Check that we have not slowed down reading of large files. + reader = new DummyAbstractContentReader(LARGE_SIZE, 0); + reader.setLimits(limits); + reader.setTransformerDebug(transformerDebug); + reader.setUseBufferedInputStream(true); + + readAndCheck(); + } + + private void readAndCheck() throws Exception + { + Exception exception = null; + + long length = 0; + long time = System.currentTimeMillis(); + try + { + String content = reader.getContentString(); + length = content.length(); + } + catch(ContentIOException e) + { + exception = e; + } + time = System.currentTimeMillis() - time; + + calcMaxMinValues(); + + System.out.printf("Time %04d %04d..%04d length %04d %04d..%04d %s\n", + time, minTime, maxTime, + length, minLength, maxLength, + (exception == null ? "" : exception.getClass().getSimpleName())); + + assertTrue("Reader is too fast ("+time+"ms range is "+minTime+"..."+maxTime+"ms)", time >= minTime); + assertTrue("Reader is too slow ("+time+"ms range is "+minTime+"..."+maxTime+"ms)", time <= maxTime); + + if (exception != null) + throw exception; + + assertTrue("Content is too short ("+length+" bytes range is "+minLength+"..."+maxLength+")", length >= minLength); + assertTrue("Content is too long ("+length+" bytes range is "+minLength+"..."+maxLength+")", length <= maxLength); + } + + private void calcMaxMinValues() + { + long size = reader.size; + + long timeout = limits.getTimePair().getValue(); + assertTrue("The test time value ("+timeout+ + "ms) should be lowered given the file size ("+size+ + ") and the margin of error (of "+marginOfError(timeout)+"ms)", + timeout <= 0 || msToBytes(timeout+marginOfError(timeout)) <= size); + + long readKBytes = limits.getKBytesPair().getValue(); + long readBytes = readKBytes * K; + assertTrue("The test KByte value ("+readKBytes+ + "K) should be lowered given the file size ("+size+ + ") and the margin of error (of "+marginOfError(readBytes)+"bytes)", + readBytes <= 0 || readBytes+marginOfError(readBytes) <= size); + + long bytes = (readBytes > 0) ? readBytes : size; + long readTime = bytesToMs(bytes); + + minTime = (timeout > 0) ? Math.min(timeout, readTime) : readTime; + maxTime = minTime + marginOfError(minTime); + minLength = (timeout > 0) ? msToBytes(minTime-marginOfError(minTime)) : bytes; + maxLength = (timeout > 0) ? Math.min(msToBytes(maxTime), size) : bytes; + } + + private long msToBytes(long ms) + { + return (reader.msPerByte > 0) + ? ms / reader.msPerByte + : ms / MS_PER_MB * M; + } + + private long bytesToMs(long bytes) + { + return (reader.msPerByte > 0) + ? bytes * reader.msPerByte + : bytes * MS_PER_MB / M; + } + + private long marginOfError(long value) + { + return + value * + ((reader.msPerByte > 0) + ? MARGIN_OF_ERROR_PERCENTAGE_SLOW + : MARGIN_OF_ERROR_PERCENTAGE_FAST) / + 100; + } + + /** + * A dummy AbstractContentReader that returns a given number of bytes + * (all 'a') very slowly. There is a configurable delay returning each byte. + * Used to test timeouts and read limits. + */ + public static class DummyAbstractContentReader extends AbstractContentReader + { + final long size; + final long msPerByte; + + /** + * @param size of the dummy data + * @param msPerByte milliseconds between byte reads + */ + public DummyAbstractContentReader(long size, long msPerByte) + { + super("a"); + this.size = size; + this.msPerByte = msPerByte; + } + + /** + * @return Returns an instance of the this class + */ + @Override + protected ContentReader createReader() throws ContentIOException + { + return new DummyAbstractContentReader(size, msPerByte); + } + + @Override + protected ReadableByteChannel getDirectReadableChannel() throws ContentIOException + { + InputStream is = new InputStream() + { + long read = 0; + long start = 0; + + @Override + public int read() throws IOException + { + if (read >= size) + return -1; + + read++; + + if (msPerByte > 0) + { + long elapse = System.currentTimeMillis() - start; + if (read == 1) + { + start = elapse; + } + else + { + // On Windows it is possible to just wait 1 ms per byte but this + // does not work on linux hence (end up with a full read taking + // 40 seconds rather than 5) the need to wait if elapse time + // is too fast. + long delay = (read * msPerByte) - elapse; + if (delay > 0) + { + try + { + Thread.sleep(delay); + } + catch (InterruptedException e) + { + // ignore + } + } + } + } + + return 'a'; + } + + // Just a way to tell AbstractContentReader not to wrap the ChannelInputStream + // in a BufferedInputStream + @Override + public boolean markSupported() + { + return true; + } + }; + return Channels.newChannel(is); + } + + public boolean exists() + { + return true; + } + + public long getLastModified() + { + return 0L; + } + + public long getSize() + { + return size; + } + }; +} diff --git a/source/java/org/alfresco/repo/content/ContentMinimalContextTestSuite.java b/source/java/org/alfresco/repo/content/ContentMinimalContextTestSuite.java index 56636688cb..0316f94177 100644 --- a/source/java/org/alfresco/repo/content/ContentMinimalContextTestSuite.java +++ b/source/java/org/alfresco/repo/content/ContentMinimalContextTestSuite.java @@ -29,6 +29,7 @@ import org.alfresco.repo.content.metadata.PdfBoxMetadataExtracterTest; import org.alfresco.repo.content.metadata.PoiMetadataExtracterTest; import org.alfresco.repo.content.metadata.RFC822MetadataExtracterTest; import org.alfresco.repo.content.metadata.TikaAutoMetadataExtracterTest; +import org.alfresco.repo.content.transform.AbstractContentTransformerLimitsTest; import org.alfresco.repo.content.transform.AppleIWorksContentTransformerTest; import org.alfresco.repo.content.transform.BinaryPassThroughContentTransformerTest; import org.alfresco.repo.content.transform.ComplexContentTransformerTest; @@ -47,9 +48,14 @@ import org.alfresco.repo.content.transform.TextMiningContentTransformerTest; import org.alfresco.repo.content.transform.TextToPdfContentTransformerTest; import org.alfresco.repo.content.transform.TikaAutoContentTransformerTest; import org.alfresco.repo.content.transform.magick.ImageMagickContentTransformerTest; +import org.alfresco.service.cmr.repository.TransformationOptionLimitsTest; +import org.alfresco.service.cmr.repository.TransformationOptionPairTest; import org.alfresco.util.ApplicationContextHelper; import org.springframework.context.ApplicationContext; +import com.google.gdata.data.acl.AclRole; + +import junit.framework.JUnit4TestAdapter; import junit.framework.Test; import junit.framework.TestSuite; @@ -85,6 +91,12 @@ public class ContentMinimalContextTestSuite extends TestSuite // Off we go TestSuite suite = new TestSuite(); + // Limits + suite.addTest(new JUnit4TestAdapter(AbstractContentReaderLimitTest.class)); + suite.addTest(new JUnit4TestAdapter(AbstractContentTransformerLimitsTest.class)); + suite.addTest(new JUnit4TestAdapter(TransformationOptionLimitsTest.class)); + suite.addTest(new JUnit4TestAdapter(TransformationOptionPairTest.class)); + // Metadata tests suite.addTestSuite( DWGMetadataExtracterTest.class ); suite.addTestSuite( HtmlMetadataExtracterTest.class ); diff --git a/source/java/org/alfresco/repo/content/ContentServiceImpl.java b/source/java/org/alfresco/repo/content/ContentServiceImpl.java index b17caba098..1114435fbf 100644 --- a/source/java/org/alfresco/repo/content/ContentServiceImpl.java +++ b/source/java/org/alfresco/repo/content/ContentServiceImpl.java @@ -35,6 +35,7 @@ import org.alfresco.repo.content.cleanup.EagerContentStoreCleaner; import org.alfresco.repo.content.filestore.FileContentStore; import org.alfresco.repo.content.transform.ContentTransformer; import org.alfresco.repo.content.transform.ContentTransformerRegistry; +import org.alfresco.repo.content.transform.TransformerDebug; import org.alfresco.repo.node.NodeServicePolicies; import org.alfresco.repo.policy.ClassPolicyDelegate; import org.alfresco.repo.policy.JavaBehaviour; @@ -89,7 +90,8 @@ public class ContentServiceImpl implements ContentService, ApplicationContextAwa private MimetypeService mimetypeService; private RetryingTransactionHelper transactionHelper; private ApplicationContext applicationContext; - + protected TransformerDebug transformerDebug; + /** a registry of all available content transformers */ private ContentTransformerRegistry transformerRegistry; /** The cleaner that will ensure that rollbacks clean up after themselves */ @@ -177,6 +179,15 @@ public class ContentServiceImpl implements ContentService, ApplicationContextAwa this.applicationContext = applicationContext; } + /** + * Helper setter of the transformer debug. + * @param transformerDebug + */ + public void setTransformerDebug(TransformerDebug transformerDebug) + { + this.transformerDebug = transformerDebug; + } + /** * Service initialise */ @@ -559,40 +570,74 @@ public class ContentServiceImpl implements ContentService, ApplicationContextAwa { throw new AlfrescoRuntimeException("The content writer mimetype must be set: " + writer); } - // look for a transformer - ContentTransformer transformer = transformerRegistry.getTransformer(sourceMimetype, targetMimetype, options); - if (transformer == null) + + try { - throw new NoTransformerException(sourceMimetype, targetMimetype); + // look for a transformer + transformerDebug.pushAvailable(reader.getContentUrl(), sourceMimetype, targetMimetype); + long sourceSize = reader.getSize(); + List transformers = getActiveTransformers(sourceMimetype, sourceSize, targetMimetype, options); + transformerDebug.availableTransformers(transformers, sourceSize, "ContentService.transform(...)"); + + if (transformers.isEmpty()) + { + throw new NoTransformerException(sourceMimetype, targetMimetype); + } + + // we have a transformer, so do it + ContentTransformer transformer = transformers.size() == 0 ? null : transformers.get(0); + transformer.transform(reader, writer, options); + // done + } + finally + { + transformerDebug.popAvailable(); } - // we have a transformer, so do it - transformer.transform(reader, writer, options); - // done } - + /** * @see org.alfresco.repo.content.transform.ContentTransformerRegistry * @see org.alfresco.repo.content.transform.ContentTransformer */ public ContentTransformer getTransformer(String sourceMimetype, String targetMimetype) { - return getTransformer(sourceMimetype, targetMimetype, new TransformationOptions()); + return getTransformer(null, sourceMimetype, -1, targetMimetype, new TransformationOptions()); + } + + public ContentTransformer getTransformer(String sourceMimetype, String targetMimetype, TransformationOptions options) + { + return getTransformer(null, sourceMimetype, -1, targetMimetype, options); } /** - * @see org.alfresco.service.cmr.repository.ContentService#getTransformer(java.lang.String, java.lang.String, org.alfresco.service.cmr.repository.TransformationOptions) + * @see org.alfresco.service.cmr.repository.ContentService#getTransformer(String, java.lang.String, long, java.lang.String, org.alfresco.service.cmr.repository.TransformationOptions) */ - public ContentTransformer getTransformer(String sourceMimetype, String targetMimetype, TransformationOptions options) + public ContentTransformer getTransformer(String sourceUrl, String sourceMimetype, long sourceSize, String targetMimetype, TransformationOptions options) { - return transformerRegistry.getTransformer(sourceMimetype, targetMimetype, options); + try + { + // look for a transformer + transformerDebug.pushAvailable(sourceUrl, sourceMimetype, targetMimetype); + List transformers = getActiveTransformers(sourceMimetype, sourceSize, targetMimetype, options); + transformerDebug.availableTransformers(transformers, sourceSize, "ContentService.getTransformer(...)"); + return (transformers.isEmpty()) ? null : transformers.get(0); + } + finally + { + transformerDebug.popAvailable(); + } } public List getActiveTransformers(String sourceMimetype, String targetMimetype, TransformationOptions options) { - return transformerRegistry.getActiveTransformers(sourceMimetype, targetMimetype, options); + return getActiveTransformers(sourceMimetype, -1, targetMimetype, options); + } + + public List getActiveTransformers(String sourceMimetype, long sourceSize, String targetMimetype, TransformationOptions options) + { + return transformerRegistry.getActiveTransformers(sourceMimetype, sourceSize, targetMimetype, options); } - /** * @see org.alfresco.service.cmr.repository.ContentService#getImageTransformer() */ @@ -628,7 +673,7 @@ public class ContentServiceImpl implements ContentService, ApplicationContextAwa } // look for a transformer - ContentTransformer transformer = transformerRegistry.getTransformer(sourceMimetype, targetMimetype, options); + ContentTransformer transformer = transformerRegistry.getTransformer(sourceMimetype, reader.getSize(), targetMimetype, options); return (transformer != null); } diff --git a/source/java/org/alfresco/repo/content/transform/AbstractContentTransformer.java b/source/java/org/alfresco/repo/content/transform/AbstractContentTransformer.java index 0a8c7f3e63..fbd6e4a0b8 100644 --- a/source/java/org/alfresco/repo/content/transform/AbstractContentTransformer.java +++ b/source/java/org/alfresco/repo/content/transform/AbstractContentTransformer.java @@ -1,5 +1,5 @@ /* - * Copyright (C) 2005-2010 Alfresco Software Limited. + * Copyright (C) 2005-2011 Alfresco Software Limited. * * This file is part of Alfresco * @@ -198,9 +198,21 @@ public abstract class AbstractContentTransformer implements ContentTransformer } // it all checks out OK } + + /** + * {@inheritDoc}

+ * + * Implementation calls the deprecated overloaded method without the sourceSize parameter. + * Note: source size checked has not been added to this deprecated class. + */ + @Override + public boolean isTransformable(String sourceMimetype, long sourceSize, String targetMimetype, TransformationOptions options) + { + return isTransformable(sourceMimetype, targetMimetype, options); + } /** - * @see org.alfresco.repo.content.transform.ContentTransformer#isTransformable(java.lang.String, java.lang.String, org.alfresco.service.cmr.repository.TransformationOptions) + * @deprecated use version with extra sourceSize parameter. */ public boolean isTransformable(String sourceMimetype, String targetMimetype, TransformationOptions options) { diff --git a/source/java/org/alfresco/repo/content/transform/AbstractContentTransformer2.java b/source/java/org/alfresco/repo/content/transform/AbstractContentTransformer2.java index d5a65121d2..41e4226ff3 100644 --- a/source/java/org/alfresco/repo/content/transform/AbstractContentTransformer2.java +++ b/source/java/org/alfresco/repo/content/transform/AbstractContentTransformer2.java @@ -1,5 +1,5 @@ /* - * Copyright (C) 2005-2010 Alfresco Software Limited. + * Copyright (C) 2005-2011 Alfresco Software Limited. * * This file is part of Alfresco * @@ -38,7 +38,7 @@ import org.apache.commons.logging.LogFactory; * @author Derek Hulley * @author Roy Wetherall */ -public abstract class AbstractContentTransformer2 extends ContentTransformerHelper implements ContentTransformer +public abstract class AbstractContentTransformer2 extends AbstractContentTransformerLimits { private static final Log logger = LogFactory.getLog(AbstractContentTransformer2.class); @@ -103,12 +103,14 @@ public abstract class AbstractContentTransformer2 extends ContentTransformerHelp { String sourceMimetype = getMimetype(reader); String targetMimetype = getMimetype(writer); - boolean transformable = isTransformable(sourceMimetype, targetMimetype, options); + long sourceSize = reader.getSize(); + boolean transformable = isTransformable(sourceMimetype, sourceSize, targetMimetype, options); if (transformable == false) { - throw new AlfrescoRuntimeException("Unsuported transformation attempted: \n" + + AlfrescoRuntimeException e = new AlfrescoRuntimeException("Unsuported transformation attempted: \n" + " reader: " + reader + "\n" + " writer: " + writer); + throw transformerDebug.setCause(e); } // it all checks out OK } @@ -154,8 +156,16 @@ public abstract class AbstractContentTransformer2 extends ContentTransformerHelp try { + if (transformerDebug.isEnabled()) + { + transformerDebug.pushTransform(this, reader.getContentUrl(), reader.getMimetype(), writer.getMimetype(), reader.getSize()); + } + // Check the transformability checkTransformable(reader, writer, options); + + // Pass on any limits to the reader + setReaderLimits(reader, writer, options); // Transform transformInternal(reader, writer, options); @@ -174,18 +184,22 @@ public abstract class AbstractContentTransformer2 extends ContentTransformerHelp // Report the error if(differentType == null) { - throw new ContentIOException("Content conversion failed: \n" + + transformerDebug.debug("Failed", e); + throw new ContentIOException("Content conversion failed: \n" + " reader: " + reader + "\n" + " writer: " + writer + "\n" + - " options: " + options, + " options: " + options.toString(false) + "\n" + + " limits: " + getLimits(reader, writer, options), e); } else { + transformerDebug.debug("Failed: Mime type was '"+differentType+"'", e); throw new ContentIOException("Content conversion failed: \n" + " reader: " + reader + "\n" + " writer: " + writer + "\n" + - " options: " + options + "\n" + + " options: " + options.toString(false) + "\n" + + " limits: " + getLimits(reader, writer, options) + "\n" + " claimed mime type: " + reader.getMimetype() + "\n" + " detected mime type: " + differentType, e); @@ -193,6 +207,8 @@ public abstract class AbstractContentTransformer2 extends ContentTransformerHelp } finally { + transformerDebug.popTransform(); + // check that the reader and writer are both closed if (reader.isChannelOpen()) { diff --git a/source/java/org/alfresco/repo/content/transform/AbstractContentTransformerLimits.java b/source/java/org/alfresco/repo/content/transform/AbstractContentTransformerLimits.java new file mode 100644 index 0000000000..78af71c77e --- /dev/null +++ b/source/java/org/alfresco/repo/content/transform/AbstractContentTransformerLimits.java @@ -0,0 +1,380 @@ +/* + * Copyright (C) 2005-2011 Alfresco Software Limited. + * + * This file is part of Alfresco + * + * Alfresco is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * Alfresco is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with Alfresco. If not, see . + */ +package org.alfresco.repo.content.transform; + +import java.util.Map; + +import org.alfresco.repo.content.AbstractContentReader; +import org.alfresco.service.cmr.repository.ContentReader; +import org.alfresco.service.cmr.repository.ContentWriter; +import org.alfresco.service.cmr.repository.TransformationOptionLimits; +import org.alfresco.service.cmr.repository.TransformationOptions; +import org.springframework.beans.factory.BeanNameAware; + +/** + * Provides transformation limits for {@link org.alfresco.repo.content.transform.ContentTransformer} + * implementations. + *

+ * This class maintains the limits and provides methods that combine limits: + * a) for the transformer as a whole + * b) for specific combinations if source and target mimetypes + * c) for the {@link TransformationOptions} provided for a specific transform. + * + * @author Alan Davis + */ +public abstract class AbstractContentTransformerLimits extends ContentTransformerHelper implements ContentTransformer, BeanNameAware +{ + /** Transformer wide Time, KBytes and page limits */ + private TransformationOptionLimits limits = new TransformationOptionLimits(); + + /** + * Time, KBytes and page limits by source and target mimetype combination. + * The first map's key is the source mimetype. The second map's key is the + * target mimetype and the value are the limits. */ + private Map> mimetypeLimits; + + /** Indicates if 'page' limits are supported. */ + private boolean pageLimitsSupported; + + /** For debug **/ + protected TransformerDebug transformerDebug; + + /** The bean name. Used in debug only. */ + private String beanName; + + /** + * Indicates if 'page' limits are supported. + * @return false by default. + */ + protected boolean isPageLimitSupported() + { + return pageLimitsSupported; + } + + /** + * Indicates if 'page' limits are supported. + */ + public void setPageLimitsSuported(boolean pageLimitsSupported) + { + this.pageLimitsSupported = pageLimitsSupported; + } + + /** + * Helper setter of the transformer debug. + * @param transformerDebug + */ + public void setTransformerDebug(TransformerDebug transformerDebug) + { + this.transformerDebug = transformerDebug; + } + + /** + * {@inheritDoc}

+ * + * Implementation calls the deprecated overloaded method without the sourceSize parameter + * and then {@link #isTransformableSize(String, long, String, TransformationOptions)}. + */ + @Override + @SuppressWarnings("deprecation") + public boolean isTransformable(String sourceMimetype, long sourceSize, String targetMimetype, TransformationOptions options) + { + return + isTransformable(sourceMimetype, targetMimetype, options) && + isTransformableSize(sourceMimetype, sourceSize, targetMimetype, options); + } + + /** + * Indicates if this transformer is able to transform the given {@code sourceSize}. + * The {@code maxSourceSizeKBytes} property may indicate that only small source files + * may be transformed. + * @param sourceSize size in bytes of the source. If negative, the source size is unknown. + * @return {@code true} if the source is transformable. + */ + protected boolean isTransformableSize(String sourceMimetype, long sourceSize, String targetMimetype, TransformationOptions options) + { + boolean sizeOkay = true; + if (sourceSize >= 0) + { + TransformationOptionLimits limits = getLimits(sourceMimetype, targetMimetype, options); + + // The maxSourceSizeKbytes value is ignored if this transformer is able to use + // page limits and the limits include a pageLimit. Normally used in the creation + // of icons. Note the readLimitKBytes value is not checked as the combined limits + // only have the max or limit kbytes value set (the smaller value is returned). + if (!isPageLimitSupported() || limits.getPageLimit() <= 0) + { + // if maxSourceSizeKBytes == 0 this implies the transformation is disabled + long maxSourceSizeKBytes = limits.getMaxSourceSizeKBytes(); + sizeOkay = maxSourceSizeKBytes < 0 || (maxSourceSizeKBytes > 0 && sourceSize <= maxSourceSizeKBytes*1024); + if (!sizeOkay && transformerDebug.isEnabled()) + { + transformerDebug.unavailableTransformer(this, maxSourceSizeKBytes); + } + } + } + return sizeOkay; + } + + /** + * Gets the timeout (ms) on the InputStream after which an IOExecption is thrown + * to terminate very slow transformations or a subprocess is terminated (killed). + * @return timeoutMs in milliseconds. If less than or equal to zero (the default) + * there is no timeout. + */ + protected long getTimeoutMs() + { + return limits.getTimeoutMs(); + } + + /** + * Sets a timeout (ms) on the InputStream after which an IOExecption is thrown + * to terminate very slow transformations or to terminate (kill) a subprocess. + * @param timeoutMs in milliseconds. If less than or equal to zero (the default) + * there is no timeout. + * If greater than zero the {@code readLimitTimeMs} must not be set. + */ + public void setTimeoutMs(long timeoutMs) + { + limits.setTimeoutMs(timeoutMs); + } + + /** + * Gets the limit in terms of the amount of data read (by time) to limit transformations where + * only the start of the content is needed. After this limit is reached the InputStream reports + * end of file. + * @return readLimitBytes if less than or equal to zero (the default) there is no limit. + */ + protected long getReadLimitTimeMs() + { + return limits.getReadLimitTimeMs(); + } + + /** + * Sets a limit in terms of the amount of data read (by time) to limit transformations where + * only the start of the content is needed. After this limit is reached the InputStream reports + * end of file. + * @param readLimitBytes if less than or equal to zero (the default) there is no limit. + * If greater than zero the {@code timeoutMs} must not be set. + */ + public void setReadLimitTimeMs(long readLimitTimeMs) + { + limits.setReadLimitTimeMs(readLimitTimeMs); + } + + /** + * Gets the maximum source content size, to skip transformations where + * the source is just too large to expect it to perform. If the source is larger + * the transformer indicates it is not available. + * @return maxSourceSizeKBytes if less than or equal to zero (the default) there is no limit. + */ + protected long getMaxSourceSizeKBytes() + { + return limits.getMaxSourceSizeKBytes(); + } + + /** + * Sets a maximum source content size, to skip transformations where + * the source is just too large to expect it to perform. If the source is larger + * the transformer indicates it is not available. + * @param maxSourceSizeKBytes if less than or equal to zero (the default) there is no limit. + * If greater than zero the {@code readLimitKBytes} must not be set. + */ + public void setMaxSourceSizeKBytes(long maxSourceSizeKBytes) + { + limits.setMaxSourceSizeKBytes(maxSourceSizeKBytes); + } + + /** + * Gets the limit in terms of the about of data read to limit transformations where + * only the start of the content is needed. After this limit is reached the InputStream reports + * end of file. + * @return readLimitKBytes if less than or equal to zero (the default) no limit should be applied. + */ + protected long getReadLimitKBytes() + { + return limits.getReadLimitKBytes(); + } + + /** + * Sets a limit in terms of the about of data read to limit transformations where + * only the start of the content is needed. After this limit is reached the InputStream reports + * end of file. + * @param readLimitKBytes if less than or equal to zero (the default) there is no limit. + * If greater than zero the {@code maxSourceSizeKBytes} must not be set. + */ + public void setReadLimitKBytes(long readLimitKBytes) + { + limits.setReadLimitKBytes(readLimitKBytes); + } + + /** + * Get the maximum number of pages read before an exception is thrown. + * @return If less than or equal to zero (the default) no limit should be applied. + */ + protected int getMaxPages() + { + return limits.getMaxPages(); + } + + /** + * Set the number of pages read from the source before an exception is thrown. + * + * @param maxPages the number of pages to be read from the source. If less than or equal to zero + * (the default) no limit is applied. + */ + public void setMaxPages(int maxPages) + { + limits.setMaxPages(maxPages); + } + + /** + * Get the page limit before returning EOF. + * @return If less than or equal to zero (the default) no limit should be applied. + */ + protected int getPageLimit() + { + return limits.getPageLimit(); + } + + /** + * Set the number of pages read from the source before returning EOF. + * + * @param pageLimit the number of pages to be read from the source. If less + * than or equal to zero (the default) no limit is applied. + */ + public void setPageLimit(int pageLimit) + { + limits.setPageLimit(pageLimit); + } + + /** + * Returns max and limit values for time, size and pages in a single operation. + */ + protected TransformationOptionLimits getLimits() + { + return limits; + } + + /** + * Sets max and limit values for time, size and pages in a single operation. + */ + public void setLimits(TransformationOptionLimits limits) + { + this.limits = limits; + } + + /** + * Gets the max and limit values for time, size and pages per source and target mimetype + * combination. + */ + protected Map> getMimetypeLimits() + { + return mimetypeLimits; + } + + /** + * Sets the max and limit values for time, size and pages per source and target mimetype + * combination. + */ + public void setMimetypeLimits(Map> mimetypeLimits) + { + this.mimetypeLimits = mimetypeLimits; + } + + /** + * Returns max and limit values for time, size and pages for a specified source and + * target mimetypes, combined with this Transformer's general limits and optionally + * the supplied transformation option's limits. + */ + protected TransformationOptionLimits getLimits(ContentReader reader, ContentWriter writer, + TransformationOptions options) + { + return (reader == null || writer == null) + ? limits.combine(options.getLimits()) + : getLimits(reader.getMimetype(), writer.getMimetype(), options); + } + + /** + * Returns max and limit values for time, size and pages for a specified source and + * target mimetypes, combined with this Transformer's general limits and optionally + * the supplied transformation option's limits. + */ + protected TransformationOptionLimits getLimits(String sourceMimetype, String targetMimetype, + TransformationOptions options) + { + // Get the limits for the source and target mimetypes + TransformationOptionLimits mimetypeLimits = null; + if (this.mimetypeLimits != null) + { + Map targetLimits = + this.mimetypeLimits.get(sourceMimetype); + if (targetLimits == null) + { + targetLimits = this.mimetypeLimits.get("*"); + } + if (targetLimits != null) + { + mimetypeLimits = targetLimits.get(targetMimetype); + if (mimetypeLimits == null) + { + mimetypeLimits = targetLimits.get("*"); + } + } + } + + TransformationOptionLimits combined = (mimetypeLimits == null) ? limits : limits.combine(mimetypeLimits); + return (options == null) ? combined : combined.combine(options.getLimits()); + } + + /** + * Sets the Spring bean name - only for use in debug. + */ + @Override + public void setBeanName(String beanName) + { + this.beanName = beanName; + } + + /** + * Returns the Spring bean name - only for use in debug. + */ + public String getBeanName() + { + return beanName; + } + + /** + * Pass on any limits to the reader. Will only do so if the reader is an + * {@link AbstractContentReader}. + * @param reader passed to {@link #transform(ContentReader, ContentWriter, TransformationOptions). + * @param writer passed to {@link #transform(ContentReader, ContentWriter, TransformationOptions). + * @param options passed to {@link #transform(ContentReader, ContentWriter, TransformationOptions). + */ + protected void setReaderLimits(ContentReader reader, ContentWriter writer, + TransformationOptions options) + { + if (reader instanceof AbstractContentReader) + { + AbstractContentReader abstractContentReader = (AbstractContentReader)reader; + TransformationOptionLimits limits = getLimits(reader, writer, options); + abstractContentReader.setLimits(limits); + abstractContentReader.setTransformerDebug(transformerDebug); + } + } +} \ No newline at end of file diff --git a/source/java/org/alfresco/repo/content/transform/AbstractContentTransformerLimitsTest.java b/source/java/org/alfresco/repo/content/transform/AbstractContentTransformerLimitsTest.java new file mode 100644 index 0000000000..0879af6dc0 --- /dev/null +++ b/source/java/org/alfresco/repo/content/transform/AbstractContentTransformerLimitsTest.java @@ -0,0 +1,319 @@ +/* + * Copyright (C) 2005-2011 Alfresco Software Limited. + * + * This file is part of Alfresco + * + * Alfresco is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * Alfresco is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with Alfresco. If not, see . + */ +package org.alfresco.repo.content.transform; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; + +import java.util.HashMap; +import java.util.Map; + +import org.alfresco.repo.content.AbstractContentReader; +import org.alfresco.repo.content.ContentMinimalContextTestSuite; +import org.alfresco.repo.content.AbstractContentReaderLimitTest.DummyAbstractContentReader; +import org.alfresco.service.ServiceRegistry; +import org.alfresco.service.cmr.repository.ContentReader; +import org.alfresco.service.cmr.repository.ContentWriter; +import org.alfresco.service.cmr.repository.MimetypeService; +import org.alfresco.service.cmr.repository.TransformationOptionLimits; +import org.alfresco.service.cmr.repository.TransformationOptions; +import org.junit.Before; +import org.junit.Test; +import org.springframework.context.ApplicationContext; + +/** + * Test methods that control limits in {@link AbstractContentTransformerLimits} + */ +public class AbstractContentTransformerLimitsTest +{ + private static final String A = "a"; + private static final String B = "b"; + private static final String C = "c"; + + private AbstractContentTransformerLimits transformer; + private TransformationOptionLimits limits; + private Map> mimetypeLimits; + private TransformationOptions options; + + @Before + public void setUp() throws Exception + { + ApplicationContext ctx = ContentMinimalContextTestSuite.getContext(); + ServiceRegistry serviceRegistry = (ServiceRegistry) ctx.getBean(ServiceRegistry.SERVICE_REGISTRY); + MimetypeService mimetypeService = serviceRegistry.getMimetypeService(); + TransformerDebug transformerDebug = (TransformerDebug) ctx.getBean("transformerDebug"); + + transformer = new AbstractContentTransformer2() + { + @Override + public boolean isTransformable(String sourceMimetype, String targetMimetype, + TransformationOptions options) + { + return false; + } + + @Override + protected void transformInternal(ContentReader reader, ContentWriter writer, + TransformationOptions options) throws Exception + { + } + }; + transformer.setMimetypeService(mimetypeService); + transformer.setTransformerDebug(transformerDebug); + + limits = new TransformationOptionLimits(); + options = new TransformationOptions(); + } + + private void addMimetypeLimits(String source, String target, TransformationOptionLimits limits) + { + if (mimetypeLimits == null) + { + mimetypeLimits = new HashMap>(); + } + + Map targetLimits = mimetypeLimits.get(source); + if (targetLimits == null) + { + targetLimits = new HashMap(); + mimetypeLimits.put(source, targetLimits); + } + + targetLimits.put(target, limits); + } + + @Test + public void testTimeoutMs() throws Exception + { + long value = 1234; + transformer.setTimeoutMs(value); + long actual = transformer.getTimeoutMs(); + assertEquals("Getter did not return set value", value, actual); + } + + @Test + public void testReadLimitTimeMs() throws Exception + { + long value = 1234; + transformer.setReadLimitTimeMs(value); + long actual = transformer.getReadLimitTimeMs(); + assertEquals("Getter did not return set value", value, actual); + } + + @Test + public void testMaxSourceSizeKBytes() throws Exception + { + long value = 1234; + transformer.setMaxSourceSizeKBytes(value); + long actual = transformer.getMaxSourceSizeKBytes(); + assertEquals("Getter did not return set value", value, actual); + } + + @Test + public void testReadLimitKBytes() throws Exception + { + long value = 1234; + transformer.setReadLimitKBytes(value); + long actual = transformer.getReadLimitKBytes(); + assertEquals("Getter did not return set value", value, actual); + } + + @Test + public void testMaxPages() throws Exception + { + int value = 1234; + transformer.setMaxPages(value); + int actual = transformer.getMaxPages(); + assertEquals("Getter did not return set value", value, actual); + } + + @Test + public void testPageLimit() throws Exception + { + int value = 1234; + transformer.setPageLimit(value); + int actual = transformer.getPageLimit(); + assertEquals("Getter did not return set value", value, actual); + } + + @Test + public void testMimetypeLimit() throws Exception + { + long value = 1234; + limits.setMaxSourceSizeKBytes(value); + addMimetypeLimits(A, B, limits); + + transformer.setMimetypeLimits(mimetypeLimits); + long actual = transformer.getLimits(A, B, options).getMaxSourceSizeKBytes(); + assertEquals("Getter did not return set value", value, actual); + + actual = transformer.getLimits(A, C, options).getMaxSourceSizeKBytes(); + assertEquals("Other values should not be set", -1, actual); + } + + @Test + public void testMimetypeLimitTargetWildcard() throws Exception + { + long value = 1234; + limits.setMaxSourceSizeKBytes(value); + addMimetypeLimits(A, "*", limits); + + transformer.setMimetypeLimits(mimetypeLimits); + long actual = transformer.getLimits(A, B, options).getMaxSourceSizeKBytes(); + assertEquals("Getter did not return set value", value, actual); + + actual = transformer.getLimits(B, A, options).getMaxSourceSizeKBytes(); + assertEquals("Other values should not be set", -1, actual); + } + + @Test + public void testMimetypeLimitSourceWildcard() throws Exception + { + long value = 1234; + limits.setMaxSourceSizeKBytes(value); + addMimetypeLimits("*", B, limits); + + transformer.setMimetypeLimits(mimetypeLimits); + long actual = transformer.getLimits(A, B, options).getMaxSourceSizeKBytes(); + assertEquals("Getter did not return set value", value, actual); + + actual = transformer.getLimits(B, A, options).getMaxSourceSizeKBytes(); + assertEquals("Other values should not be set", -1, actual); + } + + @Test + public void testPassedInOptions() throws Exception + { + long value = 1234; + limits.setMaxSourceSizeKBytes(value+1); + addMimetypeLimits(A, B, limits); + + transformer.setMimetypeLimits(mimetypeLimits); + long actual = transformer.getLimits(A, B, options).getMaxSourceSizeKBytes(); + assertEquals("Getter did not return set value", value+1, actual); + + options.setMaxSourceSizeKBytes(value); + actual = transformer.getLimits(A, B, options).getMaxSourceSizeKBytes(); + assertEquals("Getter did not return set value", value, actual); + } + + @Test + public void testIsTransformableSize() throws Exception + { + long kValue = 12; + long byteValue = kValue*1024; + + // Not set mimetype limits yet + assertTrue("No limits so should have been ok", + transformer.isTransformableSize(A, byteValue+1, B, options)); + + // Set limit for A to B mimetypes and test + limits.setMaxSourceSizeKBytes(kValue); + addMimetypeLimits(A, B, limits); + transformer.setMimetypeLimits(mimetypeLimits); + + assertTrue("Size is less than limit so should have been ok", + transformer.isTransformableSize(A, byteValue-1, B, options)); + assertTrue("Size is equal to limit so should have been ok", + transformer.isTransformableSize(A, byteValue, B, options)); + assertFalse("Size is greater than limit so should not have failed", + transformer.isTransformableSize(A, byteValue+1, B, options)); + + // With a mimetype that does not have any specific limits + assertTrue("No limits so should have been ok", + transformer.isTransformableSize(C, byteValue+1, B, options)); + assertTrue("No limits so should have been ok", + transformer.isTransformableSize(A, byteValue+1, C, options)); + assertTrue("No limits so should have been ok", + transformer.isTransformableSize(C, byteValue+1, C, options)); + + // Clear the mimetype limits and double check + limits.setMaxSourceSizeKBytes(-1); + + assertTrue("No limits so should have been ok", + transformer.isTransformableSize(A, byteValue+1, B, options)); + + // Check for combinations with transformer limits + + // a) Using just transformer limit to start with + transformer.setMaxSourceSizeKBytes(kValue); + assertTrue("Size is equal to limit so should have been ok", + transformer.isTransformableSize(A, byteValue, B, options)); + assertFalse("Size is greater than limit so should not have failed", + transformer.isTransformableSize(A, byteValue+1, B, options)); + + // b) combination where transformer limit is used + transformer.setMaxSourceSizeKBytes(kValue+1); + limits.setMaxSourceSizeKBytes(kValue); + assertTrue("Size is equal to limit so should have been ok", + transformer.isTransformableSize(A, byteValue, B, options)); + assertFalse("Size is greater than limit so should not have failed", + transformer.isTransformableSize(A, byteValue+1, B, options)); + + // c) combination where mimetype limit is used + transformer.setMaxSourceSizeKBytes(kValue); + limits.setMaxSourceSizeKBytes(kValue+1); + assertTrue("Size is equal to limit so should have been ok", + transformer.isTransformableSize(A, byteValue, B, options)); + assertFalse("Size is greater than limit so should not have failed", + transformer.isTransformableSize(A, byteValue+1, B, options)); + } + + @Test + public void testIsTransformableSizeWithPageLimit() throws Exception + { + long kValue = 12; + long byteValue = kValue*1024; + + transformer.setMaxSourceSizeKBytes(kValue); + transformer.setPageLimitsSuported(true); + + // Test works as normal before setting the pageLimit + assertTrue("Size is less than limit so should have been ok", + transformer.isTransformableSize(A, byteValue-1, B, options)); + assertTrue("Size is equal to limit so should have been ok", + transformer.isTransformableSize(A, byteValue, B, options)); + assertFalse("Size is greater than limit so should not have failed", + transformer.isTransformableSize(A, byteValue+1, B, options)); + + // test with pageLimit set + options.getLimits().setPageLimit(1); + assertTrue("Size is greater than limit BUT pageLimit is set so should have been ok", + transformer.isTransformableSize(A, byteValue+1, B, options)); + } + + @Test + public void testSetReaderLimits() throws Exception + { + AbstractContentReader reader = new DummyAbstractContentReader(0, 0); + + long value = 1234; + transformer.setTimeoutMs(value); + + assertEquals("Limit should not have been set in the reader", null, reader.getLimits()); + + transformer.setReaderLimits(reader, null, options); + assertEquals("Limit should have been set in the reader", value, reader.getLimits().getTimeoutMs()); + + options.setTimeoutMs(--value); + transformer.setReaderLimits(reader, null, options); + assertEquals("Limit should have been set in the reader", value, reader.getLimits().getTimeoutMs()); + } +} diff --git a/source/java/org/alfresco/repo/content/transform/AbstractContentTransformerTest.java b/source/java/org/alfresco/repo/content/transform/AbstractContentTransformerTest.java index 4c7349eb97..1e66141149 100644 --- a/source/java/org/alfresco/repo/content/transform/AbstractContentTransformerTest.java +++ b/source/java/org/alfresco/repo/content/transform/AbstractContentTransformerTest.java @@ -68,6 +68,7 @@ public abstract class AbstractContentTransformerTest extends TestCase protected ServiceRegistry serviceRegistry; protected MimetypeService mimetypeService; + protected TransformerDebug transformerDebug; /** * Fetches a transformer to test for a given transformation. The transformer @@ -94,6 +95,7 @@ public abstract class AbstractContentTransformerTest extends TestCase // Grab other useful beans serviceRegistry = (ServiceRegistry) ctx.getBean(ServiceRegistry.SERVICE_REGISTRY); mimetypeService = serviceRegistry.getMimetypeService(); + transformerDebug = (TransformerDebug) ctx.getBean("transformerDebug"); // perform a little cleaning up long now = System.currentTimeMillis(); TempFileProvider.TempFileCleanerJob.removeFiles(now); @@ -212,7 +214,14 @@ public abstract class AbstractContentTransformerTest extends TestCase { // attempt to get a source file for each mimetype String[] quickFiles = getQuickFilenames(sourceMimetype); - sb.append(" Source Files: ").append(quickFiles).append("\n"); + sb.append(" Source Files: "); + for (String quickFile: quickFiles) + { + sb.append(quickFile); + sb.append(' '); + } + sb.append("\n"); + for (String quickFile : quickFiles) { @@ -232,7 +241,7 @@ public abstract class AbstractContentTransformerTest extends TestCase // must we test the transformation? ContentTransformer transformer = getTransformer(sourceMimetype, targetMimetype); - if (transformer == null || transformer.isTransformable(sourceMimetype, targetMimetype, null) == false) + if (transformer == null || transformer.isTransformable(sourceMimetype, -1, targetMimetype, null) == false) { // no transformer continue; diff --git a/source/java/org/alfresco/repo/content/transform/ArchiveContentTransformerTest.java b/source/java/org/alfresco/repo/content/transform/ArchiveContentTransformerTest.java index 9b6f6da54d..880f7febdf 100644 --- a/source/java/org/alfresco/repo/content/transform/ArchiveContentTransformerTest.java +++ b/source/java/org/alfresco/repo/content/transform/ArchiveContentTransformerTest.java @@ -45,6 +45,8 @@ public class ArchiveContentTransformerTest extends AbstractContentTransformerTes super.setUp(); transformer = new ArchiveContentTransformer(); + transformer.setMimetypeService(mimetypeService); + transformer.setTransformerDebug(transformerDebug); } protected ContentTransformer getTransformer(String sourceMimetype, String targetMimetype) @@ -54,9 +56,9 @@ public class ArchiveContentTransformerTest extends AbstractContentTransformerTes public void testIsTransformable() throws Exception { - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_ZIP, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); - assertTrue(transformer.isTransformable("application/x-tar", MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); - assertTrue(transformer.isTransformable("application/x-gtar", MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_ZIP, -1, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); + assertTrue(transformer.isTransformable("application/x-tar", -1, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); + assertTrue(transformer.isTransformable("application/x-gtar", -1, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); } @Override diff --git a/source/java/org/alfresco/repo/content/transform/BinaryPassThroughContentTransformerTest.java b/source/java/org/alfresco/repo/content/transform/BinaryPassThroughContentTransformerTest.java index 3c3dda2f20..565251833e 100644 --- a/source/java/org/alfresco/repo/content/transform/BinaryPassThroughContentTransformerTest.java +++ b/source/java/org/alfresco/repo/content/transform/BinaryPassThroughContentTransformerTest.java @@ -1,5 +1,5 @@ /* - * Copyright (C) 2005-2010 Alfresco Software Limited. + * Copyright (C) 2005-2011 Alfresco Software Limited. * * This file is part of Alfresco * @@ -28,7 +28,7 @@ import org.alfresco.service.cmr.repository.TransformationOptions; */ public class BinaryPassThroughContentTransformerTest extends AbstractContentTransformerTest { - private ContentTransformer transformer; + private BinaryPassThroughContentTransformer transformer; @Override public void setUp() throws Exception @@ -36,6 +36,8 @@ public class BinaryPassThroughContentTransformerTest extends AbstractContentTran super.setUp(); transformer = new BinaryPassThroughContentTransformer(); + transformer.setMimetypeService(mimetypeService); + transformer.setTransformerDebug(transformerDebug); } /** @@ -51,13 +53,13 @@ public class BinaryPassThroughContentTransformerTest extends AbstractContentTran TransformationOptions options = new TransformationOptions(); boolean reliability = false; - reliability = transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, MimetypeMap.MIMETYPE_TEXT_PLAIN, options); + reliability = transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, -1, MimetypeMap.MIMETYPE_TEXT_PLAIN, options); assertFalse("Mimetype should not be supported", reliability); - reliability = transformer.isTransformable(MimetypeMap.MIMETYPE_XML, MimetypeMap.MIMETYPE_XML, options); + reliability = transformer.isTransformable(MimetypeMap.MIMETYPE_XML, -1, MimetypeMap.MIMETYPE_XML, options); assertFalse("Mimetype should not be supported", reliability); - reliability = transformer.isTransformable(MimetypeMap.MIMETYPE_WORD, MimetypeMap.MIMETYPE_WORD, options); + reliability = transformer.isTransformable(MimetypeMap.MIMETYPE_WORD, -1, MimetypeMap.MIMETYPE_WORD, options); assertTrue("Mimetype should be supported", reliability); - reliability = transformer.isTransformable(MimetypeMap.MIMETYPE_EXCEL, MimetypeMap.MIMETYPE_EXCEL, options); + reliability = transformer.isTransformable(MimetypeMap.MIMETYPE_EXCEL, -1, MimetypeMap.MIMETYPE_EXCEL, options); assertTrue("Mimetype should be supported", reliability); } } diff --git a/source/java/org/alfresco/repo/content/transform/ComplexContentTransformer.java b/source/java/org/alfresco/repo/content/transform/ComplexContentTransformer.java index a6caa6d52a..7063739cd6 100644 --- a/source/java/org/alfresco/repo/content/transform/ComplexContentTransformer.java +++ b/source/java/org/alfresco/repo/content/transform/ComplexContentTransformer.java @@ -130,7 +130,7 @@ public class ComplexContentTransformer extends AbstractContentTransformer2 imple * * @see org.alfresco.repo.content.transform.ContentTransformer#isTransformable(java.lang.String, java.lang.String, org.alfresco.service.cmr.repository.TransformationOptions) */ - public boolean isTransformable(String sourceMimetype, String targetMimetype, TransformationOptions options) + public boolean isTransformable(String sourceMimetype, long sourceSize, String targetMimetype, TransformationOptions options) { boolean result = true; String currentSourceMimetype = sourceMimetype; @@ -178,6 +178,7 @@ public class ComplexContentTransformer extends AbstractContentTransformer2 imple } } + boolean first = true; Iterator transformerIterator = transformers.iterator(); Iterator intermediateMimetypeIterator = intermediateMimetypes.iterator(); while (transformerIterator.hasNext()) @@ -195,8 +196,9 @@ public class ComplexContentTransformer extends AbstractContentTransformer2 imple currentTargetMimetype = intermediateMimetypeIterator.next(); } - // check we can tranform the current stage - if (transformer.isTransformable(currentSourceMimetype, currentTargetMimetype, options) == false) + // check we can tranform the current stage (using -1 if not the first stage as we can't know the size) + long size = first ? sourceSize : -1; + if (transformer.isTransformable(currentSourceMimetype, size, currentTargetMimetype, options) == false) { result = false; break; @@ -204,6 +206,12 @@ public class ComplexContentTransformer extends AbstractContentTransformer2 imple // move on currentSourceMimetype = currentTargetMimetype; + first = false; + } + + if (result && !isTransformableSize(sourceMimetype, sourceSize, targetMimetype, options)) + { + result = false; } return result; @@ -255,4 +263,15 @@ public class ComplexContentTransformer extends AbstractContentTransformer2 imple { return Collections.unmodifiableList(intermediateMimetypes); } + + /** + * @deprecated This method should no longer be called as the overloaded method + * that calls it has the overridden. + */ + @Override + public boolean isTransformable(String sourceMimetype, String targetMimetype, + TransformationOptions options) + { + return false; + } } diff --git a/source/java/org/alfresco/repo/content/transform/ComplexContentTransformerTest.java b/source/java/org/alfresco/repo/content/transform/ComplexContentTransformerTest.java index ae4548a840..6316d101cd 100644 --- a/source/java/org/alfresco/repo/content/transform/ComplexContentTransformerTest.java +++ b/source/java/org/alfresco/repo/content/transform/ComplexContentTransformerTest.java @@ -1,5 +1,5 @@ /* - * Copyright (C) 2005-2010 Alfresco Software Limited. + * Copyright (C) 2005-2011 Alfresco Software Limited. * * This file is part of Alfresco * @@ -45,12 +45,12 @@ public class ComplexContentTransformerTest extends AbstractContentTransformerTes ContentTransformer unoTransformer = (ContentTransformer) ctx.getBean("transformer.OpenOffice"); ContentTransformer pdfBoxTransformer = (ContentTransformer) ctx.getBean("transformer.PdfBox"); // make sure that they are working for this test - if (unoTransformer.isTransformable(MimetypeMap.MIMETYPE_PPT, MimetypeMap.MIMETYPE_PDF, new TransformationOptions()) == false) + if (unoTransformer.isTransformable(MimetypeMap.MIMETYPE_PPT, -1, MimetypeMap.MIMETYPE_PDF, new TransformationOptions()) == false) { isAvailable = false; return; } - else if (pdfBoxTransformer.isTransformable(MimetypeMap.MIMETYPE_PDF, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions()) == false) + else if (pdfBoxTransformer.isTransformable(MimetypeMap.MIMETYPE_PDF, -1, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions()) == false) { isAvailable = false; return; @@ -62,6 +62,7 @@ public class ComplexContentTransformerTest extends AbstractContentTransformerTes transformer = new ComplexContentTransformer(); transformer.setMimetypeService(mimetypeService); + transformer.setTransformerDebug(transformerDebug); // set the transformer list List transformers = new ArrayList(2); transformers.add(unoTransformer); @@ -86,9 +87,9 @@ public class ComplexContentTransformerTest extends AbstractContentTransformerTes { return; } - boolean reliability = transformer.isTransformable(MimetypeMap.MIMETYPE_PPT, MimetypeMap.MIMETYPE_PDF, new TransformationOptions()); + boolean reliability = transformer.isTransformable(MimetypeMap.MIMETYPE_PPT, -1, MimetypeMap.MIMETYPE_PDF, new TransformationOptions()); assertEquals("Mimetype should not be supported", false, reliability); - reliability = transformer.isTransformable(MimetypeMap.MIMETYPE_PPT, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions()); + reliability = transformer.isTransformable(MimetypeMap.MIMETYPE_PPT, -1, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions()); assertEquals("Mimetype should be supported", true, reliability); } } diff --git a/source/java/org/alfresco/repo/content/transform/ContentTransformer.java b/source/java/org/alfresco/repo/content/transform/ContentTransformer.java index f18b3cbde2..f644ba6aef 100644 --- a/source/java/org/alfresco/repo/content/transform/ContentTransformer.java +++ b/source/java/org/alfresco/repo/content/transform/ContentTransformer.java @@ -1,5 +1,5 @@ /* - * Copyright (C) 2005-2010 Alfresco Software Limited. + * Copyright (C) 2005-2011 Alfresco Software Limited. * * This file is part of Alfresco * @@ -33,16 +33,22 @@ import org.alfresco.service.cmr.repository.TransformationOptions; */ public interface ContentTransformer extends ContentWorker { + /** + * @deprecated use version with extra sourceSize parameter. + */ + public boolean isTransformable(String sourceMimetype, String targetMimetype, TransformationOptions options); + /** * Indicates whether the provided source mimetype can be transformed into the target mimetype with * the options specified by this content transformer. * * @param sourceMimetype the source mimetype - * @param destinationMimetype the destination mimetype + * @param sourceSize the size (bytes) of the source. If negative it is unknown. + * @param targetMimetype the target mimetype * @param options the transformation options * @return boolean true if this content transformer can satify the mimetypes and options specified, false otherwise */ - public boolean isTransformable(String sourceMimetype, String targetMimetype, TransformationOptions options); + public boolean isTransformable(String sourceMimetype, long sourceSize, String targetMimetype, TransformationOptions options); /** * Indicates whether given the provided transformation parmaters this transformer can prvide an explict diff --git a/source/java/org/alfresco/repo/content/transform/ContentTransformerRegistry.java b/source/java/org/alfresco/repo/content/transform/ContentTransformerRegistry.java index d15a3364d4..c1206755db 100644 --- a/source/java/org/alfresco/repo/content/transform/ContentTransformerRegistry.java +++ b/source/java/org/alfresco/repo/content/transform/ContentTransformerRegistry.java @@ -71,13 +71,21 @@ public class ContentTransformerRegistry } /** - * Gets the best transformer possible. This is a combination of the most reliable - * and the most performant transformer. + * @deprecated use overloaded version with sourceSize parameter. */ public ContentTransformer getTransformer(String sourceMimetype, String targetMimetype, TransformationOptions options) + { + return getTransformer(sourceMimetype, -1, targetMimetype, options); + } + + /** + * Gets the best transformer possible. This is a combination of the most reliable + * and the most performant transformer. + */ + public ContentTransformer getTransformer(String sourceMimetype, long sourceSize, String targetMimetype, TransformationOptions options) { // Get the sorted list of transformers - List transformers = getActiveTransformers(sourceMimetype, targetMimetype, options); + List transformers = getActiveTransformers(sourceMimetype, sourceSize, targetMimetype, options); // select the most performant transformer ContentTransformer bestTransformer = null; @@ -92,10 +100,10 @@ public class ContentTransformerRegistry /** * @since 3.5 */ - public List getActiveTransformers(String sourceMimetype, String targetMimetype, TransformationOptions options) + public List getActiveTransformers(String sourceMimetype, long sourceSize, String targetMimetype, TransformationOptions options) { // Get the list of transformers - List transformers = findTransformers(sourceMimetype, targetMimetype, options); + List transformers = findTransformers(sourceMimetype, sourceSize, targetMimetype, options); final Map activeTransformers = new HashMap(); @@ -103,7 +111,7 @@ public class ContentTransformerRegistry for (ContentTransformer transformer : transformers) { // Transformability can be dynamic, i.e. it may have become unusable - if (transformer.isTransformable(sourceMimetype, targetMimetype, options) == false) + if (transformer.isTransformable(sourceMimetype, sourceSize, targetMimetype, options) == false) { // It is unreliable now. continue; @@ -135,10 +143,10 @@ public class ContentTransformerRegistry * @return Returns best transformer for the translation - null if all * score 0.0 on reliability */ - private List findTransformers(String sourceMimetype, String targetMimetype, TransformationOptions options) + private List findTransformers(String sourceMimetype, long sourceSize, String targetMimetype, TransformationOptions options) { // search for a simple transformer that can do the job - List transformers = findDirectTransformers(sourceMimetype, targetMimetype, options); + List transformers = findDirectTransformers(sourceMimetype, sourceSize, targetMimetype, options); // get the complex transformers that can do the job List complexTransformers = findComplexTransformer(sourceMimetype, targetMimetype, options); transformers.addAll(complexTransformers); @@ -161,7 +169,7 @@ public class ContentTransformerRegistry * @return Returns the most reliable transformers for the translation - empty list if there * are none. */ - private List findDirectTransformers(String sourceMimetype, String targetMimetype, TransformationOptions options) + private List findDirectTransformers(String sourceMimetype, long sourceSize, String targetMimetype, TransformationOptions options) { //double maxReliability = 0.0; @@ -171,7 +179,7 @@ public class ContentTransformerRegistry // loop through transformers for (ContentTransformer transformer : this.transformers) { - if (transformer.isTransformable(sourceMimetype, targetMimetype, options) == true) + if (transformer.isTransformable(sourceMimetype, sourceSize, targetMimetype, options) == true) { if (transformer.isExplicitTransformation(sourceMimetype, targetMimetype, options) == true) { diff --git a/source/java/org/alfresco/repo/content/transform/ContentTransformerRegistryTest.java b/source/java/org/alfresco/repo/content/transform/ContentTransformerRegistryTest.java index f989ef3c53..ea4fb31bf1 100644 --- a/source/java/org/alfresco/repo/content/transform/ContentTransformerRegistryTest.java +++ b/source/java/org/alfresco/repo/content/transform/ContentTransformerRegistryTest.java @@ -1,5 +1,5 @@ /* - * Copyright (C) 2005-2010 Alfresco Software Limited. + * Copyright (C) 2005-2011 Alfresco Software Limited. * * This file is part of Alfresco * @@ -72,17 +72,17 @@ public class ContentTransformerRegistryTest extends AbstractContentTransformerTe // create the dummyRegistry dummyRegistry = new ContentTransformerRegistry(); // create some dummy transformers for reliability tests - new DummyTransformer(mimetypeService, dummyRegistry, A, B, 10L); - new DummyTransformer(mimetypeService, dummyRegistry, A, B, 10L); - new DummyTransformer(mimetypeService, dummyRegistry, A, C, 10L); - new DummyTransformer(mimetypeService, dummyRegistry, A, C, 10L); - new DummyTransformer(mimetypeService, dummyRegistry, B, C, 10L); + new DummyTransformer(mimetypeService, transformerDebug, dummyRegistry, A, B, 10L); + new DummyTransformer(mimetypeService, transformerDebug, dummyRegistry, A, B, 10L); + new DummyTransformer(mimetypeService, transformerDebug, dummyRegistry, A, C, 10L); + new DummyTransformer(mimetypeService, transformerDebug, dummyRegistry, A, C, 10L); + new DummyTransformer(mimetypeService, transformerDebug, dummyRegistry, B, C, 10L); // create some dummy transformers for speed tests - new DummyTransformer(mimetypeService, dummyRegistry, A, D, 20L); - new DummyTransformer(mimetypeService, dummyRegistry, A, D, 30L); - new DummyTransformer(mimetypeService, dummyRegistry, A, D, 10L); // the fast one - new DummyTransformer(mimetypeService, dummyRegistry, A, D, 25L); - new DummyTransformer(mimetypeService, dummyRegistry, A, D, 25L); + new DummyTransformer(mimetypeService, transformerDebug, dummyRegistry, A, D, 20L); + new DummyTransformer(mimetypeService, transformerDebug, dummyRegistry, A, D, 30L); + new DummyTransformer(mimetypeService, transformerDebug, dummyRegistry, A, D, 10L); // the fast one + new DummyTransformer(mimetypeService, transformerDebug, dummyRegistry, A, D, 25L); + new DummyTransformer(mimetypeService, transformerDebug, dummyRegistry, A, D, 25L); } /** @@ -99,17 +99,17 @@ public class ContentTransformerRegistryTest extends AbstractContentTransformerTe */ protected ContentTransformer getTransformer(String sourceMimetype, String targetMimetype, TransformationOptions options) { - return registry.getTransformer(sourceMimetype, targetMimetype, options); + return registry.getTransformer(sourceMimetype, -1, targetMimetype, options); } public void testNullRetrieval() throws Exception { ContentTransformer transformer = null; - transformer = dummyRegistry.getTransformer(C, B, OPTIONS); + transformer = dummyRegistry.getTransformer(C, -1, B, OPTIONS); assertNull("No transformer expected", transformer); - transformer = dummyRegistry.getTransformer(C, A, OPTIONS); + transformer = dummyRegistry.getTransformer(C, -1, A, OPTIONS); assertNull("No transformer expected", transformer); - transformer = dummyRegistry.getTransformer(B, A, OPTIONS); + transformer = dummyRegistry.getTransformer(B, -1, A, OPTIONS); assertNull("No transformer expected", transformer); } @@ -117,11 +117,11 @@ public class ContentTransformerRegistryTest extends AbstractContentTransformerTe { ContentTransformer transformer = null; // B -> C expect true - transformer = dummyRegistry.getTransformer(B, C, OPTIONS); + transformer = dummyRegistry.getTransformer(B, -1, C, OPTIONS); //transformer = dummyRegistry.getTransformer(B, C, OPTIONS); assertNotNull("No transformer found", transformer); - assertTrue("Incorrect reliability", transformer.isTransformable(B, C, OPTIONS)); - assertFalse("Incorrect reliability", transformer.isTransformable(C, B, OPTIONS)); + assertTrue("Incorrect reliability", transformer.isTransformable(B, -1, C, OPTIONS)); + assertFalse("Incorrect reliability", transformer.isTransformable(C, -1, B, OPTIONS)); } /** @@ -132,13 +132,13 @@ public class ContentTransformerRegistryTest extends AbstractContentTransformerTe public void testPerformanceRetrieval() throws Exception { // A -> D expect 1.0, 10ms - ContentTransformer transformer1 = dummyRegistry.getTransformer(A, D, OPTIONS); - assertTrue("Incorrect reliability", transformer1.isTransformable(A, D, OPTIONS)); - assertFalse("Incorrect reliability", transformer1.isTransformable(D, A, OPTIONS)); + ContentTransformer transformer1 = dummyRegistry.getTransformer(A, -1, D, OPTIONS); + assertTrue("Incorrect reliability", transformer1.isTransformable(A, -1, D, OPTIONS)); + assertFalse("Incorrect reliability", transformer1.isTransformable(D, -1, A, OPTIONS)); assertEquals("Incorrect transformation time", 10L, transformer1.getTransformationTime()); // A -> D has 10, 20, 25, 25, 30 - List activeTransformers = dummyRegistry.getActiveTransformers(A, D, OPTIONS); + List activeTransformers = dummyRegistry.getActiveTransformers(A, -1, D, OPTIONS); assertEquals("Not all found", 5, activeTransformers.size()); assertEquals("Incorrect order", 10L, activeTransformers.get(0).getTransformationTime()); assertEquals("Incorrect order", 20L, activeTransformers.get(1).getTransformationTime()); @@ -150,7 +150,7 @@ public class ContentTransformerRegistryTest extends AbstractContentTransformerTe ((DummyTransformer)activeTransformers.get(2)).disable(); ((DummyTransformer)activeTransformers.get(4)).disable(); - activeTransformers = dummyRegistry.getActiveTransformers(A, D, OPTIONS); + activeTransformers = dummyRegistry.getActiveTransformers(A, -1, D, OPTIONS); assertEquals("Not all found", 3, activeTransformers.size()); assertEquals("Incorrect order", 10L, activeTransformers.get(0).getTransformationTime()); assertEquals("Incorrect order", 20L, activeTransformers.get(1).getTransformationTime()); @@ -161,15 +161,15 @@ public class ContentTransformerRegistryTest extends AbstractContentTransformerTe { ContentTransformer transformer = null; // A -> B expect 0.6 - transformer = dummyRegistry.getTransformer(A, B, OPTIONS); + transformer = dummyRegistry.getTransformer(A, -1, B, OPTIONS); assertNotNull("No transformer found", transformer); - assertTrue("Incorrect reliability", transformer.isTransformable(A, B, OPTIONS)); - assertFalse("Incorrect reliability", transformer.isTransformable(B, A, OPTIONS)); + assertTrue("Incorrect reliability", transformer.isTransformable(A, -1, B, OPTIONS)); + assertFalse("Incorrect reliability", transformer.isTransformable(B, -1, A, OPTIONS)); // A -> C expect 1.0 - transformer = dummyRegistry.getTransformer(A, C, OPTIONS); + transformer = dummyRegistry.getTransformer(A, -1, C, OPTIONS); assertNotNull("No transformer found", transformer); - assertTrue("Incorrect reliability", transformer.isTransformable(A, C, OPTIONS)); - assertFalse("Incorrect reliability", transformer.isTransformable(C, A, OPTIONS)); + assertTrue("Incorrect reliability", transformer.isTransformable(A, -1, C, OPTIONS)); + assertFalse("Incorrect reliability", transformer.isTransformable(C, -1, A, OPTIONS)); } /** @@ -180,9 +180,9 @@ public class ContentTransformerRegistryTest extends AbstractContentTransformerTe { AbstractContentTransformer2 dummyTransformer = new DummyTransformer( mimetypeService, - dummyRegistry, - MimetypeMap.MIMETYPE_FLASH, MimetypeMap.MIMETYPE_EXCEL, - 12345); + transformerDebug, + dummyRegistry, MimetypeMap.MIMETYPE_FLASH, + MimetypeMap.MIMETYPE_EXCEL, 12345); // set an explicit transformation ExplictTransformationDetails key = new ExplictTransformationDetails( @@ -193,7 +193,7 @@ public class ContentTransformerRegistryTest extends AbstractContentTransformerTe dummyTransformer.register(); // get the appropriate transformer for the bizarre mapping - ContentTransformer checkTransformer = dummyRegistry.getTransformer(MimetypeMap.MIMETYPE_FLASH, MimetypeMap.MIMETYPE_EXCEL, OPTIONS); + ContentTransformer checkTransformer = dummyRegistry.getTransformer(MimetypeMap.MIMETYPE_FLASH, -1, MimetypeMap.MIMETYPE_EXCEL, OPTIONS); assertNotNull("No explicit transformer found", checkTransformer); assertTrue("Expected explicit transformer", dummyTransformer == checkTransformer); @@ -212,11 +212,12 @@ public class ContentTransformerRegistryTest extends AbstractContentTransformerTe public DummyTransformer( MimetypeService mimetypeService, - ContentTransformerRegistry registry, - String sourceMimetype, String targetMimetype, - long transformationTime) + TransformerDebug transformerDebug, + ContentTransformerRegistry registry, String sourceMimetype, + String targetMimetype, long transformationTime) { super.setMimetypeService(mimetypeService); + super.setTransformerDebug(transformerDebug); super.setRegistry(registry); this.sourceMimetype = sourceMimetype; this.targetMimetype = targetMimetype; diff --git a/source/java/org/alfresco/repo/content/transform/FailoverContentTransformer.java b/source/java/org/alfresco/repo/content/transform/FailoverContentTransformer.java index 96fc5066aa..ac6ada5789 100644 --- a/source/java/org/alfresco/repo/content/transform/FailoverContentTransformer.java +++ b/source/java/org/alfresco/repo/content/transform/FailoverContentTransformer.java @@ -80,9 +80,9 @@ public class FailoverContentTransformer extends AbstractContentTransformer2 impl /** * - * @see org.alfresco.repo.content.transform.ContentTransformer#isTransformable(java.lang.String, java.lang.String, org.alfresco.service.cmr.repository.TransformationOptions) + * @see org.alfresco.repo.content.transform.ContentTransformer#isTransformable(java.lang.String, long sourceSize, java.lang.String, org.alfresco.service.cmr.repository.TransformationOptions) */ - public boolean isTransformable(String sourceMimetype, String targetMimetype, TransformationOptions options) + public boolean isTransformable(String sourceMimetype, long sourceSize, String targetMimetype, TransformationOptions options) { // For this transformer to be considered operational, there must be at least one transformer // in the chain that can perform for us. @@ -90,15 +90,27 @@ public class FailoverContentTransformer extends AbstractContentTransformer2 impl for (ContentTransformer ct : this.transformers) { - if (ct.isTransformable(sourceMimetype, targetMimetype, options)) + if (ct.isTransformable(sourceMimetype, sourceSize, targetMimetype, options)) { - result = true; + // There may be size limits on this transformer as well as those it contains. + result = isTransformableSize(sourceMimetype, sourceSize, targetMimetype, options); break; } } return result; } + + /** + * @deprecated This method should no longer be called as the overloaded method + * that calls it has the overridden. + */ + @Override + public boolean isTransformable(String sourceMimetype, String targetMimetype, + TransformationOptions options) + { + return false; + } public boolean isExplicitTransformation(String sourceMimetype, String targetMimetype, TransformationOptions options) { @@ -130,7 +142,7 @@ public class FailoverContentTransformer extends AbstractContentTransformer2 impl // then move on to the next transformer. In the event that they all fail, we will throw // the final exception. Exception transformationException = null; - + for (int i = 0; i < transformers.size(); i++) { int oneBasedCount = i + 1; @@ -198,6 +210,7 @@ public class FailoverContentTransformer extends AbstractContentTransformer2 impl // At this point we have tried all transformers in the sequence without apparent success. if (transformationException != null) { + transformerDebug.debug("No more transformations to failover to"); if (logger.isDebugEnabled()) { logger.debug("All transformations were unsuccessful. Throwing latest exception.", transformationException); diff --git a/source/java/org/alfresco/repo/content/transform/FailoverContentTransformerTest.java b/source/java/org/alfresco/repo/content/transform/FailoverContentTransformerTest.java index ae61de91fe..f0faf92015 100644 --- a/source/java/org/alfresco/repo/content/transform/FailoverContentTransformerTest.java +++ b/source/java/org/alfresco/repo/content/transform/FailoverContentTransformerTest.java @@ -1,5 +1,5 @@ /* - * Copyright (C) 2005-2010 Alfresco Software Limited. + * Copyright (C) 2005-2011 Alfresco Software Limited. * * This file is part of Alfresco * @@ -56,6 +56,7 @@ public class FailoverContentTransformerTest extends AbstractContentTransformerTe transformer = (FailoverContentTransformer) failoverAppContext.getBean("transformer.failover.Test-FailThenSucceed"); transformer.setMimetypeService(mimetypeService); + transformer.setTransformerDebug(transformerDebug); } /** @@ -70,7 +71,7 @@ public class FailoverContentTransformerTest extends AbstractContentTransformerTe { // The MIME types here are rather arbitrary - boolean reliability = transformer.isTransformable(sourceMimeType, targetMimeType, new TransformationOptions()); + boolean reliability = transformer.isTransformable(sourceMimeType, -1, targetMimeType, new TransformationOptions()); assertEquals("Mimetype should be supported", true, reliability); } } diff --git a/source/java/org/alfresco/repo/content/transform/HtmlParserContentTransformerTest.java b/source/java/org/alfresco/repo/content/transform/HtmlParserContentTransformerTest.java index ff5b816cc3..feadb5e3fd 100644 --- a/source/java/org/alfresco/repo/content/transform/HtmlParserContentTransformerTest.java +++ b/source/java/org/alfresco/repo/content/transform/HtmlParserContentTransformerTest.java @@ -28,13 +28,15 @@ import org.alfresco.service.cmr.repository.TransformationOptions; */ public class HtmlParserContentTransformerTest extends AbstractContentTransformerTest { - private ContentTransformer transformer; + private HtmlParserContentTransformer transformer; @Override public void setUp() throws Exception { super.setUp(); transformer = new HtmlParserContentTransformer(); + transformer.setMimetypeService(mimetypeService); + transformer.setTransformerDebug(transformerDebug); } protected ContentTransformer getTransformer(String sourceMimetype, String targetMimetype) @@ -50,11 +52,11 @@ public class HtmlParserContentTransformerTest extends AbstractContentTransformer public void checkIsTransformable() throws Exception { // check reliability - boolean reliability = transformer.isTransformable(MimetypeMap.MIMETYPE_HTML, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions()); + boolean reliability = transformer.isTransformable(MimetypeMap.MIMETYPE_HTML, -1, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions()); assertTrue(reliability); // plain text to plain text is supported // check other way around - reliability = transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, MimetypeMap.MIMETYPE_HTML, new TransformationOptions()); + reliability = transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, -1, MimetypeMap.MIMETYPE_HTML, new TransformationOptions()); assertFalse(reliability); // plain text to plain text is not supported } } diff --git a/source/java/org/alfresco/repo/content/transform/MailContentTransformerTest.java b/source/java/org/alfresco/repo/content/transform/MailContentTransformerTest.java index e199e49c54..3b410590f1 100644 --- a/source/java/org/alfresco/repo/content/transform/MailContentTransformerTest.java +++ b/source/java/org/alfresco/repo/content/transform/MailContentTransformerTest.java @@ -1,5 +1,5 @@ /* - * Copyright (C) 2005-2010 Alfresco Software Limited. + * Copyright (C) 2005-2011 Alfresco Software Limited. * * This file is part of Alfresco * @@ -35,7 +35,7 @@ import org.alfresco.util.TempFileProvider; */ public class MailContentTransformerTest extends AbstractContentTransformerTest { - private ContentTransformer transformer; + private MailContentTransformer transformer; @Override public void setUp() throws Exception @@ -43,6 +43,8 @@ public class MailContentTransformerTest extends AbstractContentTransformerTest super.setUp(); transformer = new MailContentTransformer(); + transformer.setMimetypeService(mimetypeService); + transformer.setTransformerDebug(transformerDebug); } /** @@ -55,8 +57,8 @@ public class MailContentTransformerTest extends AbstractContentTransformerTest public void testIsTransformable() throws Exception { - assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, MimetypeMap.MIMETYPE_OUTLOOK_MSG, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_OUTLOOK_MSG, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); + assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, -1, MimetypeMap.MIMETYPE_OUTLOOK_MSG, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_OUTLOOK_MSG, -1, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); } /** diff --git a/source/java/org/alfresco/repo/content/transform/MediaWikiContentTransformerTest.java b/source/java/org/alfresco/repo/content/transform/MediaWikiContentTransformerTest.java index 88ec12dace..b69b281d09 100644 --- a/source/java/org/alfresco/repo/content/transform/MediaWikiContentTransformerTest.java +++ b/source/java/org/alfresco/repo/content/transform/MediaWikiContentTransformerTest.java @@ -1,5 +1,5 @@ /* - * Copyright (C) 2005-2010 Alfresco Software Limited. + * Copyright (C) 2005-2011 Alfresco Software Limited. * * This file is part of Alfresco * @@ -39,7 +39,7 @@ import de.schlichtherle.io.FileOutputStream; */ public class MediaWikiContentTransformerTest extends AbstractContentTransformerTest { - private ContentTransformer transformer; + private MediaWikiContentTransformer transformer; private static final String WIKI_TEXT = "== This is a title ==\n" + @@ -69,6 +69,8 @@ public class MediaWikiContentTransformerTest extends AbstractContentTransformerT { super.setUp(); transformer = new MediaWikiContentTransformer(); + transformer.setMimetypeService(mimetypeService); + transformer.setTransformerDebug(transformerDebug); } protected ContentTransformer getTransformer(String sourceMimetype, String targetMimetype) @@ -84,8 +86,8 @@ public class MediaWikiContentTransformerTest extends AbstractContentTransformerT public void testIsTransformable() throws Exception { // check reliability - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_MEDIAWIKI, MimetypeMap.MIMETYPE_HTML, new TransformationOptions())); - assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_HTML, MimetypeMap.MIMETYPE_TEXT_MEDIAWIKI, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_MEDIAWIKI, -1, MimetypeMap.MIMETYPE_HTML, new TransformationOptions())); + assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_HTML, -1, MimetypeMap.MIMETYPE_TEXT_MEDIAWIKI, new TransformationOptions())); } public void testMediaWikiToHTML() throws Exception diff --git a/source/java/org/alfresco/repo/content/transform/OpenOfficeContentTransformerTest.java b/source/java/org/alfresco/repo/content/transform/OpenOfficeContentTransformerTest.java index 29269a7efd..d3728cb03c 100644 --- a/source/java/org/alfresco/repo/content/transform/OpenOfficeContentTransformerTest.java +++ b/source/java/org/alfresco/repo/content/transform/OpenOfficeContentTransformerTest.java @@ -1,5 +1,5 @@ /* - * Copyright (C) 2005-2010 Alfresco Software Limited. + * Copyright (C) 2005-2011 Alfresco Software Limited. * * This file is part of Alfresco * @@ -39,7 +39,7 @@ public class OpenOfficeContentTransformerTest extends AbstractContentTransformer private static String MIMETYPE_RUBBISH = "text/rubbish"; private ContentTransformerWorker worker; - private ContentTransformer transformer; + private ProxyContentTransformer transformer; @Override public void setUp() throws Exception @@ -47,10 +47,10 @@ public class OpenOfficeContentTransformerTest extends AbstractContentTransformer super.setUp(); this.worker = (ContentTransformerWorker) ctx.getBean("transformer.worker.OpenOffice"); - ProxyContentTransformer transformer = new ProxyContentTransformer(); + transformer = new ProxyContentTransformer(); transformer.setMimetypeService(mimetypeService); + transformer.setTransformerDebug(transformerDebug); transformer.setWorker(this.worker); - this.transformer = transformer; } /** @@ -74,15 +74,15 @@ public class OpenOfficeContentTransformerTest extends AbstractContentTransformer // no connection return; } - boolean reliability = transformer.isTransformable(MIMETYPE_RUBBISH, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions()); + boolean reliability = transformer.isTransformable(MIMETYPE_RUBBISH, -1, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions()); assertEquals("Mimetype should not be supported", false, reliability); - reliability = transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, MIMETYPE_RUBBISH, new TransformationOptions()); + reliability = transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, -1, MIMETYPE_RUBBISH, new TransformationOptions()); assertEquals("Mimetype should not be supported", false, reliability); - reliability = transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, MimetypeMap.MIMETYPE_XHTML, new TransformationOptions()); + reliability = transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, -1, MimetypeMap.MIMETYPE_XHTML, new TransformationOptions()); assertEquals("Mimetype should not be supported", false, reliability); - reliability = transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, MimetypeMap.MIMETYPE_WORD, new TransformationOptions()); + reliability = transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, -1, MimetypeMap.MIMETYPE_WORD, new TransformationOptions()); assertEquals("Mimetype should be supported", true, reliability); - reliability = transformer.isTransformable(MimetypeMap.MIMETYPE_WORD, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions()); + reliability = transformer.isTransformable(MimetypeMap.MIMETYPE_WORD, -1, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions()); assertEquals("Mimetype should be supported", true, reliability); } diff --git a/source/java/org/alfresco/repo/content/transform/PdfBoxContentTransformerTest.java b/source/java/org/alfresco/repo/content/transform/PdfBoxContentTransformerTest.java index bcceb505de..d83eae72b3 100644 --- a/source/java/org/alfresco/repo/content/transform/PdfBoxContentTransformerTest.java +++ b/source/java/org/alfresco/repo/content/transform/PdfBoxContentTransformerTest.java @@ -1,5 +1,5 @@ /* - * Copyright (C) 2005-2010 Alfresco Software Limited. + * Copyright (C) 2005-2011 Alfresco Software Limited. * * This file is part of Alfresco * @@ -28,7 +28,7 @@ import org.alfresco.service.cmr.repository.TransformationOptions; */ public class PdfBoxContentTransformerTest extends AbstractContentTransformerTest { - private ContentTransformer transformer; + private PdfBoxContentTransformer transformer; @Override public void setUp() throws Exception @@ -36,6 +36,8 @@ public class PdfBoxContentTransformerTest extends AbstractContentTransformerTest super.setUp(); transformer = new PdfBoxContentTransformer(); + transformer.setMimetypeService(mimetypeService); + transformer.setTransformerDebug(transformerDebug); } /** @@ -48,9 +50,9 @@ public class PdfBoxContentTransformerTest extends AbstractContentTransformerTest public void testIsTransformable() throws Exception { - assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, MimetypeMap.MIMETYPE_PDF, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_PDF, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_PDF, MimetypeMap.MIMETYPE_HTML, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_PDF, MimetypeMap.MIMETYPE_XML, new TransformationOptions())); + assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, -1, MimetypeMap.MIMETYPE_PDF, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_PDF, -1, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_PDF, -1, MimetypeMap.MIMETYPE_HTML, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_PDF, -1, MimetypeMap.MIMETYPE_XML, new TransformationOptions())); } } diff --git a/source/java/org/alfresco/repo/content/transform/PdfBoxPdfToImageContentTransformerTest.java b/source/java/org/alfresco/repo/content/transform/PdfBoxPdfToImageContentTransformerTest.java index 2cb042393a..ea3f0c0a32 100644 --- a/source/java/org/alfresco/repo/content/transform/PdfBoxPdfToImageContentTransformerTest.java +++ b/source/java/org/alfresco/repo/content/transform/PdfBoxPdfToImageContentTransformerTest.java @@ -36,7 +36,7 @@ import org.alfresco.util.TempFileProvider; */ public class PdfBoxPdfToImageContentTransformerTest extends AbstractContentTransformerTest { - private ContentTransformer transformer; + private PdfBoxPdfToImageContentTransformer transformer; @Override public void setUp() throws Exception @@ -44,7 +44,8 @@ public class PdfBoxPdfToImageContentTransformerTest extends AbstractContentTrans super.setUp(); transformer = new PdfBoxPdfToImageContentTransformer(); - ((ContentTransformerHelper)transformer).setMimetypeService(mimetypeService); + transformer.setMimetypeService(mimetypeService); + transformer.setTransformerDebug(transformerDebug); } /** @@ -57,7 +58,7 @@ public class PdfBoxPdfToImageContentTransformerTest extends AbstractContentTrans public void testIsTransformable() throws Exception { - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_PDF, MimetypeMap.MIMETYPE_IMAGE_PNG, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_PDF, -1, MimetypeMap.MIMETYPE_IMAGE_PNG, new TransformationOptions())); } /** diff --git a/source/java/org/alfresco/repo/content/transform/PoiContentTransformerTest.java b/source/java/org/alfresco/repo/content/transform/PoiContentTransformerTest.java index 28e49c261a..06386995c6 100644 --- a/source/java/org/alfresco/repo/content/transform/PoiContentTransformerTest.java +++ b/source/java/org/alfresco/repo/content/transform/PoiContentTransformerTest.java @@ -1,5 +1,5 @@ /* - * Copyright (C) 2005-2010 Alfresco Software Limited. + * Copyright (C) 2005-2011 Alfresco Software Limited. * * This file is part of Alfresco * @@ -28,7 +28,7 @@ import org.alfresco.service.cmr.repository.TransformationOptions; */ public class PoiContentTransformerTest extends AbstractContentTransformerTest { - private ContentTransformer transformer; + private PoiContentTransformer transformer; @Override public void setUp() throws Exception @@ -36,6 +36,8 @@ public class PoiContentTransformerTest extends AbstractContentTransformerTest super.setUp(); transformer = new PoiContentTransformer(); + transformer.setMimetypeService(mimetypeService); + transformer.setTransformerDebug(transformerDebug); } /** @@ -48,22 +50,22 @@ public class PoiContentTransformerTest extends AbstractContentTransformerTest public void testIsTransformable() throws Exception { - assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, MimetypeMap.MIMETYPE_WORD, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_WORD, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_WORD, MimetypeMap.MIMETYPE_HTML, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_WORD, MimetypeMap.MIMETYPE_XML, new TransformationOptions())); + assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, -1, MimetypeMap.MIMETYPE_WORD, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_WORD, -1, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_WORD, -1, MimetypeMap.MIMETYPE_HTML, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_WORD, -1, MimetypeMap.MIMETYPE_XML, new TransformationOptions())); - assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, MimetypeMap.MIMETYPE_PPT, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_PPT, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_PPT, MimetypeMap.MIMETYPE_HTML, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_PPT, MimetypeMap.MIMETYPE_XML, new TransformationOptions())); + assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, -1, MimetypeMap.MIMETYPE_PPT, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_PPT, -1, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_PPT, -1, MimetypeMap.MIMETYPE_HTML, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_PPT, -1, MimetypeMap.MIMETYPE_XML, new TransformationOptions())); - assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, MimetypeMap.MIMETYPE_OUTLOOK_MSG, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_OUTLOOK_MSG, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_OUTLOOK_MSG, MimetypeMap.MIMETYPE_HTML, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_OUTLOOK_MSG, MimetypeMap.MIMETYPE_XML, new TransformationOptions())); + assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, -1, MimetypeMap.MIMETYPE_OUTLOOK_MSG, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_OUTLOOK_MSG, -1, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_OUTLOOK_MSG, -1, MimetypeMap.MIMETYPE_HTML, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_OUTLOOK_MSG, -1, MimetypeMap.MIMETYPE_XML, new TransformationOptions())); // Doesn't claim excel - assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_EXCEL, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); + assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_EXCEL, -1, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); } } diff --git a/source/java/org/alfresco/repo/content/transform/PoiHssfContentTransformer.java b/source/java/org/alfresco/repo/content/transform/PoiHssfContentTransformer.java index 02f6405aad..8963fd5343 100644 --- a/source/java/org/alfresco/repo/content/transform/PoiHssfContentTransformer.java +++ b/source/java/org/alfresco/repo/content/transform/PoiHssfContentTransformer.java @@ -72,19 +72,30 @@ public class PoiHssfContentTransformer extends TikaPoweredContentTransformer * We support transforming to HTML, XML, Text or CSV */ @Override - public boolean isTransformable(String sourceMimetype, String targetMimetype, TransformationOptions options) + public boolean isTransformable(String sourceMimetype, long sourceSize, String targetMimetype, TransformationOptions options) { if(sourceMimeTypes.contains(sourceMimetype) && MimetypeMap.MIMETYPE_TEXT_CSV.equals(targetMimetype)) { - // Special case for CSV - return true; + // Special case for CSV + return isTransformableSize(sourceMimetype, sourceSize, targetMimetype, options); } // Otherwise fall back on the default Tika rules - return super.isTransformable(sourceMimetype, targetMimetype, options); + return super.isTransformable(sourceMimetype, sourceSize, targetMimetype, options); } + /** + * @deprecated This method should no longer be called as the overloaded method + * that calls it has been overridden. + */ + @Override + public boolean isTransformable(String sourceMimetype, String targetMimetype, + TransformationOptions options) + { + return isTransformable(sourceMimetype, -1, targetMimetype, options); + } + /** * Make sure we win over openoffice when it comes to producing * HTML diff --git a/source/java/org/alfresco/repo/content/transform/PoiHssfContentTransformerTest.java b/source/java/org/alfresco/repo/content/transform/PoiHssfContentTransformerTest.java index 29ab1d8122..1a8e1ccbfc 100644 --- a/source/java/org/alfresco/repo/content/transform/PoiHssfContentTransformerTest.java +++ b/source/java/org/alfresco/repo/content/transform/PoiHssfContentTransformerTest.java @@ -1,5 +1,5 @@ /* - * Copyright (C) 2005-2010 Alfresco Software Limited. + * Copyright (C) 2005-2011 Alfresco Software Limited. * * This file is part of Alfresco * @@ -36,7 +36,7 @@ import org.alfresco.util.TempFileProvider; */ public class PoiHssfContentTransformerTest extends TikaPoweredContentTransformerTest { - private ContentTransformer transformer; + private PoiHssfContentTransformer transformer; @Override public void setUp() throws Exception @@ -44,6 +44,8 @@ public class PoiHssfContentTransformerTest extends TikaPoweredContentTransformer super.setUp(); transformer = new PoiHssfContentTransformer(); + transformer.setMimetypeService(mimetypeService); + transformer.setTransformerDebug(transformerDebug); } @Override @@ -63,11 +65,11 @@ public class PoiHssfContentTransformerTest extends TikaPoweredContentTransformer public void testIsTransformable() throws Exception { - assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, MimetypeMap.MIMETYPE_EXCEL, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_EXCEL, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_EXCEL, MimetypeMap.MIMETYPE_TEXT_CSV, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_EXCEL, MimetypeMap.MIMETYPE_HTML, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_EXCEL, MimetypeMap.MIMETYPE_XML, new TransformationOptions())); + assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, -1, MimetypeMap.MIMETYPE_EXCEL, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_EXCEL, -1, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_EXCEL, -1, MimetypeMap.MIMETYPE_TEXT_CSV, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_EXCEL, -1, MimetypeMap.MIMETYPE_HTML, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_EXCEL, -1, MimetypeMap.MIMETYPE_XML, new TransformationOptions())); } public void testCsvOutput() throws Exception diff --git a/source/java/org/alfresco/repo/content/transform/PoiOOXMLContentTransformerTest.java b/source/java/org/alfresco/repo/content/transform/PoiOOXMLContentTransformerTest.java index aea7de61d2..6f29a9a974 100644 --- a/source/java/org/alfresco/repo/content/transform/PoiOOXMLContentTransformerTest.java +++ b/source/java/org/alfresco/repo/content/transform/PoiOOXMLContentTransformerTest.java @@ -1,5 +1,5 @@ /* - * Copyright (C) 2005-2010 Alfresco Software Limited. + * Copyright (C) 2005-2011 Alfresco Software Limited. * * This file is part of Alfresco * @@ -28,7 +28,7 @@ import org.alfresco.service.cmr.repository.TransformationOptions; */ public class PoiOOXMLContentTransformerTest extends AbstractContentTransformerTest { - private ContentTransformer transformer; + private PoiOOXMLContentTransformer transformer; @Override public void setUp() throws Exception @@ -36,6 +36,8 @@ public class PoiOOXMLContentTransformerTest extends AbstractContentTransformerTe super.setUp(); transformer = new PoiOOXMLContentTransformer(); + transformer.setMimetypeService(mimetypeService); + transformer.setTransformerDebug(transformerDebug); } /** @@ -48,19 +50,19 @@ public class PoiOOXMLContentTransformerTest extends AbstractContentTransformerTe public void testIsTransformable() throws Exception { - assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, MimetypeMap.MIMETYPE_OPENXML_WORDPROCESSING, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_OPENXML_WORDPROCESSING, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_OPENXML_WORDPROCESSING, MimetypeMap.MIMETYPE_HTML, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_OPENXML_WORDPROCESSING, MimetypeMap.MIMETYPE_XML, new TransformationOptions())); + assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, -1, MimetypeMap.MIMETYPE_OPENXML_WORDPROCESSING, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_OPENXML_WORDPROCESSING, -1, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_OPENXML_WORDPROCESSING, -1, MimetypeMap.MIMETYPE_HTML, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_OPENXML_WORDPROCESSING, -1, MimetypeMap.MIMETYPE_XML, new TransformationOptions())); - assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, MimetypeMap.MIMETYPE_OPENXML_PRESENTATION, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_OPENXML_PRESENTATION, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_OPENXML_PRESENTATION, MimetypeMap.MIMETYPE_HTML, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_OPENXML_PRESENTATION, MimetypeMap.MIMETYPE_XML, new TransformationOptions())); + assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, -1, MimetypeMap.MIMETYPE_OPENXML_PRESENTATION, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_OPENXML_PRESENTATION, -1, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_OPENXML_PRESENTATION, -1, MimetypeMap.MIMETYPE_HTML, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_OPENXML_PRESENTATION, -1, MimetypeMap.MIMETYPE_XML, new TransformationOptions())); - assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, MimetypeMap.MIMETYPE_OPENXML_SPREADSHEET, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_OPENXML_SPREADSHEET, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_OPENXML_SPREADSHEET, MimetypeMap.MIMETYPE_HTML, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_OPENXML_SPREADSHEET, MimetypeMap.MIMETYPE_XML, new TransformationOptions())); + assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, -1, MimetypeMap.MIMETYPE_OPENXML_SPREADSHEET, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_OPENXML_SPREADSHEET, -1, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_OPENXML_SPREADSHEET, -1, MimetypeMap.MIMETYPE_HTML, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_OPENXML_SPREADSHEET, -1, MimetypeMap.MIMETYPE_XML, new TransformationOptions())); } } diff --git a/source/java/org/alfresco/repo/content/transform/ProxyContentTransformer.java b/source/java/org/alfresco/repo/content/transform/ProxyContentTransformer.java index 79a79159c4..071a8a6dc6 100644 --- a/source/java/org/alfresco/repo/content/transform/ProxyContentTransformer.java +++ b/source/java/org/alfresco/repo/content/transform/ProxyContentTransformer.java @@ -22,6 +22,7 @@ import net.sf.jooreports.converter.DocumentFormatRegistry; import org.alfresco.service.cmr.repository.ContentReader; import org.alfresco.service.cmr.repository.ContentWriter; +import org.alfresco.service.cmr.repository.TransformationOptionLimits; import org.alfresco.service.cmr.repository.TransformationOptions; /** @@ -65,6 +66,18 @@ public class ProxyContentTransformer extends AbstractContentTransformer2 protected void transformInternal(ContentReader reader, ContentWriter writer, TransformationOptions options) throws Exception { - this.worker.transform(reader, writer, options); - } + TransformationOptionLimits original = options.getLimits(); + try + { + // Combine the transformer's limit values into the options so they are available to the worker + options.setLimits(getLimits(reader, writer, options)); + + // Perform the transformation + this.worker.transform(reader, writer, options); + } + finally + { + options.setLimits(original); + } + } } diff --git a/source/java/org/alfresco/repo/content/transform/RuntimeExecutableContentTransformerTest.java b/source/java/org/alfresco/repo/content/transform/RuntimeExecutableContentTransformerTest.java index 8fc5e7ad75..40fdbb8a62 100644 --- a/source/java/org/alfresco/repo/content/transform/RuntimeExecutableContentTransformerTest.java +++ b/source/java/org/alfresco/repo/content/transform/RuntimeExecutableContentTransformerTest.java @@ -73,8 +73,11 @@ public class RuntimeExecutableContentTransformerTest extends BaseAlfrescoTestCas // initialise so that it doesn't score 0 worker.afterPropertiesSet(); + TransformerDebug transformerDebug = (TransformerDebug) ctx.getBean("transformerDebug"); + ProxyContentTransformer transformer = new ProxyContentTransformer(); transformer.setMimetypeService(serviceRegistry.getMimetypeService()); + transformer.setTransformerDebug(transformerDebug); transformer.setWorker(worker); this.transformer = transformer; } diff --git a/source/java/org/alfresco/repo/content/transform/RuntimeExecutableContentTransformerWorker.java b/source/java/org/alfresco/repo/content/transform/RuntimeExecutableContentTransformerWorker.java index 7198b72c9f..9537fed4b6 100644 --- a/source/java/org/alfresco/repo/content/transform/RuntimeExecutableContentTransformerWorker.java +++ b/source/java/org/alfresco/repo/content/transform/RuntimeExecutableContentTransformerWorker.java @@ -256,10 +256,11 @@ public class RuntimeExecutableContentTransformerWorker extends ContentTransforme reader.getContent(sourceFile); // execute the transformation command + long timeoutMs = options.getTimeoutMs(); ExecutionResult result = null; try { - result = transformCommand.execute(properties); + result = transformCommand.execute(properties, timeoutMs); } catch (Throwable e) { diff --git a/source/java/org/alfresco/repo/content/transform/StringExtractingContentTransformer.java b/source/java/org/alfresco/repo/content/transform/StringExtractingContentTransformer.java index 2f95e45f44..742c36d21e 100644 --- a/source/java/org/alfresco/repo/content/transform/StringExtractingContentTransformer.java +++ b/source/java/org/alfresco/repo/content/transform/StringExtractingContentTransformer.java @@ -118,7 +118,7 @@ public class StringExtractingContentTransformer extends AbstractContentTransform charWriter = new OutputStreamWriter(writer.getContentOutputStream(), writer.getEncoding()); } // copy from the one to the other - char[] buffer = new char[1024]; + char[] buffer = new char[8192]; int readCount = 0; while (readCount > -1) { diff --git a/source/java/org/alfresco/repo/content/transform/StringExtractingContentTransformerTest.java b/source/java/org/alfresco/repo/content/transform/StringExtractingContentTransformerTest.java index 17c3551b3a..d00b5f0932 100644 --- a/source/java/org/alfresco/repo/content/transform/StringExtractingContentTransformerTest.java +++ b/source/java/org/alfresco/repo/content/transform/StringExtractingContentTransformerTest.java @@ -56,7 +56,7 @@ public class StringExtractingContentTransformerTest extends AbstractContentTrans } } - private ContentTransformer transformer; + private StringExtractingContentTransformer transformer; /** the final destination of transformations */ private ContentWriter targetWriter; @@ -66,6 +66,8 @@ public class StringExtractingContentTransformerTest extends AbstractContentTrans super.setUp(); transformer = new StringExtractingContentTransformer(); + transformer.setMimetypeService(mimetypeService); + transformer.setTransformerDebug(transformerDebug); targetWriter = new FileContentWriter(getTempFile()); targetWriter.setMimetype("text/plain"); targetWriter.setEncoding("UTF-8"); @@ -112,7 +114,7 @@ public class StringExtractingContentTransformerTest extends AbstractContentTrans ContentReader reader = writeContent("text/plain", "MacDingbat"); // check transformability - assertTrue(transformer.isTransformable(reader.getMimetype(), targetWriter.getMimetype(), new TransformationOptions())); + assertTrue(transformer.isTransformable(reader.getMimetype(), -1, targetWriter.getMimetype(), new TransformationOptions())); // transform transformer.transform(reader, targetWriter); @@ -128,7 +130,7 @@ public class StringExtractingContentTransformerTest extends AbstractContentTrans ContentReader reader = writeContent("text/xml", "MacDingbat"); // check transformability - assertTrue(transformer.isTransformable(reader.getMimetype(), targetWriter.getMimetype(), new TransformationOptions())); + assertTrue(transformer.isTransformable(reader.getMimetype(), -1, targetWriter.getMimetype(), new TransformationOptions())); // transform transformer.transform(reader, targetWriter); diff --git a/source/java/org/alfresco/repo/content/transform/TextMiningContentTransformerTest.java b/source/java/org/alfresco/repo/content/transform/TextMiningContentTransformerTest.java index 5e0d677ba6..3005d57d43 100644 --- a/source/java/org/alfresco/repo/content/transform/TextMiningContentTransformerTest.java +++ b/source/java/org/alfresco/repo/content/transform/TextMiningContentTransformerTest.java @@ -1,5 +1,5 @@ /* - * Copyright (C) 2005-2010 Alfresco Software Limited. + * Copyright (C) 2005-2011 Alfresco Software Limited. * * This file is part of Alfresco * @@ -36,7 +36,7 @@ import org.alfresco.util.TempFileProvider; */ public class TextMiningContentTransformerTest extends AbstractContentTransformerTest { - private ContentTransformer transformer; + private TextMiningContentTransformer transformer; @Override public void setUp() throws Exception @@ -44,6 +44,8 @@ public class TextMiningContentTransformerTest extends AbstractContentTransformer super.setUp(); transformer = new TextMiningContentTransformer(); + transformer.setMimetypeService(mimetypeService); + transformer.setTransformerDebug(transformerDebug); } /** @@ -63,8 +65,8 @@ public class TextMiningContentTransformerTest extends AbstractContentTransformer public void testIsTransformable() throws Exception { - assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, MimetypeMap.MIMETYPE_WORD, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_WORD, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); + assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, -1, MimetypeMap.MIMETYPE_WORD, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_WORD, -1, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); } /** diff --git a/source/java/org/alfresco/repo/content/transform/TextToPdfContentTransformer.java b/source/java/org/alfresco/repo/content/transform/TextToPdfContentTransformer.java index 01da29f134..8c3269bdbc 100644 --- a/source/java/org/alfresco/repo/content/transform/TextToPdfContentTransformer.java +++ b/source/java/org/alfresco/repo/content/transform/TextToPdfContentTransformer.java @@ -1,5 +1,5 @@ /* - * Copyright (C) 2005-2010 Alfresco Software Limited. + * Copyright (C) 2005-2011 Alfresco Software Limited. * * This file is part of Alfresco * @@ -18,24 +18,31 @@ */ package org.alfresco.repo.content.transform; +import java.io.BufferedReader; +import java.io.IOException; import java.io.InputStream; import java.io.InputStreamReader; import java.io.OutputStream; +import java.io.Reader; import java.nio.charset.Charset; +import java.util.Map; import org.alfresco.error.AlfrescoRuntimeException; import org.alfresco.repo.content.MimetypeMap; import org.alfresco.service.cmr.repository.ContentReader; import org.alfresco.service.cmr.repository.ContentWriter; +import org.alfresco.service.cmr.repository.TransformationOptionLimits; +import org.alfresco.service.cmr.repository.TransformationOptionPair; +import org.alfresco.service.cmr.repository.TransformationOptionPair.Action; import org.alfresco.service.cmr.repository.TransformationOptions; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import org.apache.pdfbox.TextToPDF; import org.apache.pdfbox.pdmodel.PDDocument; +import org.apache.pdfbox.pdmodel.PDPage; +import org.apache.pdfbox.pdmodel.edit.PDPageContentStream; import org.apache.pdfbox.pdmodel.font.PDTrueTypeFont; import org.apache.pdfbox.pdmodel.font.PDType1Font; -import org.apache.tika.io.IOUtils; - /** * Makes use of the {@link http://www.pdfbox.org/ PDFBox} library's TextToPDF utility. * @@ -46,11 +53,12 @@ public class TextToPdfContentTransformer extends AbstractContentTransformer2 { private static final Log logger = LogFactory.getLog(TextToPdfContentTransformer.class); - private TextToPDF transformer; + private PagedTextToPDF transformer; public TextToPdfContentTransformer() { - transformer = new TextToPDF(); + setPageLimitsSuported(true); + transformer = new PagedTextToPDF(); } public void setStandardFont(String fontName) @@ -92,7 +100,7 @@ public class TextToPdfContentTransformer extends AbstractContentTransformer2 /** * Only supports Text to PDF */ - public boolean isTransformable(String sourceMimetype, String targetMimetype, TransformationOptions options) + public boolean isTransformable(String sourceMimetype, long sourceSize, String targetMimetype, TransformationOptions options) { if ( (!MimetypeMap.MIMETYPE_TEXT_PLAIN.equals(sourceMimetype) && !MimetypeMap.MIMETYPE_TEXT_CSV.equals(sourceMimetype) && @@ -104,10 +112,21 @@ public class TextToPdfContentTransformer extends AbstractContentTransformer2 } else { - return true; + return isTransformableSize(sourceMimetype, sourceSize, targetMimetype, options); } } + /** + * @deprecated This method should no longer be called as the overloaded method + * that calls it has the overridden. + */ + @Override + public boolean isTransformable(String sourceMimetype, String targetMimetype, + TransformationOptions options) + { + return false; + } + @Override protected void transformInternal( ContentReader reader, @@ -122,8 +141,11 @@ public class TextToPdfContentTransformer extends AbstractContentTransformer2 { is = reader.getContentInputStream(); ir = buildReader(is, reader.getEncoding(), reader.getContentUrl()); - - pdf = transformer.createPDFFromText(ir); + + TransformationOptionLimits limits = getLimits(reader, writer, options); + TransformationOptionPair pageLimits = limits.getPagesPair(); + pdf = transformer.createPDFFromText(ir, pageLimits, reader.getContentUrl()); + // dump it all to the writer os = writer.getContentOutputStream(); pdf.save(os); @@ -174,4 +196,124 @@ public class TextToPdfContentTransformer extends AbstractContentTransformer2 logger.debug("Processing plain text using system default encoding"); return new InputStreamReader(is); } + + private class PagedTextToPDF extends TextToPDF + { + // The following code is based on the code in TextToPDF with the addition of + // checks for page limits + public PDDocument createPDFFromText(Reader text, TransformationOptionPair pageLimits, + String contentUrl) throws IOException + { + int pageLimit = (int)pageLimits.getValue(); + PDDocument doc = null; + int pageCount = 0; + try + { + final int margin = 40; + float height = getFont().getFontDescriptor().getFontBoundingBox().getHeight()/1000; + + //calculate font height and increase by 5 percent. + height = height*getFontSize()*1.05f; + doc = new PDDocument(); + BufferedReader data = new BufferedReader( text ); + String nextLine = null; + PDPage page = new PDPage(); + PDPageContentStream contentStream = null; + float y = -1; + float maxStringLength = page.getMediaBox().getWidth() - 2*margin; + + // There is a special case of creating a PDF document from an empty string. + boolean textIsEmpty = true; + + outer: + while( (nextLine = data.readLine()) != null ) + { + + // The input text is nonEmpty. New pages will be created and added + // to the PDF document as they are needed, depending on the length of + // the text. + textIsEmpty = false; + + String[] lineWords = nextLine.trim().split( " " ); + int lineIndex = 0; + while( lineIndex < lineWords.length ) + { + StringBuffer nextLineToDraw = new StringBuffer(); + float lengthIfUsingNextWord = 0; + do + { + nextLineToDraw.append( lineWords[lineIndex] ); + nextLineToDraw.append( " " ); + lineIndex++; + if( lineIndex < lineWords.length ) + { + String lineWithNextWord = nextLineToDraw.toString() + lineWords[lineIndex]; + lengthIfUsingNextWord = + (getFont().getStringWidth( lineWithNextWord )/1000) * getFontSize(); + } + } + while( lineIndex < lineWords.length && + lengthIfUsingNextWord < maxStringLength ); + if( y < margin ) + { + if (pageLimit > 0 && pageCount++ >= pageLimit) + { + pageLimits.getAction().throwIOExceptionIfRequired("Page limit ("+pageLimit+ + ") reached.", transformerDebug); + break outer; + } + + // We have crossed the end-of-page boundary and need to extend the + // document by another page. + page = new PDPage(); + doc.addPage( page ); + if( contentStream != null ) + { + contentStream.endText(); + contentStream.close(); + } + contentStream = new PDPageContentStream(doc, page); + contentStream.setFont(getFont(), getFontSize()); + contentStream.beginText(); + y = page.getMediaBox().getHeight() - margin + height; + contentStream.moveTextPositionByAmount( + margin, y ); + } + //System.out.println( "Drawing string at " + x + "," + y ); + + if( contentStream == null ) + { + throw new IOException( "Error:Expected non-null content stream." ); + } + contentStream.moveTextPositionByAmount( 0, -height); + y -= height; + contentStream.drawString( nextLineToDraw.toString() ); + } + } + + // If the input text was the empty string, then the above while loop will have short-circuited + // and we will not have added any PDPages to the document. + // So in order to make the resultant PDF document readable by Adobe Reader etc, we'll add an empty page. + if (textIsEmpty) + { + doc.addPage(page); + } + + if( contentStream != null ) + { + contentStream.endText(); + contentStream.close(); + } + } + catch( IOException io ) + { + if( doc != null ) + { + doc.close(); + } + throw io; + } + return doc; + } + } } diff --git a/source/java/org/alfresco/repo/content/transform/TextToPdfContentTransformerTest.java b/source/java/org/alfresco/repo/content/transform/TextToPdfContentTransformerTest.java index 5992bf4425..dbc79e45ae 100644 --- a/source/java/org/alfresco/repo/content/transform/TextToPdfContentTransformerTest.java +++ b/source/java/org/alfresco/repo/content/transform/TextToPdfContentTransformerTest.java @@ -1,5 +1,5 @@ /* - * Copyright (C) 2005-2010 Alfresco Software Limited. + * Copyright (C) 2005-2011 Alfresco Software Limited. * * This file is part of Alfresco * @@ -19,6 +19,7 @@ package org.alfresco.repo.content.transform; import java.io.File; +import java.io.IOException; import java.io.StringWriter; import java.nio.charset.Charset; @@ -47,8 +48,11 @@ public class TextToPdfContentTransformerTest extends AbstractContentTransformerT super.setUp(); transformer = new TextToPdfContentTransformer(); + transformer.setMimetypeService(mimetypeService); + transformer.setTransformerDebug(transformerDebug); transformer.setStandardFont("Times-Roman"); transformer.setFontSize(20); + transformer.setPageLimit(-1); } /** @@ -62,11 +66,11 @@ public class TextToPdfContentTransformerTest extends AbstractContentTransformerT public void testReliability() throws Exception { - boolean reliability = transformer.isTransformable(MimetypeMap.MIMETYPE_PDF, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions()); + boolean reliability = transformer.isTransformable(MimetypeMap.MIMETYPE_PDF, -1, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions()); assertEquals("Mimetype should not be supported", false, reliability); - reliability = transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, MimetypeMap.MIMETYPE_PDF, new TransformationOptions()); + reliability = transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, -1, MimetypeMap.MIMETYPE_PDF, new TransformationOptions()); assertEquals("Mimetype should be supported", true, reliability); - reliability = transformer.isTransformable(MimetypeMap.MIMETYPE_XML, MimetypeMap.MIMETYPE_PDF, new TransformationOptions()); + reliability = transformer.isTransformable(MimetypeMap.MIMETYPE_XML, -1, MimetypeMap.MIMETYPE_PDF, new TransformationOptions()); assertEquals("Mimetype should be supported", true, reliability); } @@ -82,44 +86,90 @@ public class TextToPdfContentTransformerTest extends AbstractContentTransformerT String european = "En français où les choses sont accentués\n" + "En español, así"; - ContentReader reader; for(String text : new String[] {allAscii, european}) { for(String encoding : new String[] {"ISO-8859-1", "UTF-8", "UTF-16"}) { - // Get a reader for the text - reader = buildContentReader(text, Charset.forName(encoding)); - - // And a temp writer - File out = TempFileProvider.createTempFile("AlfrescoTest_", ".pdf"); - ContentWriter writer = new FileContentWriter(out); - writer.setMimetype("application/pdf"); - - // Transform to PDF - transformer.transform(reader, writer); - - // Read back in the PDF and check it - PDDocument doc = PDDocument.load(out); - PDFTextStripper textStripper = new PDFTextStripper(); - StringWriter textWriter = new StringWriter(); - textStripper.writeText(doc, textWriter); - doc.close(); - // Newlines etc may be different, so zap them String checkText = clean(text); - String roundTrip = clean(textWriter.toString()); - - // Now check it -// System.err.println("== " + encoding + " =="); -// System.err.println(roundTrip); -// System.err.println("===="); - assertEquals( - "Incorrect text in PDF when starting from text in " + encoding, - checkText, roundTrip - ); + + transformTextAndCheck(text, encoding, checkText); } } } + + public void testUnlimitedPages() throws Exception + { + transformTextAndCheckPageLength(-1); + } + + public void testLimitedTo1Page() throws Exception + { + transformTextAndCheckPageLength(1); + } + + public void testLimitedTo2Pages() throws Exception + { + transformTextAndCheckPageLength(2); + } + + public void testLimitedTo50Pages() throws Exception + { + transformTextAndCheckPageLength(50); + } + + private void transformTextAndCheckPageLength(int pageLimit) throws IOException + { + transformer.setPageLimit(pageLimit); + + int pageLength = 32; + int lines = (pageLength+10) * ((pageLimit > 0) ? pageLimit : 1); + StringBuilder sb = new StringBuilder(); + String checkText = null; + int cutoff = pageLimit * pageLength; + for (int i=1; i<=lines; i++) + { + sb.append(i); + sb.append(" I must not talk in class or feed my homework to my cat.\n"); + if (i == cutoff) + checkText = sb.toString(); + } + sb.append("\nBart\n"); + String text = sb.toString(); + checkText = (checkText == null) ? clean(text) : clean(checkText); + + transformTextAndCheck(text, "UTF-8", checkText); + } + + private void transformTextAndCheck(String text, String encoding, String checkText) + throws IOException + { + // Get a reader for the text + ContentReader reader = buildContentReader(text, Charset.forName(encoding)); + + // And a temp writer + File out = TempFileProvider.createTempFile("AlfrescoTest_", ".pdf"); + ContentWriter writer = new FileContentWriter(out); + writer.setMimetype("application/pdf"); + + // Transform to PDF + transformer.transform(reader, writer); + + // Read back in the PDF and check it + PDDocument doc = PDDocument.load(out); + PDFTextStripper textStripper = new PDFTextStripper(); + StringWriter textWriter = new StringWriter(); + textStripper.writeText(doc, textWriter); + doc.close(); + + String roundTrip = clean(textWriter.toString()); + + assertEquals( + "Incorrect text in PDF when starting from text in " + encoding, + checkText, roundTrip + ); + } + private String clean(String text) { text = text.replaceAll("\\s+\\r", ""); diff --git a/source/java/org/alfresco/repo/content/transform/TikaAutoContentTransformerTest.java b/source/java/org/alfresco/repo/content/transform/TikaAutoContentTransformerTest.java index 076c6bb6e0..bb31e33e31 100644 --- a/source/java/org/alfresco/repo/content/transform/TikaAutoContentTransformerTest.java +++ b/source/java/org/alfresco/repo/content/transform/TikaAutoContentTransformerTest.java @@ -1,5 +1,5 @@ /* - * Copyright (C) 2005-2010 Alfresco Software Limited. + * Copyright (C) 2005-2011 Alfresco Software Limited. * * This file is part of Alfresco * @@ -32,7 +32,7 @@ import org.apache.tika.config.TikaConfig; */ public class TikaAutoContentTransformerTest extends TikaPoweredContentTransformerTest { - private ContentTransformer transformer; + private TikaAutoContentTransformer transformer; @Override public void setUp() throws Exception @@ -41,6 +41,8 @@ public class TikaAutoContentTransformerTest extends TikaPoweredContentTransforme TikaConfig config = (TikaConfig)ctx.getBean("tikaConfig"); transformer = new TikaAutoContentTransformer( config ); + transformer.setMimetypeService(mimetypeService); + transformer.setTransformerDebug(transformerDebug); } /** @@ -58,36 +60,36 @@ public class TikaAutoContentTransformerTest extends TikaPoweredContentTransforme public void testIsTransformable() throws Exception { // Excel (but this isn't normally used) - assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, MimetypeMap.MIMETYPE_EXCEL, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_EXCEL, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_EXCEL, MimetypeMap.MIMETYPE_HTML, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_EXCEL, MimetypeMap.MIMETYPE_XML, new TransformationOptions())); + assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, -1, MimetypeMap.MIMETYPE_EXCEL, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_EXCEL, -1, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_EXCEL, -1, MimetypeMap.MIMETYPE_HTML, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_EXCEL, -1, MimetypeMap.MIMETYPE_XML, new TransformationOptions())); // Word - assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, MimetypeMap.MIMETYPE_WORD, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_WORD, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_WORD, MimetypeMap.MIMETYPE_HTML, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_WORD, MimetypeMap.MIMETYPE_XML, new TransformationOptions())); + assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, -1, MimetypeMap.MIMETYPE_WORD, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_WORD, -1, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_WORD, -1, MimetypeMap.MIMETYPE_HTML, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_WORD, -1, MimetypeMap.MIMETYPE_XML, new TransformationOptions())); // PDF - assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, MimetypeMap.MIMETYPE_PDF, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_PDF, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_PDF, MimetypeMap.MIMETYPE_HTML, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_PDF, MimetypeMap.MIMETYPE_XML, new TransformationOptions())); + assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, -1, MimetypeMap.MIMETYPE_PDF, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_PDF, -1, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_PDF, -1, MimetypeMap.MIMETYPE_HTML, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_PDF, -1, MimetypeMap.MIMETYPE_XML, new TransformationOptions())); // Open Office - assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, MimetypeMap.MIMETYPE_OPENDOCUMENT_PRESENTATION, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_OPENDOCUMENT_PRESENTATION, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_OPENDOCUMENT_PRESENTATION, MimetypeMap.MIMETYPE_HTML, new TransformationOptions())); - assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_OPENDOCUMENT_PRESENTATION, MimetypeMap.MIMETYPE_XML, new TransformationOptions())); + assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_TEXT_PLAIN, -1, MimetypeMap.MIMETYPE_OPENDOCUMENT_PRESENTATION, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_OPENDOCUMENT_PRESENTATION, -1, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_OPENDOCUMENT_PRESENTATION, -1, MimetypeMap.MIMETYPE_HTML, new TransformationOptions())); + assertTrue(transformer.isTransformable(MimetypeMap.MIMETYPE_OPENDOCUMENT_PRESENTATION, -1, MimetypeMap.MIMETYPE_XML, new TransformationOptions())); // We don't do images - assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_IMAGE_JPEG, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); - assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_IMAGE_JPEG, MimetypeMap.MIMETYPE_HTML, new TransformationOptions())); - assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_IMAGE_JPEG, MimetypeMap.MIMETYPE_XML, new TransformationOptions())); + assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_IMAGE_JPEG, -1, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); + assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_IMAGE_JPEG, -1, MimetypeMap.MIMETYPE_HTML, new TransformationOptions())); + assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_IMAGE_JPEG, -1, MimetypeMap.MIMETYPE_XML, new TransformationOptions())); // Ditto music - assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_MP3, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); - assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_MP3, MimetypeMap.MIMETYPE_HTML, new TransformationOptions())); - assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_MP3, MimetypeMap.MIMETYPE_XML, new TransformationOptions())); + assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_MP3, -1, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions())); + assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_MP3, -1, MimetypeMap.MIMETYPE_HTML, new TransformationOptions())); + assertFalse(transformer.isTransformable(MimetypeMap.MIMETYPE_MP3, -1, MimetypeMap.MIMETYPE_XML, new TransformationOptions())); } } diff --git a/source/java/org/alfresco/repo/content/transform/TikaPoweredContentTransformer.java b/source/java/org/alfresco/repo/content/transform/TikaPoweredContentTransformer.java index 0d848f9b30..d9d135975f 100644 --- a/source/java/org/alfresco/repo/content/transform/TikaPoweredContentTransformer.java +++ b/source/java/org/alfresco/repo/content/transform/TikaPoweredContentTransformer.java @@ -87,7 +87,7 @@ public abstract class TikaPoweredContentTransformer extends AbstractContentTrans * Can we do the requested transformation via Tika? * We support transforming to HTML, XML or Text */ - public boolean isTransformable(String sourceMimetype, String targetMimetype, TransformationOptions options) + public boolean isTransformable(String sourceMimetype, long sourceSize, String targetMimetype, TransformationOptions options) { if(! sourceMimeTypes.contains(sourceMimetype)) { @@ -101,7 +101,8 @@ public abstract class TikaPoweredContentTransformer extends AbstractContentTrans MimetypeMap.MIMETYPE_XML.equals(targetMimetype)) { // We can output to this - return true; + // But there may be size limits on this transformer. + return isTransformableSize(sourceMimetype, sourceSize, targetMimetype, options); } else { @@ -109,6 +110,17 @@ public abstract class TikaPoweredContentTransformer extends AbstractContentTrans return false; } } + + /** + * @deprecated This method should no longer be called as the overloaded method + * that calls it has been overridden. + */ + @Override + public boolean isTransformable(String sourceMimetype, String targetMimetype, + TransformationOptions options) + { + return isTransformable(sourceMimetype, -1, targetMimetype, options); + } /** * Returns an appropriate Tika ContentHandler for the diff --git a/source/java/org/alfresco/repo/content/transform/TransformerDebug.java b/source/java/org/alfresco/repo/content/transform/TransformerDebug.java new file mode 100644 index 0000000000..80b2761014 --- /dev/null +++ b/source/java/org/alfresco/repo/content/transform/TransformerDebug.java @@ -0,0 +1,570 @@ +/* + * Copyright (C) 2005-2012 Alfresco Software Limited. + * + * This file is part of Alfresco + * + * Alfresco is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * Alfresco is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with Alfresco. If not, see . + */ +package org.alfresco.repo.content.transform; + +import java.text.DecimalFormat; +import java.util.ArrayDeque; +import java.util.Deque; +import java.util.Formatter; +import java.util.HashSet; +import java.util.Iterator; +import java.util.List; +import java.util.Set; +import java.util.concurrent.atomic.AtomicInteger; + +import org.alfresco.service.cmr.repository.MimetypeService; +import org.alfresco.util.EqualsHelper; +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; + +/** + * Debugs transformers selection and activity.

+ * + * As transformations are frequently composed of lower level transformations, log + * messages include a prefix to identify the transformation. A numeric dot notation + * is used (such as {@code 123.1.2} indicating the second third level transformation + * of the 123rd top level transformation).

+ * + * In order to track of the nesting of transforms, this class has a stack to represent + * the Transformers. Each Transformer calls {@link #pushTransform} at the start of a + * transform and {@link #popTransform} at the end. However the top level transform may + * be selected from a list of available transformers. To record this activity, + * {@link #pushAvailable}, {@link #unavailableTransformer} (to record the reason a + * transformer is rejected), {@link #availableTransformers} (to record the available + * transformers) and {@link #popAvailable} are called.

+ * + * @author Alan Davis + */ +public class TransformerDebug +{ + private static final Log logger = LogFactory.getLog(TransformerDebug.class); + + private enum Call + { + AVAILABLE, + TRANSFORM, + AVAILABLE_AND_TRANSFORM + }; + + private static class ThreadInfo + { + private static final ThreadLocal threadInfo = new ThreadLocal() + { + @Override + protected ThreadInfo initialValue() + { + return new ThreadInfo(); + } + }; + + private final Deque stack = new ArrayDeque(); + private boolean debugOutput = true; + + public static Deque getStack() + { + return threadInfo.get().stack; + } + + public static boolean getDebug() + { + return threadInfo.get().debugOutput; + } + + public static boolean setDebugOutput(boolean debugOutput) + { + ThreadInfo thisThreadInfo = threadInfo.get(); + boolean orig = thisThreadInfo.debugOutput; + thisThreadInfo.debugOutput = debugOutput; + return orig; + } + } + + private static class Frame + { + private static final AtomicInteger uniqueId = new AtomicInteger(0); + + private final int id; + private final String fromUrl; + private final String sourceMimetype; + private final String targetMimetype; + private final long start; + + private Call callType; + private int childId; + private Set unavailableTransformers; +// See debug(String, Throwable) as to why this is commented out +// private Throwable lastThrowable; + + private Frame(Frame parent, String fromUrl, String sourceMimetype, String targetMimetype, Call pushCall) + { + this.id = parent == null ? uniqueId.getAndIncrement() : ++parent.childId; + this.fromUrl = fromUrl; + this.sourceMimetype = sourceMimetype; + this.targetMimetype = targetMimetype; + this.callType = pushCall; + start = System.currentTimeMillis(); + } + } + + private class UnavailableTransformer + { + private final String name; + private final String reason; + private final transient boolean debug; + + UnavailableTransformer(String name, String reason, boolean debug) + { + this.name = name; + this.reason = reason; + this.debug = debug; + } + + @Override + public int hashCode() + { + int hashCode = 37 * name.hashCode(); + hashCode += 37 * reason.hashCode(); + return hashCode; + } + + @Override + public boolean equals(Object obj) + { + if (this == obj) + { + return true; + } + else if (obj instanceof UnavailableTransformer) + { + UnavailableTransformer that = (UnavailableTransformer) obj; + return + EqualsHelper.nullSafeEquals(name, that.name) && + EqualsHelper.nullSafeEquals(reason, that.reason); + } + else + { + return false; + } + } + } + + private final MimetypeService mimetypeService; + + /** + * Constructor + */ + public TransformerDebug(MimetypeService mimetypeService) + { + this.mimetypeService = mimetypeService; + } + + /** + * Called prior to working out what transformers are available. + */ + public void pushAvailable(String fromUrl, String sourceMimetype, String targetMimetype) + { + if (isEnabled()) + { + push(null, fromUrl, sourceMimetype, targetMimetype, -1, Call.AVAILABLE); + } + } + + /** + * Called prior to performing a transform. + */ + public void pushTransform(ContentTransformer transformer, String fromUrl, String sourceMimetype, String targetMimetype, long sourceSize) + { + if (isEnabled()) + { + push(getName(transformer), fromUrl, sourceMimetype, targetMimetype, sourceSize, Call.TRANSFORM); + } + } + + private void push(String name, String fromUrl, String sourceMimetype, String targetMimetype, long sourceSize, Call callType) + { + Deque ourStack = ThreadInfo.getStack(); + Frame frame = ourStack.peek(); + + if (callType == Call.TRANSFORM && frame != null && frame.callType == Call.AVAILABLE) + { + frame.callType = Call.AVAILABLE_AND_TRANSFORM; + } + else + { + frame = new Frame(frame, fromUrl, sourceMimetype, targetMimetype, callType); + ourStack.push(frame); + + if (callType == Call.TRANSFORM) + { + // Log the basic info about this transformation + logBasicDetails(frame, sourceSize, name, (ourStack.size() == 1)); + } + } + } + + /** + * Called to identify a transformer that cannot be used during working out + * available transformers. + */ + public void unavailableTransformer(ContentTransformer transformer, long maxSourceSizeKBytes) + { + if (isEnabled()) + { + Deque ourStack = ThreadInfo.getStack(); + Frame frame = ourStack.peek(); + + if (frame != null) + { + String name = getName(transformer); + String reason = String.format("> %,dK", maxSourceSizeKBytes); + boolean debug = (maxSourceSizeKBytes != 0); + if (ourStack.size() == 1) + { + if (frame.unavailableTransformers == null) + { + frame.unavailableTransformers = new HashSet(); + } + frame.unavailableTransformers.add(new UnavailableTransformer(name, reason, debug)); + } + else + { + log("-- " + name + ' ' + reason, debug); + } + } + } + } + + /** + * Called once all available transformers have been identified. + */ + public void availableTransformers(List transformers, long sourceSize, String calledFrom) + { + if (isEnabled()) + { + Deque ourStack = ThreadInfo.getStack(); + Frame frame = ourStack.peek(); + + // Log the basic info about this transformation + logBasicDetails(frame, sourceSize, + calledFrom + ((transformers.size() == 0) ? " NO transformers" : ""), + (ourStack.size() == 1)); + + // Report available and unavailable transformers + char c = 'a'; + int longestNameLength = getLongestTransformerNameLength(transformers, frame); + for (ContentTransformer trans : transformers) + { + String name = getName(trans); + int pad = longestNameLength - name.length(); + log((c == 'a' ? "**" : " ") + (c++) + ") " + + name + spaces(pad+1) + trans.getTransformationTime() + " ms"); + } + if (frame.unavailableTransformers != null) + { + for (UnavailableTransformer unavailable: frame.unavailableTransformers) + { + int pad = longestNameLength - unavailable.name.length(); + log("--" + (c++) + ") " + unavailable.name + spaces(pad+1) + unavailable.reason, + unavailable.debug); + } + } + } + } + + private int getLongestTransformerNameLength(List transformers, + Frame frame) + { + int longestNameLength = 0; + for (ContentTransformer trans : transformers) + { + int length = getName(trans).length(); + if (longestNameLength < length) + longestNameLength = length; + } + if (frame.unavailableTransformers != null) + { + for (UnavailableTransformer unavailable: frame.unavailableTransformers) + { + int length = unavailable.name.length(); + if (longestNameLength < length) + longestNameLength = length; + } + } + return longestNameLength; + } + + private void logBasicDetails(Frame frame, long sourceSize, String message, boolean firstLevel) + { + // Log the source URL, but there is no point if the parent has logged it + if (frame.fromUrl != null && (firstLevel || frame.id != 1)) + { + log(frame.fromUrl, firstLevel); + } + + log(getMimetypeExt(frame.sourceMimetype)+getMimetypeExt(frame.targetMimetype) + String.format("%,dK ", (sourceSize/1024)) + message); + + log(frame.sourceMimetype+' '+frame.targetMimetype, false); + } + + /** + * Called after working out what transformers are available and any + * resulting transform has been called. + */ + public void popAvailable() + { + if (isEnabled()) + { + pop(Call.AVAILABLE); + } + } + + /** + * Called after performing a transform. + */ + public void popTransform() + { + if (isEnabled()) + { + pop(Call.TRANSFORM); + } + } + + private void pop(Call callType) + { + Deque ourStack = ThreadInfo.getStack(); + if (!ourStack.isEmpty()) + { + Frame frame = ourStack.peek(); + if ((frame.callType == callType) || + (frame.callType == Call.AVAILABLE_AND_TRANSFORM && callType == Call.AVAILABLE)) + { + if (ourStack.size() == 1 || logger.isTraceEnabled()) + { + boolean topFrame = ourStack.size() == 1; + log("Finished in " + + (System.currentTimeMillis() - frame.start) + " ms" + + (frame.callType == Call.AVAILABLE ? " Transformer NOT called" : "") + + (topFrame ? "\n" : ""), + topFrame); + } + + ourStack.pop(); + +// See debug(String, Throwable) as to why this is commented out +// if (ourStack.size() >= 1) +// { +// ourStack.peek().lastThrowable = frame.lastThrowable; +// } + } + } + } + + /** + * Indicates if any logging is required. + */ + public boolean isEnabled() + { + return + (logger.isDebugEnabled() && ThreadInfo.getDebug()) || + logger.isTraceEnabled(); + } + + /** + * Enable or disable debug log output. Normally used to hide calls to + * getTransformer as trace rather than debug level log messages. There + * are lots of these and it makes it hard to see what is going on. + * @param debugOutput if {@code true} both debug and trace is generated. Otherwise all output is trace. + * @return the original value. + */ + public static boolean setDebugOutput(boolean debugOutput) + { + return ThreadInfo.setDebugOutput(debugOutput); + } + + /** + * Log a message prefixed with the current transformation reference. + * @param message + */ + public void debug(String message) + { + if (isEnabled() && message != null) + { + log(message); + } + } + + /** + * Log a message prefixed with the current transformation reference + * and include a exception, suppressing the stack trace if repeated + * as we return up the stack of transformers. + * @param message + */ + public void debug(String message, Throwable t) + { + if (isEnabled()) + { + log(message + ' ' + t.getMessage()); + +// // Generally the full stack is not needed as transformer +// // Exceptions get logged as a Error higher up, so including +// // the stack trace has been found not to be needed. Keeping +// // the following code and code that sets lastThrowable just +// // in case we need it after all. +// +// Frame frame = ThreadInfo.getStack().peek(); +// boolean newThrowable = isNewThrowable(frame.lastThrowable, t); +// frame.lastThrowable = t; +// +// if (newThrowable) +// { +// log(message, t, true); +// } +// else +// { +// log(message + ' ' + t.getMessage()); +// } + } + } + +// private boolean isNewThrowable(Throwable lastThrowable, Throwable t) +// { +// while (t != null) +// { +// if (lastThrowable == t) +// { +// return false; +// } +// t = t.getCause(); +// } +// return true; +// } + + private void log(String message) + { + log(message, true); + } + + private void log(String message, boolean debug) + { + log(message, null, debug); + } + + private void log(String message, Throwable t, boolean debug) + { + if (debug && ThreadInfo.getDebug()) + { + logger.debug(getReference()+message, t); + } + else + { + logger.trace(getReference()+message, t); + } + } + + /** + * Sets the cause of a transformation failure, so that only the + * message of the Throwable is reported later rather than the full + * stack trace over and over. + */ + public T setCause(T t) + { +// See debug(String, Throwable) as to why this is commented out +// if (isEnabled()) +// { +// Deque ourStack = ThreadInfo.getStack(); +// if (!ourStack.isEmpty()) +// { +// ourStack.peek().lastThrowable = t; +// } +// } + return t; + } + + private String getReference() + { + StringBuilder sb = new StringBuilder(""); + Frame frame = null; + Iterator iterator = ThreadInfo.getStack().descendingIterator(); + int lengthOfFirstId = 0; + while (iterator.hasNext()) + { + frame = iterator.next(); + if (sb.length() == 0) + { + sb.append(frame.id); + lengthOfFirstId = sb.length(); + } + else + { + sb.append('.'); + sb.append(frame.id); + } + } + if (frame != null) + { + sb.append(spaces(9-sb.length()+lengthOfFirstId)); // Try to pad to level 5 + } + return sb.toString(); + } + + private String getName(ContentTransformer transformer) + { + return + (transformer instanceof AbstractContentTransformer2 + ? ((AbstractContentTransformerLimits)transformer).getBeanName() + : transformer.getClass().getSimpleName())+ + + (transformer instanceof ComplexContentTransformer + ? "<>" + : transformer instanceof FailoverContentTransformer + ? "<>" + : transformer instanceof ProxyContentTransformer + ? (((ProxyContentTransformer)transformer).getWorker() instanceof RuntimeExecutableContentTransformerWorker) + ? "<>" + : "<>" + : ""); + } + + private String getMimetypeExt(String mimetype) + { + StringBuilder sb = new StringBuilder(""); + if (mimetypeService == null) + { + sb.append(mimetype); + sb.append(' '); + } + else + { + String mimetypeExt = mimetypeService.getExtension(mimetype); + sb.append(mimetypeExt); + sb.append(spaces(5-mimetypeExt.length())); // Pad to normal max ext (4) plus 1 + } + return sb.toString(); + } + + private String spaces(int i) + { + StringBuilder sb = new StringBuilder(""); + while (--i >= 0) + { + sb.append(' '); + } + return sb.toString(); + } +} diff --git a/source/java/org/alfresco/repo/content/transform/magick/ImageMagickContentTransformerTest.java b/source/java/org/alfresco/repo/content/transform/magick/ImageMagickContentTransformerTest.java index 0e6c3c9ccd..deed319f80 100644 --- a/source/java/org/alfresco/repo/content/transform/magick/ImageMagickContentTransformerTest.java +++ b/source/java/org/alfresco/repo/content/transform/magick/ImageMagickContentTransformerTest.java @@ -1,5 +1,5 @@ /* - * Copyright (C) 2005-2010 Alfresco Software Limited. + * Copyright (C) 2005-2011 Alfresco Software Limited. * * This file is part of Alfresco * @@ -35,7 +35,7 @@ import org.alfresco.util.exec.RuntimeExec; public class ImageMagickContentTransformerTest extends AbstractContentTransformerTest { private ImageMagickContentTransformerWorker worker; - private ContentTransformer transformer; + private ProxyContentTransformer transformer; @Override public void setUp() throws Exception @@ -51,11 +51,10 @@ public class ImageMagickContentTransformerTest extends AbstractContentTransforme worker.setExecuter(executer); worker.afterPropertiesSet(); - ProxyContentTransformer transformer = new ProxyContentTransformer(); + transformer = new ProxyContentTransformer(); transformer.setMimetypeService(mimetypeService); + transformer.setTransformerDebug(transformerDebug); transformer.setWorker(worker); - this.transformer = transformer; - } /** @@ -73,10 +72,10 @@ public class ImageMagickContentTransformerTest extends AbstractContentTransforme return; } boolean reliability = transformer.isTransformable( - MimetypeMap.MIMETYPE_IMAGE_GIF, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions()); + MimetypeMap.MIMETYPE_IMAGE_GIF, -1, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions()); assertEquals("Mimetype should not be supported", false, reliability); reliability = transformer.isTransformable( - MimetypeMap.MIMETYPE_IMAGE_GIF, MimetypeMap.MIMETYPE_IMAGE_JPEG, new TransformationOptions()); + MimetypeMap.MIMETYPE_IMAGE_GIF, -1, MimetypeMap.MIMETYPE_IMAGE_JPEG, new TransformationOptions()); assertEquals("Mimetype should be supported", true, reliability); } } diff --git a/source/java/org/alfresco/repo/content/transform/magick/ImageMagickContentTransformerWorker.java b/source/java/org/alfresco/repo/content/transform/magick/ImageMagickContentTransformerWorker.java index c06f444544..9f3ffda96f 100644 --- a/source/java/org/alfresco/repo/content/transform/magick/ImageMagickContentTransformerWorker.java +++ b/source/java/org/alfresco/repo/content/transform/magick/ImageMagickContentTransformerWorker.java @@ -172,7 +172,8 @@ public class ImageMagickContentTransformerWorker extends AbstractImageMagickCont properties.put(VAR_TARGET, targetFile.getAbsolutePath()); // execute the statement - RuntimeExec.ExecutionResult result = executer.execute(properties); + long timeoutMs = options.getTimeoutMs(); + RuntimeExec.ExecutionResult result = executer.execute(properties, timeoutMs); if (result.getExitValue() != 0 && result.getStdErr() != null && result.getStdErr().length() > 0) { throw new ContentIOException("Failed to perform ImageMagick transformation: \n" + result); diff --git a/source/java/org/alfresco/repo/jscript/ScriptNode.java b/source/java/org/alfresco/repo/jscript/ScriptNode.java index f555ec67d8..f455265e40 100644 --- a/source/java/org/alfresco/repo/jscript/ScriptNode.java +++ b/source/java/org/alfresco/repo/jscript/ScriptNode.java @@ -76,6 +76,7 @@ import org.alfresco.service.cmr.repository.NodeService; import org.alfresco.service.cmr.repository.Path; import org.alfresco.service.cmr.repository.StoreRef; import org.alfresco.service.cmr.repository.TemplateImageResolver; +import org.alfresco.service.cmr.repository.datatype.DefaultTypeConverter; import org.alfresco.service.cmr.search.QueryParameterDefinition; import org.alfresco.service.cmr.security.AccessPermission; import org.alfresco.service.cmr.security.AccessStatus; @@ -2730,7 +2731,9 @@ public class ScriptNode implements Scopeable, NamespacePrefixResolverProvider // If there's nothing currently registered to generate thumbnails for the // specified mimetype, then log a message and bail out String nodeMimeType = getMimetype(); - if (!registry.isThumbnailDefinitionAvailable(nodeMimeType, details)) + Serializable value = this.nodeService.getProperty(nodeRef, ContentModel.PROP_CONTENT); + ContentData contentData = DefaultTypeConverter.INSTANCE.convert(ContentData.class, value); + if (!registry.isThumbnailDefinitionAvailable(contentData.getContentUrl(), nodeMimeType, getSize(), details)) { logger.info("Unable to create thumbnail '" + details.getName() + "' for " + nodeMimeType + " as no transformer is currently available"); @@ -2825,7 +2828,7 @@ public class ScriptNode implements Scopeable, NamespacePrefixResolverProvider if (contentReader != null) { String mimetype = contentReader.getMimetype(); - List thumbnailDefinitions = thumbnailService.getThumbnailRegistry().getThumnailDefintions(mimetype); + List thumbnailDefinitions = thumbnailService.getThumbnailRegistry().getThumbnailDefinitions(contentReader.getContentUrl(), mimetype, contentReader.getSize()); for (ThumbnailDefinition thumbnailDefinition : thumbnailDefinitions) { result.add(thumbnailDefinition.getName()); diff --git a/source/java/org/alfresco/repo/rendition/executer/AbstractTransformationRenderingEngine.java b/source/java/org/alfresco/repo/rendition/executer/AbstractTransformationRenderingEngine.java index 301a0664b3..a82db6d390 100644 --- a/source/java/org/alfresco/repo/rendition/executer/AbstractTransformationRenderingEngine.java +++ b/source/java/org/alfresco/repo/rendition/executer/AbstractTransformationRenderingEngine.java @@ -1,5 +1,5 @@ /* - * Copyright (C) 2005-2010 Alfresco Software Limited. + * Copyright (C) 2005-2011 Alfresco Software Limited. * * This file is part of Alfresco * @@ -20,10 +20,12 @@ package org.alfresco.repo.rendition.executer; import org.alfresco.repo.content.transform.ContentTransformer; +import org.alfresco.repo.content.transform.TransformerDebug; import org.alfresco.service.cmr.rendition.RenditionServiceException; import org.alfresco.service.cmr.repository.ContentReader; import org.alfresco.service.cmr.repository.ContentWriter; import org.alfresco.service.cmr.repository.NoTransformerException; +import org.alfresco.service.cmr.repository.TransformationOptionLimits; import org.alfresco.service.cmr.repository.TransformationOptions; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; @@ -35,6 +37,42 @@ public abstract class AbstractTransformationRenderingEngine extends AbstractRend { private static Log logger = LogFactory.getLog(AbstractTransformationRenderingEngine.class); + /** + * This optional {@link Long} parameter specifies the timeout for reading + * the source before an exception is thrown. + */ + public static final String PARAM_TIMEOUT_MS = TransformationOptionLimits.OPT_TIMEOUT_MS; + + /** + * This optional {@link Long} parameter specifies how timeout for reading + * the source before EOF is returned. + */ + public static final String PARAM_READ_LIMIT_TIME_MS = TransformationOptionLimits.OPT_READ_LIMIT_TIME_MS; + + /** + * This optional {@link Long} parameter specifies the maximum number of kbytes of + * the source may be read. An exception is thrown before any are read if larger. + */ + public static final String PARAM_MAX_SOURCE_SIZE_K_BYTES = TransformationOptionLimits.OPT_MAX_SOURCE_SIZE_K_BYTES; + + /** + * This optional {@link Long} parameter specifies how many kbytes of + * the source to read in order to create an image. + */ + public static final String PARAM_READ_LIMIT_K_BYTES = TransformationOptionLimits.OPT_READ_LIMIT_K_BYTES; + + /** + * This optional {@link Integer} parameter specifies the maximum number of pages of + * the source that may be read. An exception is thrown before any are read if larger. + */ + public static final String PARAM_MAX_PAGES = TransformationOptionLimits.OPT_MAX_PAGES; + + /** + * This optional {@link Integer} parameter specifies how many source + * pages should be read in order to create an image. + */ + public static final String PARAM_PAGE_LIMIT = TransformationOptionLimits.OPT_PAGE_LIMIT; + /* Error messages */ private static final String TRANSFORMER_NOT_EXISTS_MESSAGE_PATTERN = "Transformer for '%s' source mime type and '%s' target mime type was not found. Operation can't be performed"; private static final String NOT_TRANSFORMABLE_MESSAGE_PATTERN = "Content not transformable for '%s' source mime type and '%s' target mime type. Operation can't be performed"; @@ -48,13 +86,24 @@ public abstract class AbstractTransformationRenderingEngine extends AbstractRend protected void render(RenderingContext context) { ContentReader contentReader = context.makeContentReader(); + String sourceUrl = contentReader.getContentUrl(); String sourceMimeType = contentReader.getMimetype(); String targetMimeType = getTargetMimeType(context); TransformationOptions options = getTransformOptions(context); - ContentTransformer transformer = this.contentService.getTransformer(sourceMimeType, targetMimeType, options); - + // Log the following getTransform() as trace so we can see the wood for the trees + ContentTransformer transformer; + boolean orig = TransformerDebug.setDebugOutput(false); + try + { + transformer = this.contentService.getTransformer(sourceUrl, sourceMimeType, contentReader.getSize(), targetMimeType, options); + } + finally + { + TransformerDebug.setDebugOutput(orig); + } + // Actually perform the rendition. if (null == transformer) { @@ -63,7 +112,7 @@ public abstract class AbstractTransformationRenderingEngine extends AbstractRend targetMimeType)); } - if (transformer.isTransformable(sourceMimeType, targetMimeType, options)) + if (transformer.isTransformable(sourceMimeType, contentReader.getSize(), targetMimeType, options)) { ContentWriter contentWriter = context.makeContentWriter(); try @@ -86,4 +135,45 @@ public abstract class AbstractTransformationRenderingEngine extends AbstractRend } protected abstract TransformationOptions getTransformOptions(RenderingContext context); + + protected TransformationOptions getTransformOptionsImpl(TransformationOptions options, RenderingContext context) + { + Long timeoutMs = context.getCheckedParam(PARAM_TIMEOUT_MS, Long.class); + if (timeoutMs != null) + { + options.setTimeoutMs(timeoutMs); + } + + Long readLimitTimeMs = context.getCheckedParam(PARAM_READ_LIMIT_TIME_MS, Long.class); + if (readLimitTimeMs != null) + { + options.setReadLimitTimeMs(readLimitTimeMs); + } + + Long maxSourceSizeKBytes = context.getCheckedParam(PARAM_MAX_SOURCE_SIZE_K_BYTES, Long.class); + if (maxSourceSizeKBytes != null) + { + options.setMaxSourceSizeKBytes(maxSourceSizeKBytes); + } + + Long readLimitKBytes = context.getCheckedParam(PARAM_READ_LIMIT_K_BYTES, Long.class); + if (readLimitKBytes != null) + { + options.setReadLimitKBytes(readLimitKBytes); + } + + Integer maxPages = context.getCheckedParam(PARAM_MAX_PAGES, Integer.class); + if (maxPages != null) + { + options.setMaxPages(maxPages); + } + + Integer pageLimit = context.getCheckedParam(PARAM_PAGE_LIMIT, Integer.class); + if (pageLimit != null) + { + options.setPageLimit(pageLimit); + } + + return options; + } } diff --git a/source/java/org/alfresco/repo/rendition/executer/ImageRenderingEngine.java b/source/java/org/alfresco/repo/rendition/executer/ImageRenderingEngine.java index b065f09dcc..c0407cdf98 100644 --- a/source/java/org/alfresco/repo/rendition/executer/ImageRenderingEngine.java +++ b/source/java/org/alfresco/repo/rendition/executer/ImageRenderingEngine.java @@ -1,5 +1,5 @@ /* - * Copyright (C) 2005-2010 Alfresco Software Limited. + * Copyright (C) 2005-2011 Alfresco Software Limited. * * This file is part of Alfresco * @@ -25,6 +25,7 @@ import org.alfresco.repo.action.ParameterDefinitionImpl; import org.alfresco.repo.content.transform.magick.ImageCropOptions; import org.alfresco.repo.content.transform.magick.ImageResizeOptions; import org.alfresco.repo.content.transform.magick.ImageTransformationOptions; +import org.alfresco.repo.rendition.executer.AbstractRenderingEngine.RenderingContext; import org.alfresco.service.cmr.action.Action; import org.alfresco.service.cmr.action.ParameterDefinition; import org.alfresco.service.cmr.dictionary.DataTypeDefinition; @@ -228,7 +229,7 @@ public class ImageRenderingEngine extends AbstractTransformationRenderingEngine * appended after the various crop and resize options. */ public static final String PARAM_COMMAND_OPTIONS = "commandOptions"; - + /** * This optional {@link Boolean} flag parameter specifies if the engine should * automatically rotate and image based on the EXIF orientation flag. If @@ -249,13 +250,20 @@ public class ImageRenderingEngine extends AbstractTransformationRenderingEngine @Override protected TransformationOptions getTransformOptions(RenderingContext context) { + return getTransformOptionsImpl(new ImageTransformationOptions(), context); + } + + @Override + protected TransformationOptions getTransformOptionsImpl(TransformationOptions options, RenderingContext context) + { + ImageTransformationOptions imageTransformationOptions = (ImageTransformationOptions)options; + String commandOptions = context.getCheckedParam(PARAM_COMMAND_OPTIONS, String.class); ImageResizeOptions imageResizeOptions = getImageResizeOptions(context); ImageCropOptions cropOptions = getImageCropOptions(context); boolean autoOrient = context.getParamWithDefault(PARAM_AUTO_ORIENTATION, true); - ImageTransformationOptions imageTransformationOptions = new ImageTransformationOptions(); imageTransformationOptions.setResizeOptions(imageResizeOptions); imageTransformationOptions.setCropOptions(cropOptions); imageTransformationOptions.setAutoOrient(autoOrient); @@ -263,7 +271,8 @@ public class ImageRenderingEngine extends AbstractTransformationRenderingEngine { imageTransformationOptions.setCommandOptions(commandOptions); } - return imageTransformationOptions; + + return super.getTransformOptionsImpl(options, context); } /* @@ -408,7 +417,21 @@ public class ImageRenderingEngine extends AbstractTransformationRenderingEngine getParamDisplayLabel(PARAM_IS_PERCENT_CROP))); paramList.add(new ParameterDefinitionImpl(PARAM_COMMAND_OPTIONS, DataTypeDefinition.TEXT, false, - getParamDisplayLabel(PARAM_COMMAND_OPTIONS))); + getParamDisplayLabel(PARAM_COMMAND_OPTIONS))); + + paramList.add(new ParameterDefinitionImpl(PARAM_TIMEOUT_MS, DataTypeDefinition.LONG, false, + getParamDisplayLabel(PARAM_TIMEOUT_MS))); + paramList.add(new ParameterDefinitionImpl(PARAM_READ_LIMIT_TIME_MS, DataTypeDefinition.LONG, false, + getParamDisplayLabel(PARAM_READ_LIMIT_TIME_MS))); + paramList.add(new ParameterDefinitionImpl(PARAM_MAX_SOURCE_SIZE_K_BYTES, DataTypeDefinition.LONG, false, + getParamDisplayLabel(PARAM_MAX_SOURCE_SIZE_K_BYTES))); + paramList.add(new ParameterDefinitionImpl(PARAM_READ_LIMIT_K_BYTES, DataTypeDefinition.LONG, false, + getParamDisplayLabel(PARAM_READ_LIMIT_K_BYTES))); + paramList.add(new ParameterDefinitionImpl(PARAM_MAX_PAGES, DataTypeDefinition.INT, false, + getParamDisplayLabel(PARAM_MAX_PAGES))); + paramList.add(new ParameterDefinitionImpl(PARAM_PAGE_LIMIT, DataTypeDefinition.INT, false, + getParamDisplayLabel(PARAM_PAGE_LIMIT))); + return paramList; } } \ No newline at end of file diff --git a/source/java/org/alfresco/repo/rendition/executer/ReformatRenderingEngine.java b/source/java/org/alfresco/repo/rendition/executer/ReformatRenderingEngine.java index c371710bab..ae6304f0ea 100644 --- a/source/java/org/alfresco/repo/rendition/executer/ReformatRenderingEngine.java +++ b/source/java/org/alfresco/repo/rendition/executer/ReformatRenderingEngine.java @@ -1,5 +1,5 @@ /* - * Copyright (C) 2005-2010 Alfresco Software Limited. + * Copyright (C) 2005-2011 Alfresco Software Limited. * * This file is part of Alfresco * @@ -22,6 +22,8 @@ package org.alfresco.repo.rendition.executer; import java.util.Collection; import org.alfresco.repo.action.ParameterDefinitionImpl; +import org.alfresco.repo.content.transform.magick.ImageTransformationOptions; +import org.alfresco.repo.rendition.executer.AbstractRenderingEngine.RenderingContext; import org.alfresco.service.cmr.action.ParameterDefinition; import org.alfresco.service.cmr.dictionary.DataTypeDefinition; import org.alfresco.service.cmr.rendition.RenditionService; @@ -78,9 +80,16 @@ public class ReformatRenderingEngine extends AbstractTransformationRenderingEngi @Override protected TransformationOptions getTransformOptions(RenderingContext context) { - NodeRef sourceNode = context.getSourceNode(); - NodeRef destinationNode = context.getDestinationNode(); - return new TransformationOptions(sourceNode, null, destinationNode, null); + return getTransformOptionsImpl(new TransformationOptions(), context); + } + + @Override + protected TransformationOptions getTransformOptionsImpl(TransformationOptions options, RenderingContext context) + { + options.setSourceNodeRef(context.getSourceNode()); + options.setTargetNodeRef(context.getDestinationNode()); + + return super.getTransformOptionsImpl(options, context); } /* diff --git a/source/java/org/alfresco/repo/rule/RuleServiceCoverageTest.java b/source/java/org/alfresco/repo/rule/RuleServiceCoverageTest.java index b6bd092622..af63501a13 100644 --- a/source/java/org/alfresco/repo/rule/RuleServiceCoverageTest.java +++ b/source/java/org/alfresco/repo/rule/RuleServiceCoverageTest.java @@ -1067,7 +1067,7 @@ public class RuleServiceCoverageTest extends TestCase public void testTransformAction() throws Throwable { ContentTransformer transformer = transformerRegistry.getTransformer( - MimetypeMap.MIMETYPE_EXCEL, + MimetypeMap.MIMETYPE_EXCEL, -1, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions()); if (transformer == null) @@ -1148,7 +1148,7 @@ public class RuleServiceCoverageTest extends TestCase public void testImageTransformAction() throws Throwable { ContentTransformer transformer = transformerRegistry.getTransformer( - MimetypeMap.MIMETYPE_IMAGE_GIF, + MimetypeMap.MIMETYPE_IMAGE_GIF, -1, MimetypeMap.MIMETYPE_IMAGE_JPEG, new TransformationOptions()); if (transformer == null) @@ -1939,7 +1939,7 @@ public class RuleServiceCoverageTest extends TestCase public void testAsyncExecutionWithPotentialLoop() { - if (this.transformerRegistry.getTransformer(MimetypeMap.MIMETYPE_EXCEL, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions()) != null) + if (this.transformerRegistry.getTransformer(MimetypeMap.MIMETYPE_EXCEL, -1, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions()) != null) { try { diff --git a/source/java/org/alfresco/repo/search/impl/lucene/ADMLuceneIndexerAndSearcherFactory.java b/source/java/org/alfresco/repo/search/impl/lucene/ADMLuceneIndexerAndSearcherFactory.java index cc813b6553..1fa77463d9 100644 --- a/source/java/org/alfresco/repo/search/impl/lucene/ADMLuceneIndexerAndSearcherFactory.java +++ b/source/java/org/alfresco/repo/search/impl/lucene/ADMLuceneIndexerAndSearcherFactory.java @@ -20,6 +20,7 @@ package org.alfresco.repo.search.impl.lucene; import java.util.List; +import org.alfresco.repo.content.transform.TransformerDebug; import org.alfresco.repo.search.SearcherException; import org.alfresco.repo.search.SupportsBackgroundIndexing; import org.alfresco.repo.search.impl.lucene.fts.FullTextSearchIndexer; @@ -42,6 +43,8 @@ public class ADMLuceneIndexerAndSearcherFactory extends AbstractLuceneIndexerAnd protected NodeService nodeService; protected FullTextSearchIndexer fullTextSearchIndexer; protected ContentService contentService; + private TransformerDebug transformerDebug; + protected TransactionService transactionService; /** @@ -80,6 +83,15 @@ public class ADMLuceneIndexerAndSearcherFactory extends AbstractLuceneIndexerAnd { this.contentService = contentService; } + + /** + * Sets the transformer debug. + * @param transformerDebug + */ + public void setTransformerDebug(TransformerDebug transformerDebug) + { + this.transformerDebug = transformerDebug; + } /** * Set the transaction service @@ -101,6 +113,7 @@ public class ADMLuceneIndexerAndSearcherFactory extends AbstractLuceneIndexerAnd // indexer.setLuceneIndexLock(luceneIndexLock); indexer.setFullTextSearchIndexer(fullTextSearchIndexer); indexer.setContentService(contentService); + indexer.setTransformerDebug(transformerDebug); indexer.setTransactionService(transactionService); indexer.setMaxAtomicTransformationTime(getMaxTransformationTime()); return indexer; diff --git a/source/java/org/alfresco/repo/search/impl/lucene/ADMLuceneIndexerImpl.java b/source/java/org/alfresco/repo/search/impl/lucene/ADMLuceneIndexerImpl.java index cf26e509a4..a048659133 100644 --- a/source/java/org/alfresco/repo/search/impl/lucene/ADMLuceneIndexerImpl.java +++ b/source/java/org/alfresco/repo/search/impl/lucene/ADMLuceneIndexerImpl.java @@ -46,6 +46,7 @@ import java.util.Set; import org.alfresco.model.ContentModel; import org.alfresco.repo.content.MimetypeMap; import org.alfresco.repo.content.transform.ContentTransformer; +import org.alfresco.repo.content.transform.TransformerDebug; import org.alfresco.repo.dictionary.IndexTokenisationMode; import org.alfresco.repo.search.IndexerException; import org.alfresco.repo.search.MLAnalysisMode; @@ -74,6 +75,7 @@ import org.alfresco.service.cmr.repository.NodeRef; import org.alfresco.service.cmr.repository.NodeService; import org.alfresco.service.cmr.repository.Path; import org.alfresco.service.cmr.repository.StoreRef; +import org.alfresco.service.cmr.repository.TransformationOptions; import org.alfresco.service.cmr.repository.Path.ChildAssocElement; import org.alfresco.service.cmr.repository.datatype.DefaultTypeConverter; import org.alfresco.service.cmr.repository.datatype.TypeConversionException; @@ -129,6 +131,8 @@ public class ADMLuceneIndexerImpl extends AbstractLuceneIndexerImpl imp */ ContentService contentService; + private TransformerDebug transformerDebug; + /** * Call back to make after doing non atomic indexing */ @@ -182,6 +186,15 @@ public class ADMLuceneIndexerImpl extends AbstractLuceneIndexerImpl imp this.contentService = contentService; } + /** + * Helper setter of the transformer debug. + * @param transformerDebug + */ + public void setTransformerDebug(TransformerDebug transformerDebug) + { + this.transformerDebug = transformerDebug; + } + /* * Indexer Implementation */ @@ -1263,61 +1276,78 @@ public class ADMLuceneIndexerImpl extends AbstractLuceneIndexerImpl imp // transform if necessary (it is not a UTF-8 text document) if (!EqualsHelper.nullSafeEquals(reader.getMimetype(), MimetypeMap.MIMETYPE_TEXT_PLAIN) || !EqualsHelper.nullSafeEquals(reader.getEncoding(), "UTF-8")) { - // get the transformer - ContentTransformer transformer = contentService.getTransformer(reader.getMimetype(), MimetypeMap.MIMETYPE_TEXT_PLAIN); - // is this transformer good enough? - if (transformer == null) + try { - // log it - if (s_logger.isInfoEnabled()) - { - s_logger.info("Not indexed: No transformation: \n" - + " source: " + reader + "\n" + " target: " + MimetypeMap.MIMETYPE_TEXT_PLAIN + " at " + nodeService.getPath(nodeRef)); - } - // don't index from the reader - readerReady = false; - // not indexed: no transformation - // doc.add(new Field("TEXT", NOT_INDEXED_NO_TRANSFORMATION, Field.Store.NO, - // Field.Index.TOKENIZED, Field.TermVector.NO)); - doc.add(new Field(attributeName, NOT_INDEXED_NO_TRANSFORMATION, Field.Store.NO, Field.Index.TOKENIZED, Field.TermVector.NO)); - } - else if (indexAtomicPropertiesOnly && transformer.getTransformationTime() > maxAtomicTransformationTime) - { - // only indexing atomic properties - // indexing will take too long, so push it to the background - wereAllAtomic = false; - readerReady = false; - } - else - { - // We have a transformer that is fast enough - ContentWriter writer = contentService.getTempWriter(); - writer.setMimetype(MimetypeMap.MIMETYPE_TEXT_PLAIN); - // this is what the analyzers expect on the stream - writer.setEncoding("UTF-8"); - try - { - transformer.transform(reader, writer); - // point the reader to the new-written content - reader = writer.getReader(); - // Check that the reader is a view onto something concrete - if (!reader.exists()) - { - throw new ContentIOException("The transformation did not write any content, yet: \n" - + " transformer: " + transformer + "\n" + " temp writer: " + writer); - } - } - catch (ContentIOException e) + // get the transformer + transformerDebug.pushAvailable(reader.getContentUrl(), reader.getMimetype(), MimetypeMap.MIMETYPE_TEXT_PLAIN); + long sourceSize = reader.getSize(); + List transformers = contentService.getActiveTransformers(reader.getMimetype(), sourceSize, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions()); + transformerDebug.availableTransformers(transformers, sourceSize, "ADMLuceneIndexer"); + + if (transformers.isEmpty()) { // log it if (s_logger.isInfoEnabled()) { - s_logger.info("Not indexed: Transformation failed at " + nodeService.getPath(nodeRef), e); + s_logger.info("Not indexed: No transformation: \n" + + " source: " + reader + "\n" + " target: " + MimetypeMap.MIMETYPE_TEXT_PLAIN + " at " + nodeService.getPath(nodeRef)); } // don't index from the reader readerReady = false; - doc.add(new Field(attributeName, NOT_INDEXED_TRANSFORMATION_FAILED, Field.Store.NO, Field.Index.TOKENIZED, Field.TermVector.NO)); + // not indexed: no transformation + // doc.add(new Field("TEXT", NOT_INDEXED_NO_TRANSFORMATION, Field.Store.NO, + // Field.Index.TOKENIZED, Field.TermVector.NO)); + doc.add(new Field(attributeName, NOT_INDEXED_NO_TRANSFORMATION, Field.Store.NO, Field.Index.TOKENIZED, Field.TermVector.NO)); } + // is this transformer good enough? + else if (indexAtomicPropertiesOnly && transformers.get(0).getTransformationTime() > maxAtomicTransformationTime) + { + // only indexing atomic properties + // indexing will take too long, so push it to the background + wereAllAtomic = false; + readerReady = false; + + if (transformerDebug.isEnabled()) + { + transformerDebug.debug("Run later. Transformer average ("+transformers.get(0).getTransformationTime()+" ms) > "+maxAtomicTransformationTime+" ms"); + } + } + else + { + // We have a transformer that is fast enough + ContentTransformer transformer = transformers.get(0); + ContentWriter writer = contentService.getTempWriter(); + writer.setMimetype(MimetypeMap.MIMETYPE_TEXT_PLAIN); + // this is what the analyzers expect on the stream + writer.setEncoding("UTF-8"); + try + { + transformer.transform(reader, writer); + // point the reader to the new-written content + reader = writer.getReader(); + // Check that the reader is a view onto something concrete + if (!reader.exists()) + { + throw new ContentIOException("The transformation did not write any content, yet: \n" + + " transformer: " + transformer + "\n" + " temp writer: " + writer); + } + } + catch (ContentIOException e) + { + // log it + if (s_logger.isInfoEnabled()) + { + s_logger.info("Not indexed: Transformation failed at " + nodeService.getPath(nodeRef), e); + } + // don't index from the reader + readerReady = false; + doc.add(new Field(attributeName, NOT_INDEXED_TRANSFORMATION_FAILED, Field.Store.NO, Field.Index.TOKENIZED, Field.TermVector.NO)); + } + } + } + finally + { + transformerDebug.popAvailable(); } } // add the text field using the stream from the diff --git a/source/java/org/alfresco/repo/search/impl/lucene/ADMLuceneTest.java b/source/java/org/alfresco/repo/search/impl/lucene/ADMLuceneTest.java index 8978b6e41a..1da81bc917 100644 --- a/source/java/org/alfresco/repo/search/impl/lucene/ADMLuceneTest.java +++ b/source/java/org/alfresco/repo/search/impl/lucene/ADMLuceneTest.java @@ -49,6 +49,7 @@ import javax.transaction.UserTransaction; import junit.framework.TestCase; import org.alfresco.model.ContentModel; +import org.alfresco.repo.content.transform.TransformerDebug; import org.alfresco.repo.dictionary.DictionaryDAO; import org.alfresco.repo.dictionary.DictionaryListener; import org.alfresco.repo.dictionary.DictionaryNamespaceComponent; @@ -155,6 +156,8 @@ public class ADMLuceneTest extends TestCase implements DictionaryListener TenantService tenantService; + TransformerDebug transformerDebug; + private NodeRef rootNodeRef; private NodeRef n1; @@ -271,6 +274,7 @@ public class ADMLuceneTest extends TestCase implements DictionaryListener contentService = (ContentService) ctx.getBean("contentService"); queryRegisterComponent = (QueryRegisterComponent) ctx.getBean("queryRegisterComponent"); namespacePrefixResolver = (DictionaryNamespaceComponent) ctx.getBean("namespaceService"); + transformerDebug = (TransformerDebug) ctx.getBean("transformerDebug"); indexerAndSearcher = (IndexerAndSearcher) ctx.getBean("admLuceneIndexerAndSearcherFactory"); luceneConfig = (LuceneConfig)ctx.getBean("admLuceneIndexerAndSearcherFactory"); ((LuceneConfig) indexerAndSearcher).setMaxAtomicTransformationTime(1000000); @@ -1316,7 +1320,7 @@ public class ADMLuceneTest extends TestCase implements DictionaryListener buildBaseIndex(); ADMLuceneSearcherImpl searcher = buildSearcher(); - + List expected = new ArrayList(15); SearchParameters sp = new SearchParameters(); @@ -3956,6 +3960,8 @@ public class ADMLuceneTest extends TestCase implements DictionaryListener indexer.setFullTextSearchIndexer(luceneFTS); indexer.setContentService(contentService); indexer.setTransactionService(transactionService); + indexer.setTransformerDebug(transformerDebug); + // indexer.clearIndex(); indexer.createNode(new ChildAssociationRef(null, null, null, rootNodeRef)); indexer.createNode(new ChildAssociationRef(ContentModel.ASSOC_CHILDREN, rootNodeRef, QName.createQName("{namespace}one"), n1)); diff --git a/source/java/org/alfresco/repo/search/impl/lucene/AVMLuceneIndexerAndSearcherFactory.java b/source/java/org/alfresco/repo/search/impl/lucene/AVMLuceneIndexerAndSearcherFactory.java index 3be202a280..13536cc506 100644 --- a/source/java/org/alfresco/repo/search/impl/lucene/AVMLuceneIndexerAndSearcherFactory.java +++ b/source/java/org/alfresco/repo/search/impl/lucene/AVMLuceneIndexerAndSearcherFactory.java @@ -23,6 +23,7 @@ import java.util.List; import org.alfresco.repo.avm.AVMNodeConverter; import org.alfresco.repo.content.ContentStore; +import org.alfresco.repo.content.transform.TransformerDebug; import org.alfresco.repo.search.AVMSnapShotTriggeredIndexingMethodInterceptor; import org.alfresco.repo.search.IndexMode; import org.alfresco.repo.search.SearcherException; @@ -56,6 +57,8 @@ public class AVMLuceneIndexerAndSearcherFactory extends AbstractLuceneIndexerAnd private AVMSyncService avmSyncService; private NodeService nodeService; private ContentStore contentStore; + + private TransformerDebug transformerDebug; private FullTextSearchIndexer fullTextSearchIndexer; private AVMSnapShotTriggeredIndexingMethodInterceptor avmSnapShotTriggeredIndexingMethodInterceptor; @@ -124,6 +127,15 @@ public class AVMLuceneIndexerAndSearcherFactory extends AbstractLuceneIndexerAnd { this.contentStore = contentStore; } + + /** + * Sets the transformer debug. + * @param transformerDebug + */ + public void setTransformerDebug(TransformerDebug transformerDebug) + { + this.transformerDebug = transformerDebug; + } /** * @param avmSnapShotTriggeredIndexingMethodInterceptor the avmSnapShotTriggeredIndexingMethodInterceptor to set @@ -144,6 +156,7 @@ public class AVMLuceneIndexerAndSearcherFactory extends AbstractLuceneIndexerAnd indexer.setAvmService(avmService); indexer.setAvmSyncService(avmSyncService); indexer.setContentStore(contentStore); + indexer.setTransformerDebug(transformerDebug); indexer.setFullTextSearchIndexer(fullTextSearchIndexer); return indexer; } diff --git a/source/java/org/alfresco/repo/search/impl/lucene/AVMLuceneIndexerImpl.java b/source/java/org/alfresco/repo/search/impl/lucene/AVMLuceneIndexerImpl.java index c243801518..c9b6c6482d 100644 --- a/source/java/org/alfresco/repo/search/impl/lucene/AVMLuceneIndexerImpl.java +++ b/source/java/org/alfresco/repo/search/impl/lucene/AVMLuceneIndexerImpl.java @@ -47,6 +47,7 @@ import org.alfresco.repo.avm.util.SimplePath; import org.alfresco.repo.content.ContentStore; import org.alfresco.repo.content.MimetypeMap; import org.alfresco.repo.content.transform.ContentTransformer; +import org.alfresco.repo.content.transform.TransformerDebug; import org.alfresco.repo.dictionary.IndexTokenisationMode; import org.alfresco.repo.domain.PropertyValue; import org.alfresco.repo.search.IndexMode; @@ -78,6 +79,7 @@ import org.alfresco.service.cmr.repository.InvalidNodeRefException; import org.alfresco.service.cmr.repository.MLText; import org.alfresco.service.cmr.repository.NodeRef; import org.alfresco.service.cmr.repository.StoreRef; +import org.alfresco.service.cmr.repository.TransformationOptions; import org.alfresco.service.cmr.repository.datatype.DefaultTypeConverter; import org.alfresco.service.cmr.repository.datatype.TypeConversionException; import org.alfresco.service.namespace.QName; @@ -127,6 +129,8 @@ public class AVMLuceneIndexerImpl extends AbstractLuceneIndexerImpl impl private ContentStore contentStore; private ContentService contentService; + + private TransformerDebug transformerDebug; private FTSIndexerAware callBack; @@ -183,6 +187,15 @@ public class AVMLuceneIndexerImpl extends AbstractLuceneIndexerImpl impl this.contentService = contentService; } + /** + * Helper setter of the transformer debug. + * @param transformerDebug + */ + public void setTransformerDebug(TransformerDebug transformerDebug) + { + this.transformerDebug = transformerDebug; + } + /** * Are we deleting leaves only (not meta data) * @@ -1124,65 +1137,76 @@ public class AVMLuceneIndexerImpl extends AbstractLuceneIndexerImpl impl // transform if necessary (it is not a UTF-8 text document) if (!EqualsHelper.nullSafeEquals(reader.getMimetype(), MimetypeMap.MIMETYPE_TEXT_PLAIN) || !EqualsHelper.nullSafeEquals(reader.getEncoding(), "UTF-8")) { - // get the transformer - ContentTransformer transformer = contentService.getTransformer(reader.getMimetype(), MimetypeMap.MIMETYPE_TEXT_PLAIN); - // is this transformer good enough? - if (transformer == null) + try { - // log it - if (s_logger.isDebugEnabled()) - { - s_logger.debug("Not indexed: No transformation: \n" + " source: " + reader + "\n" + " target: " + MimetypeMap.MIMETYPE_TEXT_PLAIN); - } - // don't index from the reader - readerReady = false; - // not indexed: no transformation - // doc.add(new Field("TEXT", NOT_INDEXED_NO_TRANSFORMATION, Field.Store.NO, - // Field.Index.TOKENIZED, Field.TermVector.NO)); - doc.add(new Field(attributeName, NOT_INDEXED_NO_TRANSFORMATION, Field.Store.NO, Field.Index.TOKENIZED, Field.TermVector.NO)); - } - // else if (indexAtomicPropertiesOnly - // && transformer.getTransformationTime() > maxAtomicTransformationTime) - // { - // only indexing atomic properties - // indexing will take too long, so push it to the background - // wereAllAtomic = false; - // } - else - { - // We have a transformer that is fast enough - ContentWriter writer = contentService.getTempWriter(); - writer.setMimetype(MimetypeMap.MIMETYPE_TEXT_PLAIN); - // this is what the analyzers expect on the stream - writer.setEncoding("UTF-8"); - try - { + // get the transformer + transformerDebug.pushAvailable(reader.getContentUrl(), reader.getMimetype(), MimetypeMap.MIMETYPE_TEXT_PLAIN); + long sourceSize = reader.getSize(); + List transformers = contentService.getActiveTransformers(reader.getMimetype(), sourceSize, MimetypeMap.MIMETYPE_TEXT_PLAIN, new TransformationOptions()); + transformerDebug.availableTransformers(transformers, sourceSize, "AVMLuceneIndexer"); - transformer.transform(reader, writer); - // point the reader to the new-written content - reader = writer.getReader(); - // Check that the reader is a view onto something concrete - if (!reader.exists()) - { - throw new ContentIOException("The transformation did not write any content, yet: \n" - + " transformer: " + transformer + "\n" + " temp writer: " + writer); - } - } - catch (ContentIOException e) + if (transformers.isEmpty()) { // log it if (s_logger.isDebugEnabled()) { - s_logger.debug("Not indexed: Transformation failed", e); + s_logger.debug("Not indexed: No transformation: \n" + " source: " + reader + "\n" + " target: " + MimetypeMap.MIMETYPE_TEXT_PLAIN); } // don't index from the reader readerReady = false; - // not indexed: transformation - // failed - // doc.add(new Field("TEXT", NOT_INDEXED_TRANSFORMATION_FAILED, Field.Store.NO, + // not indexed: no transformation + // doc.add(new Field("TEXT", NOT_INDEXED_NO_TRANSFORMATION, Field.Store.NO, // Field.Index.TOKENIZED, Field.TermVector.NO)); - doc.add(new Field(attributeName, NOT_INDEXED_TRANSFORMATION_FAILED, Field.Store.NO, Field.Index.TOKENIZED, Field.TermVector.NO)); + doc.add(new Field(attributeName, NOT_INDEXED_NO_TRANSFORMATION, Field.Store.NO, Field.Index.TOKENIZED, Field.TermVector.NO)); } + // is this transformer good enough? + // else if (indexAtomicPropertiesOnly + // && transformers.get(0).getTransformationTime() > maxAtomicTransformationTime) + // { + // only indexing atomic properties + // indexing will take too long, so push it to the background + // wereAllAtomic = false; + // } + else + { + // We have a transformer that is fast enough + ContentTransformer transformer = transformers.get(0); + ContentWriter writer = contentService.getTempWriter(); + writer.setMimetype(MimetypeMap.MIMETYPE_TEXT_PLAIN); + // this is what the analyzers expect on the stream + writer.setEncoding("UTF-8"); + try + { + transformer.transform(reader, writer); + // point the reader to the new-written content + reader = writer.getReader(); + // Check that the reader is a view onto something concrete + if (!reader.exists()) + { + throw new ContentIOException("The transformation did not write any content, yet: \n" + + " transformer: " + transformer + "\n" + " temp writer: " + writer); + } + } + catch (ContentIOException e) + { + // log it + if (s_logger.isDebugEnabled()) + { + s_logger.debug("Not indexed: Transformation failed", e); + } + // don't index from the reader + readerReady = false; + // not indexed: transformation + // failed + // doc.add(new Field("TEXT", NOT_INDEXED_TRANSFORMATION_FAILED, Field.Store.NO, + // Field.Index.TOKENIZED, Field.TermVector.NO)); + doc.add(new Field(attributeName, NOT_INDEXED_TRANSFORMATION_FAILED, Field.Store.NO, Field.Index.TOKENIZED, Field.TermVector.NO)); + } + } + } + finally + { + transformerDebug.popAvailable(); } } // add the text field using the stream from the diff --git a/source/java/org/alfresco/repo/thumbnail/CreateThumbnailActionExecuter.java b/source/java/org/alfresco/repo/thumbnail/CreateThumbnailActionExecuter.java index 88f94ec2c1..2557c90200 100644 --- a/source/java/org/alfresco/repo/thumbnail/CreateThumbnailActionExecuter.java +++ b/source/java/org/alfresco/repo/thumbnail/CreateThumbnailActionExecuter.java @@ -1,5 +1,5 @@ /* - * Copyright (C) 2005-2010 Alfresco Software Limited. + * Copyright (C) 2005-2011 Alfresco Software Limited. * * This file is part of Alfresco * @@ -19,12 +19,14 @@ package org.alfresco.repo.thumbnail; import java.io.Serializable; +import java.util.HashMap; import java.util.List; import org.alfresco.error.AlfrescoRuntimeException; import org.alfresco.model.ContentModel; import org.alfresco.repo.action.ParameterDefinitionImpl; import org.alfresco.repo.action.executer.ActionExecuterAbstractBase; +import org.alfresco.repo.content.transform.TransformerDebug; import org.alfresco.service.cmr.action.Action; import org.alfresco.service.cmr.action.ParameterDefinition; import org.alfresco.service.cmr.dictionary.DataTypeDefinition; @@ -42,6 +44,7 @@ import org.apache.commons.logging.LogFactory; * NOTE: This action is used to facilitate the async creation of thumbnails. It is not intended for genereral useage. * * @author Roy Wetherall + * @author Ph Dubois (optional thumbnail creation by mimetype and in general) */ public class CreateThumbnailActionExecuter extends ActionExecuterAbstractBase { @@ -53,6 +56,12 @@ public class CreateThumbnailActionExecuter extends ActionExecuterAbstractBase /** Node Service */ private NodeService nodeService; + /** Property turns on and off all thumbnail creation */ + private boolean generateThumbnails = true; + + // Size limitations (in KBytes) indexed by mimetype for thumbnail creation + private HashMap mimetypeMaxSourceSizeKBytes; + /** Action name and parameters */ public static final String NAME = "create-thumbnail"; public static final String PARAM_CONTENT_PROPERTY = "content-property"; @@ -78,12 +87,40 @@ public class CreateThumbnailActionExecuter extends ActionExecuterAbstractBase this.nodeService = nodeService; } + /** + * Set the maximum size for each mimetype above which thumbnails are not created. + * @param mimetypeMaxSourceSizeKBytes map of mimetypes to max source sizes. + */ + public void setMimetypeMaxSourceSizeKBytes(HashMap mimetypeMaxSourceSizeKBytes) + { + this.mimetypeMaxSourceSizeKBytes = mimetypeMaxSourceSizeKBytes; + } + + /** + * Enable thumbnail creation at all regardless of mimetype. + * @param generateThumbnails a {@code false} value turns off all thumbnail creation. + */ + public void setGenerateThumbnails(boolean generateThumbnails) + { + this.generateThumbnails = generateThumbnails; + } + /** * @see org.alfresco.repo.action.executer.ActionExecuterAbstractBase#executeImpl(org.alfresco.service.cmr.action.Action, org.alfresco.service.cmr.repository.NodeRef) */ @Override protected void executeImpl(Action action, NodeRef actionedUponNodeRef) { + // Check if thumbnailing is generally disabled + if (!generateThumbnails) + { + if (logger.isDebugEnabled()) + { + logger.debug("Thumbnail transformations are not enabled"); + } + return; + } + if (this.nodeService.exists(actionedUponNodeRef) == true) { // Get the thumbnail Name @@ -107,15 +144,31 @@ public class CreateThumbnailActionExecuter extends ActionExecuterAbstractBase // If there isn't a currently active transformer for this, log and skip Serializable contentProp = nodeService.getProperty(actionedUponNodeRef, contentProperty); - if(contentProp != null && contentProp instanceof ContentData) + if (contentProp == null) { - String mimetype = ((ContentData)contentProp).getMimetype(); - if (!registry.isThumbnailDefinitionAvailable(mimetype, details)) + logger.info("Creation of thumbnail, null content for " + details.getName()); + return; + } + + if(contentProp instanceof ContentData) + { + ContentData content = (ContentData)contentProp; + String mimetype = content.getMimetype(); + if (!registry.isThumbnailDefinitionAvailable(content.getContentUrl(), mimetype, content.getSize(), details)) { logger.info("Unable to create thumbnail '" + details.getName() + "' for " + mimetype + " as no transformer is currently available"); return; } + if (mimetypeMaxSourceSizeKBytes != null) + { + Long maxSourceSizeKBytes = mimetypeMaxSourceSizeKBytes.get(mimetype); + if (maxSourceSizeKBytes != null && maxSourceSizeKBytes > 0 && maxSourceSizeKBytes <= (content.getSize()/1024L)) + { + logger.info("Creation of " + details.getName()+ " thumbnail from '" + mimetype + "' , content is too big ("+(content.getSize()/1024L)+"K >= "+maxSourceSizeKBytes+"K)"); + return; //avoid transform + } + } } // Create the thumbnail @@ -144,5 +197,4 @@ public class CreateThumbnailActionExecuter extends ActionExecuterAbstractBase paramList.add(new ParameterDefinitionImpl(PARAM_THUMBANIL_NAME, DataTypeDefinition.TEXT, true, getParamDisplayLabel(PARAM_THUMBANIL_NAME))); paramList.add(new ParameterDefinitionImpl(PARAM_CONTENT_PROPERTY, DataTypeDefinition.QNAME, false, getParamDisplayLabel(PARAM_CONTENT_PROPERTY))); } - } diff --git a/source/java/org/alfresco/repo/thumbnail/SimpleThumbnailer.java b/source/java/org/alfresco/repo/thumbnail/SimpleThumbnailer.java index 16f3a6e9e8..af39196908 100644 --- a/source/java/org/alfresco/repo/thumbnail/SimpleThumbnailer.java +++ b/source/java/org/alfresco/repo/thumbnail/SimpleThumbnailer.java @@ -157,7 +157,7 @@ public class SimpleThumbnailer extends TransactionListenerAdapter implements Serializable value = this.nodeService.getProperty(nodeRef, ContentModel.PROP_CONTENT); ContentData contentData = DefaultTypeConverter.INSTANCE.convert(ContentData.class, value); List thumbnailDefinitions = this.thumbnailService.getThumbnailRegistry() - .getThumnailDefintions(contentData.getMimetype()); + .getThumbnailDefinitions(contentData.getContentUrl(), contentData.getMimetype(), contentData.getSize()); for (final ThumbnailDefinition thumbnailDefinition : thumbnailDefinitions) { final NodeRef existingThumbnail = this.thumbnailService.getThumbnailByName(nodeRef, diff --git a/source/java/org/alfresco/repo/thumbnail/ThumbnailRegistry.java b/source/java/org/alfresco/repo/thumbnail/ThumbnailRegistry.java index 6a2abceb7f..71018cb1bb 100644 --- a/source/java/org/alfresco/repo/thumbnail/ThumbnailRegistry.java +++ b/source/java/org/alfresco/repo/thumbnail/ThumbnailRegistry.java @@ -24,6 +24,7 @@ import java.util.List; import java.util.Map; import org.alfresco.repo.content.transform.ContentTransformer; +import org.alfresco.repo.content.transform.TransformerDebug; import org.alfresco.repo.security.authentication.AuthenticationUtil; import org.alfresco.repo.security.authentication.AuthenticationUtil.RunAsWork; import org.alfresco.repo.transaction.RetryingTransactionHelper; @@ -190,7 +191,15 @@ public class ThumbnailRegistry implements ApplicationContextAware, ApplicationLi return new ArrayList(this.thumbnailDefinitions.values()); } + /** + * @deprecated use overloaded version with sourceSize parameter. + */ public List getThumbnailDefinitions(String mimetype) + { + return getThumbnailDefinitions(null, mimetype, -1); + } + + public List getThumbnailDefinitions(String sourceUrl, String mimetype, long sourceSize) { List result = this.mimetypeMap.get(mimetype); @@ -201,7 +210,7 @@ public class ThumbnailRegistry implements ApplicationContextAware, ApplicationLi for (ThumbnailDefinition thumbnailDefinition : this.thumbnailDefinitions.values()) { - if (isThumbnailDefinitionAvailable(mimetype, thumbnailDefinition)) + if (isThumbnailDefinitionAvailable(sourceUrl, mimetype, sourceSize, thumbnailDefinition)) { result.add(thumbnailDefinition); foundAtLeastOneTransformer = true; @@ -243,17 +252,28 @@ public class ThumbnailRegistry implements ApplicationContextAware, ApplicationLi * Checks to see if at this moment in time, the specified {@link ThumbnailDefinition} * is able to thumbnail the source mimetype. Typically used with Thumbnail Definitions * retrieved by name, and/or when dealing with transient {@link ContentTransformer}s. - * @param thumbnailDefinition The {@link ThumbnailDefinition} to check for + * @param sourceUrl The URL of the source (optional) * @param sourceMimeType The source mimetype + * @param sourceSize the size (in bytes) of the source. Use -1 if unknown. + * @param thumbnailDefinition The {@link ThumbnailDefinition} to check for */ - public boolean isThumbnailDefinitionAvailable(String sourceMimeType, ThumbnailDefinition thumbnailDefinition) + public boolean isThumbnailDefinitionAvailable(String sourceUrl, String sourceMimeType, long sourceSize, ThumbnailDefinition thumbnailDefinition) { - return this.contentService.getTransformer( - sourceMimeType, - thumbnailDefinition.getMimetype(), - thumbnailDefinition.getTransformationOptions() - ) != null - ; + // Log the following getTransform() as trace so we can see the wood for the trees + boolean orig = TransformerDebug.setDebugOutput(false); + try + { + return this.contentService.getTransformer( + sourceUrl, + sourceMimeType, + sourceSize, + thumbnailDefinition.getMimetype(), thumbnailDefinition.getTransformationOptions() + ) != null; + } + finally + { + TransformerDebug.setDebugOutput(orig); + } } /** diff --git a/source/java/org/alfresco/repo/thumbnail/ThumbnailRenditionConvertor.java b/source/java/org/alfresco/repo/thumbnail/ThumbnailRenditionConvertor.java index 9aa85dea5a..cf25c4233d 100644 --- a/source/java/org/alfresco/repo/thumbnail/ThumbnailRenditionConvertor.java +++ b/source/java/org/alfresco/repo/thumbnail/ThumbnailRenditionConvertor.java @@ -1,5 +1,5 @@ /* - * Copyright (C) 2005-2010 Alfresco Software Limited. + * Copyright (C) 2005-2011 Alfresco Software Limited. * * This file is part of Alfresco * @@ -27,6 +27,7 @@ import org.alfresco.repo.content.transform.magick.ImageResizeOptions; import org.alfresco.repo.content.transform.magick.ImageTransformationOptions; import org.alfresco.repo.content.transform.swf.SWFTransformationOptions; import org.alfresco.repo.rendition.executer.AbstractRenderingEngine; +import org.alfresco.repo.rendition.executer.AbstractTransformationRenderingEngine; import org.alfresco.repo.rendition.executer.ImageRenderingEngine; import org.alfresco.repo.rendition.executer.ReformatRenderingEngine; import org.alfresco.service.cmr.rendition.RenditionDefinition; @@ -144,6 +145,13 @@ public class ThumbnailRenditionConvertor // putParameterIfNotNull(ImageRenderingEngine.PARAM_ASSOC_NAME, assocDetails.getAssociationName(), parameters); // putParameterIfNotNull(ImageRenderingEngine.PARAM_ASSOC_TYPE, assocDetails.getAssociationType(), parameters); + putParameterIfNotNull(AbstractTransformationRenderingEngine.PARAM_TIMEOUT_MS, transformationOptions.getTimeoutMs(), parameters); + putParameterIfNotNull(AbstractTransformationRenderingEngine.PARAM_READ_LIMIT_TIME_MS, transformationOptions.getReadLimitTimeMs(), parameters); + putParameterIfNotNull(AbstractTransformationRenderingEngine.PARAM_MAX_SOURCE_SIZE_K_BYTES, transformationOptions.getMaxSourceSizeKBytes(), parameters); + putParameterIfNotNull(AbstractTransformationRenderingEngine.PARAM_READ_LIMIT_K_BYTES, transformationOptions.getReadLimitKBytes(), parameters); + putParameterIfNotNull(AbstractTransformationRenderingEngine.PARAM_MAX_PAGES, transformationOptions.getMaxPages(), parameters); + putParameterIfNotNull(AbstractTransformationRenderingEngine.PARAM_PAGE_LIMIT, transformationOptions.getPageLimit(), parameters); + if (transformationOptions instanceof SWFTransformationOptions) { SWFTransformationOptions swfTransformationOptions = (SWFTransformationOptions)transformationOptions; diff --git a/source/java/org/alfresco/repo/thumbnail/ThumbnailServiceImplTest.java b/source/java/org/alfresco/repo/thumbnail/ThumbnailServiceImplTest.java index 7f6e27af93..85db20e12e 100644 --- a/source/java/org/alfresco/repo/thumbnail/ThumbnailServiceImplTest.java +++ b/source/java/org/alfresco/repo/thumbnail/ThumbnailServiceImplTest.java @@ -109,7 +109,7 @@ public class ThumbnailServiceImplTest extends BaseAlfrescoSpringTest // Check that it is working ImageTransformationOptions imageTransformationOptions = new ImageTransformationOptions(); - if (!transformer.isTransformable(MimetypeMap.MIMETYPE_IMAGE_JPEG, MimetypeMap.MIMETYPE_IMAGE_JPEG, + if (!transformer.isTransformable(MimetypeMap.MIMETYPE_IMAGE_JPEG, -1, MimetypeMap.MIMETYPE_IMAGE_JPEG, imageTransformationOptions)) { fail("Image transformer is not working. Please check your image conversion command setup."); @@ -542,8 +542,8 @@ public class ThumbnailServiceImplTest extends BaseAlfrescoSpringTest NodeRef nodeRef = createOriginalContent(this.folder, MimetypeMap.MIMETYPE_HTML); ThumbnailDefinition def = this.thumbnailService.getThumbnailRegistry().getThumbnailDefinition("medium"); - ContentTransformer transformer = this.contentService.getTransformer(MimetypeMap.MIMETYPE_HTML, def - .getMimetype(), def.getTransformationOptions()); + ContentTransformer transformer = this.contentService.getTransformer(null, MimetypeMap.MIMETYPE_HTML, -1, def + .getMimetype(), def.getTransformationOptions()); if (transformer != null) { NodeRef thumb = this.thumbnailService.createThumbnail(nodeRef, ContentModel.PROP_CONTENT, @@ -650,7 +650,7 @@ public class ThumbnailServiceImplTest extends BaseAlfrescoSpringTest public void testRegistry() { ThumbnailRegistry thumbnailRegistry = this.thumbnailService.getThumbnailRegistry(); - List defs = thumbnailRegistry.getThumbnailDefinitions(MimetypeMap.MIMETYPE_HTML); + List defs = thumbnailRegistry.getThumbnailDefinitions(null, MimetypeMap.MIMETYPE_HTML, -1); System.out.println("Definitions ..."); for (ThumbnailDefinition def : defs) { diff --git a/source/java/org/alfresco/repo/thumbnail/UpdateThumbnailActionExecuter.java b/source/java/org/alfresco/repo/thumbnail/UpdateThumbnailActionExecuter.java index 3ae49eaeec..23d57bb828 100644 --- a/source/java/org/alfresco/repo/thumbnail/UpdateThumbnailActionExecuter.java +++ b/source/java/org/alfresco/repo/thumbnail/UpdateThumbnailActionExecuter.java @@ -1,5 +1,5 @@ /* - * Copyright (C) 2005-2010 Alfresco Software Limited. + * Copyright (C) 2005-2011 Alfresco Software Limited. * * This file is part of Alfresco * @@ -18,6 +18,8 @@ */ package org.alfresco.repo.thumbnail; +import java.io.Serializable; +import java.util.HashMap; import java.util.List; import org.alfresco.error.AlfrescoRuntimeException; @@ -29,6 +31,7 @@ import org.alfresco.service.cmr.action.ParameterDefinition; import org.alfresco.service.cmr.dictionary.DataTypeDefinition; import org.alfresco.service.cmr.rendition.RenditionService; import org.alfresco.service.cmr.repository.ChildAssociationRef; +import org.alfresco.service.cmr.repository.ContentData; import org.alfresco.service.cmr.repository.NodeRef; import org.alfresco.service.cmr.repository.NodeService; import org.alfresco.service.cmr.thumbnail.ThumbnailService; @@ -43,6 +46,7 @@ import org.apache.commons.logging.LogFactory; * * @author Roy Wetherall * @author Neil McErlean + * @author Ph Dubois (optional thumbnail creation by mimetype and in general) */ public class UpdateThumbnailActionExecuter extends ActionExecuterAbstractBase { @@ -58,6 +62,12 @@ public class UpdateThumbnailActionExecuter extends ActionExecuterAbstractBase /** Node Service */ private NodeService nodeService; + /** Property turns on and off all thumbnail creation */ + private boolean generateThumbnails = true; + + // Size limitations indexed by mime type for thumbnail creation + private HashMap mimetypeMaxSourceSizeKBytes; + /** Action name and parameters */ public static final String NAME = "update-thumbnail"; public static final String PARAM_CONTENT_PROPERTY = "content-property"; @@ -93,12 +103,40 @@ public class UpdateThumbnailActionExecuter extends ActionExecuterAbstractBase this.nodeService = nodeService; } + /** + * Set the maximum size for each mimetype above which thumbnails are not created. + * @param mimetypeMaxSourceSizeKBytes map of mimetypes to max source sizes. + */ + public void setMimetypeMaxSourceSizeKBytes(HashMap mimetypeMaxSourceSizeKBytes) + { + this.mimetypeMaxSourceSizeKBytes = mimetypeMaxSourceSizeKBytes; + } + + /** + * Enable thumbnail creation at all regardless of mimetype. + * @param generateThumbnails a {@code false} value turns off all thumbnail creation. + */ + public void setGenerateThumbnails(boolean generateThumbnails) + { + this.generateThumbnails = generateThumbnails; + } + /** * @see org.alfresco.repo.action.executer.ActionExecuterAbstractBase#executeImpl(org.alfresco.service.cmr.action.Action, org.alfresco.service.cmr.repository.NodeRef) */ @Override protected void executeImpl(Action action, NodeRef actionedUponNodeRef) { + // Check if thumbnailing is generally disabled + if (!generateThumbnails) + { + if (logger.isDebugEnabled()) + { + logger.debug("Thumbnail transformations are not enabled"); + } + return; + } + // Get the thumbnail NodeRef thumbnailNodeRef = (NodeRef)action.getParameterValue(PARAM_THUMBNAIL_NODE); if (thumbnailNodeRef == null) @@ -128,6 +166,26 @@ public class UpdateThumbnailActionExecuter extends ActionExecuterAbstractBase contentProperty = ContentModel.PROP_CONTENT; } + Serializable contentProp = nodeService.getProperty(actionedUponNodeRef, contentProperty); + if (contentProp == null) + { + logger.info("Creation of thumbnail, null content for " + details.getName()); + return; + } + + if(contentProp instanceof ContentData) + { + ContentData content = (ContentData)contentProp; + if (mimetypeMaxSourceSizeKBytes != null) + { + Long maxSourceSizeKBytes = mimetypeMaxSourceSizeKBytes.get(content.getMimetype()); + if (maxSourceSizeKBytes != null && maxSourceSizeKBytes < (content.getSize()/1024L)) + { + logger.info("Creation of thumbnail, '" + details.getName() + " , content too big ("+maxSourceSizeKBytes+"K)"); + return; //avoid transform + } + } + } // Create the thumbnail this.thumbnailService.updateThumbnail(thumbnailNodeRef, details.getTransformationOptions()); } diff --git a/source/java/org/alfresco/service/cmr/repository/ContentService.java b/source/java/org/alfresco/service/cmr/repository/ContentService.java index 03272e1e00..dc1388a9d3 100644 --- a/source/java/org/alfresco/service/cmr/repository/ContentService.java +++ b/source/java/org/alfresco/service/cmr/repository/ContentService.java @@ -231,17 +231,24 @@ public interface ContentService * The transformation options provide a finer grain way of discovering the correct transformer, * since the values and type of the options provided are considered by the transformer when * deciding whether it can satisfy the transformation request. - * + * @param sourceUrl TODO * @param sourceMimetype the source mimetype + * @param sourceSize the source size (bytes). Ignored if negative. * @param targetMimetype the target mimetype * @param options the transformation options + * * @return ContentTransformer a transformer that can be used, or null if one was not available * * @see ContentAccessor#getMimetype() */ - @Auditable(parameters = {"sourceMimetype", "targetMimetype", "options"}) - public ContentTransformer getTransformer(String sourceMimetype, String targetMimetype, TransformationOptions options); + @Auditable(parameters = {"sourceMimetype", "sourceSize", "targetMimetype", "options"}) + public ContentTransformer getTransformer(String sourceUrl, String sourceMimetype, long sourceSize, String targetMimetype, TransformationOptions options); + /** + * @deprecated use overloaded method with sourceSize parameter. + */ + public ContentTransformer getTransformer(String sourceMimetype, String targetMimetype, TransformationOptions options); + /** * Fetch all the transformers that are capable of transforming the content in the * given source mimetype to the given target mimetype with the provided transformation @@ -256,6 +263,7 @@ public interface ContentService * as well as their current behaviour. For these reasons, this list should not be cached. * * @param sourceMimetype the source mimetype + * @param sourceSize the source size (bytes). Ignored if negative. * @param targetMimetype the target mimetype * @param options the transformation options * @return ContentTransformers a List of the transformers that can be used, or the empty list if none were available @@ -263,9 +271,14 @@ public interface ContentService * @since 3.5 * @see ContentAccessor#getMimetype() */ - @Auditable(parameters = {"sourceMimetype", "targetMimetype", "options"}) - public List getActiveTransformers(String sourceMimetype, String targetMimetype, TransformationOptions options); + @Auditable(parameters = {"sourceMimetype", "sourceSize", "targetMimetype", "options"}) + public List getActiveTransformers(String sourceMimetype, long sourceSize, String targetMimetype, TransformationOptions options); + /** + * @deprecated use overloaded method with sourceSize parameter. + */ + public List getActiveTransformers(String sourceMimetype, String targetMimetype, TransformationOptions options); + /** * Fetch the transformer that is capable of transforming image content. * diff --git a/source/java/org/alfresco/service/cmr/repository/TransformationOptionLimits.java b/source/java/org/alfresco/service/cmr/repository/TransformationOptionLimits.java new file mode 100644 index 0000000000..53a18c2de5 --- /dev/null +++ b/source/java/org/alfresco/service/cmr/repository/TransformationOptionLimits.java @@ -0,0 +1,257 @@ +/* + * Copyright (C) 2005-2011 Alfresco Software Limited. + * + * This file is part of Alfresco + * + * Alfresco is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * Alfresco is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with Alfresco. If not, see . + */ +package org.alfresco.service.cmr.repository; + +import java.util.HashMap; +import java.util.Map; + +import org.alfresco.util.EqualsHelper; + +/** + * Represents maximum source values (that result in exceptions if exceeded) or + * limits on source values (that result in EOF (End Of File) being returned + * early). Options exist for elapse time, KBytes read or number of pages read. + * + * @author Alan Davis + */ +public class TransformationOptionLimits +{ + public static final String OPT_TIMEOUT_MS = "timeoutMs"; + public static final String OPT_READ_LIMIT_TIME_MS = "readLimitTimeMs"; + + public static final String OPT_MAX_SOURCE_SIZE_K_BYTES = "maxSourceSizeKBytes"; + public static final String OPT_READ_LIMIT_K_BYTES = "readLimitKBytes"; + + public static final String OPT_MAX_PAGES = "maxPages"; + public static final String OPT_PAGE_LIMIT = "pageLimit"; + + public static final String TIME_MESSAGE = "Both timeoutMs and readLimitTimeMs should not be set."; + public static final String KBYTES_MESSAGE = "Both maxSourceSizeKBytes and readLimitKBytes should not be set."; + public static final String PAGES_MESSAGE = "Both maxPages and pageLimit should not be set."; + + private TransformationOptionPair time = new TransformationOptionPair(); + private TransformationOptionPair kbytes = new TransformationOptionPair(); + private TransformationOptionPair pages = new TransformationOptionPair(); + + public TransformationOptionLimits() + { + time = new TransformationOptionPair(); + kbytes = new TransformationOptionPair(); + pages = new TransformationOptionPair(); + } + + private TransformationOptionLimits(TransformationOptionLimits a, TransformationOptionLimits b) + { + time = a.time.combine(b.time); + kbytes = a.kbytes.combine(b.kbytes); + pages = a.pages.combine(b.pages); + } + + // --------------- Time --------------- + public TransformationOptionPair getTimePair() + { + return time; + } + + public long getTimeoutMs() + { + return time.getMax(); + } + + public void setTimeoutMs(long timeoutMs) + { + time.setMax(timeoutMs, TIME_MESSAGE); + } + + public long getReadLimitTimeMs() + { + return time.getLimit(); + } + + public void setReadLimitTimeMs(long readLimitTimeMs) + { + time.setLimit(readLimitTimeMs, TIME_MESSAGE); + } + + // --------------- KBytes --------------- + public TransformationOptionPair getKBytesPair() + { + return kbytes; + } + + public long getMaxSourceSizeKBytes() + { + return kbytes.getMax(); + } + + public void setMaxSourceSizeKBytes(long maxSourceSizeKBytes) + { + kbytes.setMax(maxSourceSizeKBytes, KBYTES_MESSAGE); + } + + public long getReadLimitKBytes() + { + return kbytes.getLimit(); + } + + public void setReadLimitKBytes(long readLimitKBytes) + { + kbytes.setLimit(readLimitKBytes, KBYTES_MESSAGE); + } + + // --------------- Pages --------------- + public TransformationOptionPair getPagesPair() + { + return pages; + } + + public int getMaxPages() + { + return (int)pages.getMax(); + } + + public void setMaxPages(int maxPages) + { + pages.setMax(maxPages, PAGES_MESSAGE); + } + + public int getPageLimit() + { + return (int)pages.getLimit(); + } + + public void setPageLimit(int pageLimit) + { + pages.setLimit(pageLimit, PAGES_MESSAGE); + } + + // --------------- Map --------------- + public Map toMap(Map optionsMap) + { + time.toMap(optionsMap, OPT_TIMEOUT_MS, OPT_READ_LIMIT_TIME_MS); + kbytes.toMap(optionsMap, OPT_MAX_SOURCE_SIZE_K_BYTES, OPT_READ_LIMIT_K_BYTES); + pages.toMap(optionsMap, OPT_MAX_PAGES, OPT_PAGE_LIMIT); + return optionsMap; + } + + public static Map removeFromMap(Map optionsMap) + { + optionsMap.remove(OPT_TIMEOUT_MS); + optionsMap.remove(OPT_READ_LIMIT_TIME_MS); + optionsMap.remove(OPT_MAX_SOURCE_SIZE_K_BYTES); + optionsMap.remove(OPT_READ_LIMIT_K_BYTES); + optionsMap.remove(OPT_MAX_PAGES); + optionsMap.remove(OPT_PAGE_LIMIT); + return optionsMap; + } + + public void set(Map optionsMap) + { + time.set(optionsMap, OPT_TIMEOUT_MS, OPT_READ_LIMIT_TIME_MS, TIME_MESSAGE); + kbytes.set(optionsMap, OPT_MAX_SOURCE_SIZE_K_BYTES, OPT_READ_LIMIT_K_BYTES, KBYTES_MESSAGE); + pages.set(optionsMap, OPT_MAX_PAGES, OPT_PAGE_LIMIT, PAGES_MESSAGE); + } + + public String toString() + { + return toMap(new HashMap()).toString(); + } + + /** + * Returns a TransformationOptionLimits that has getter methods that combine the + * the values from the getter methods of this and the supplied TransformationOptionLimits. + */ + public TransformationOptionLimits combine(final TransformationOptionLimits that) + { + return new TransformationOptionLimits(this, that) + { + @Override + public void setTimeoutMs(long timeoutMs) + { + throw new UnsupportedOperationException(); + } + + @Override + public void setReadLimitTimeMs(long readLimitTimeMs) + { + throw new UnsupportedOperationException(); + } + + @Override + public void setMaxSourceSizeKBytes(long maxSourceSizeKBytes) + { + throw new UnsupportedOperationException(); + } + + @Override + public void setReadLimitKBytes(long readLimitKBytes) + { + throw new UnsupportedOperationException(); + } + + @Override + public void setMaxPages(int maxPages) + { + throw new UnsupportedOperationException(); + } + + @Override + public void setPageLimit(int pageLimit) + { + throw new UnsupportedOperationException(); + } + + @Override + public void set(Map optionsMap) + { + throw new UnsupportedOperationException(); + } + }; + } + + @Override + public int hashCode() + { + int hashCode = 37 * time.hashCode(); + hashCode += 37 * kbytes.hashCode(); + hashCode += 37 * pages.hashCode(); + return hashCode; + } + + @Override + public boolean equals(Object obj) + { + if (this == obj) + { + return true; + } + else if (obj instanceof TransformationOptionLimits) + { + TransformationOptionLimits that = (TransformationOptionLimits) obj; + return + EqualsHelper.nullSafeEquals(time, that.time) && + EqualsHelper.nullSafeEquals(kbytes, that.kbytes) && + EqualsHelper.nullSafeEquals(pages, that.pages); + } + else + { + return false; + } + } +} diff --git a/source/java/org/alfresco/service/cmr/repository/TransformationOptionLimitsTest.java b/source/java/org/alfresco/service/cmr/repository/TransformationOptionLimitsTest.java new file mode 100644 index 0000000000..9dbc520b10 --- /dev/null +++ b/source/java/org/alfresco/service/cmr/repository/TransformationOptionLimitsTest.java @@ -0,0 +1,316 @@ +/* + * Copyright (C) 2005-2011 Alfresco Software Limited. + * + * This file is part of Alfresco + * + * Alfresco is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * Alfresco is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with Alfresco. If not, see . + */ +package org.alfresco.service.cmr.repository; + + +import static org.junit.Assert.assertEquals; + +import java.util.HashMap; +import java.util.Map; + +import org.junit.Before; +import org.junit.Test; + +/** + * Test TransformationOptionLimits + */ +public class TransformationOptionLimitsTest +{ + private TransformationOptionLimits limits; + + @Before + public void setUp() throws Exception + { + limits = new TransformationOptionLimits(); + } + + @Test + public void testTimeoutMs() throws Exception + { + long value = 1234; + limits.setTimeoutMs(value); + long actual = limits.getTimeoutMs(); + assertEquals("Getter did not return set value", value, actual); + } + + @Test + public void testReadLimitTimeMs() throws Exception + { + long value = 1234; + limits.setReadLimitTimeMs(value); + long actual = limits.getReadLimitTimeMs(); + assertEquals("Getter did not return set value", value, actual); + } + + @Test + public void testMaxSourceSizeKBytes() throws Exception + { + long value = 1234; + limits.setMaxSourceSizeKBytes(value); + long actual = limits.getMaxSourceSizeKBytes(); + assertEquals("Getter did not return set value", value, actual); + } + + @Test + public void testReadLimitKBytes() throws Exception + { + long value = 1234; + limits.setReadLimitKBytes(value); + long actual = limits.getReadLimitKBytes(); + assertEquals("Getter did not return set value", value, actual); + } + + @Test + public void testMaxPages() throws Exception + { + int value = 1234; + limits.setMaxPages(value); + int actual = limits.getMaxPages(); + assertEquals("Getter did not return set value", value, actual); + } + + @Test + public void testPageLimit() throws Exception + { + int value = 1234; + limits.setPageLimit(value); + int actual = limits.getPageLimit(); + assertEquals("Getter did not return set value", value, actual); + } + + @Test + public void testTimeException() throws Exception + { + String message = null; + limits.setTimeoutMs(1); + try + { + limits.setReadLimitTimeMs(1); + } + catch (IllegalArgumentException e) + { + message = e.getMessage(); + } + assertEquals("Wrong exception message", TransformationOptionLimits.TIME_MESSAGE, message); + } + + @Test + public void testKBytesException() throws Exception + { + String message = null; + limits.setMaxSourceSizeKBytes(1); + try + { + limits.setReadLimitKBytes(1); + } + catch (IllegalArgumentException e) + { + message = e.getMessage(); + } + assertEquals("Wrong exception message", TransformationOptionLimits.KBYTES_MESSAGE, message); + } + + @Test + public void testPageException() throws Exception + { + String message = null; + limits.setPageLimit(1); + try + { + limits.setMaxPages(1); + } + catch (IllegalArgumentException e) + { + message = e.getMessage(); + } + assertEquals("Wrong exception message", TransformationOptionLimits.PAGES_MESSAGE, message); + } + + @Test + public void testMapMax() throws Exception + { + limits.setTimeoutMs(123); + limits.setMaxSourceSizeKBytes(456); + limits.setMaxPages(789); + + Map optionsMap = new HashMap(); + limits.toMap(optionsMap); + + TransformationOptionLimits actual = new TransformationOptionLimits(); + actual.set(optionsMap); + + assertEquals("Did not match original values", limits, actual); + } + + @Test + public void testMapLimit() throws Exception + { + limits.setReadLimitTimeMs(123); + limits.setReadLimitKBytes(456); + limits.setPageLimit(789); + + Map optionsMap = new HashMap(); + limits.toMap(optionsMap); + + TransformationOptionLimits actual = new TransformationOptionLimits(); + actual.set(optionsMap); + + assertEquals("Did not match original values", limits, actual); + } + + @Test + public void testTimePair() throws Exception + { + int value = 1234; + limits.setTimeoutMs(value); + + long actual = limits.getTimePair().getMax(); + + assertEquals("Returned TransformationOptionPair did not contain set value", value, actual); + } + + + @Test + public void testKBytesPair() throws Exception + { + int value = 1234; + limits.setMaxSourceSizeKBytes(value); + + long actual = limits.getKBytesPair().getMax(); + + assertEquals("Returned TransformationOptionPair did not contain set value", value, actual); + } + + + @Test + public void testPagePair() throws Exception + { + int value = 1234; + limits.setMaxPages(value); + + long actual = limits.getPagesPair().getMax(); + + assertEquals("Returned TransformationOptionPair did not contain set value", value, actual); + } + + @Test + public void testCombineOrder() throws Exception + { + limits.setReadLimitTimeMs(123); + limits.setReadLimitKBytes(45); + limits.setMaxPages(789); + + TransformationOptionLimits second = new TransformationOptionLimits(); + second.setReadLimitTimeMs(12); + second.setReadLimitKBytes(456); + second.setMaxPages(789); + + TransformationOptionLimits combined = limits.combine(second); + TransformationOptionLimits combinedOtherWay = second.combine(limits); + assertEquals("The combine order should not matter", combined, combinedOtherWay); + } + + @Test + public void testCombine() throws Exception + { + limits.setReadLimitTimeMs(123); // limit > + limits.setReadLimitKBytes(45); // limit < + limits.setMaxPages(789); // max = + + TransformationOptionLimits second = new TransformationOptionLimits(); + second.setTimeoutMs(12); // max < + second.setMaxSourceSizeKBytes(456); // max > + second.setMaxPages(789); // max = + + TransformationOptionLimits combined = limits.combine(second); + + assertEquals("Expected the lower value", 12, combined.getTimeoutMs()); // max < + assertEquals("Expected the lower value", 45, combined.getReadLimitKBytes()); // limit < + assertEquals("Expected the lower value", 789, combined.getMaxPages()); // max = + } + + @Test + public void testCombineDynamic() throws Exception + { + limits.setReadLimitTimeMs(123); + limits.setReadLimitKBytes(45); + limits.setMaxPages(789); + + TransformationOptionLimits second = new TransformationOptionLimits(); + second.setReadLimitTimeMs(12); + second.setReadLimitKBytes(456); + second.setMaxPages(789); + + TransformationOptionLimits combined = limits.combine(second); + + // Test dynamic change of value + limits.setReadLimitKBytes(4560); + assertEquals("Expected the lower value", 456, combined.getReadLimitKBytes()); + } + + @Test(expected=UnsupportedOperationException.class) + public void testCombineSetTimeoutMs() throws Exception + { + TransformationOptionLimits combined = limits.combine(limits); // may combine with itself + combined.setTimeoutMs(1); + } + + @Test(expected=UnsupportedOperationException.class) + public void testCombineSetReadLimitTimeMs() throws Exception + { + TransformationOptionLimits combined = limits.combine(limits); // may combine with itself + combined.setReadLimitTimeMs(1); + } + + @Test(expected=UnsupportedOperationException.class) + public void testCombineSetMaxSourceSizeKBytes() throws Exception + { + TransformationOptionLimits combined = limits.combine(limits); // may combine with itself + combined.setMaxSourceSizeKBytes(1); + } + + @Test(expected=UnsupportedOperationException.class) + public void testCombineSetReadLimitKBytes() throws Exception + { + TransformationOptionLimits combined = limits.combine(limits); // may combine with itself + combined.setReadLimitKBytes(1); + } + + @Test(expected=UnsupportedOperationException.class) + public void testCombineSetMaxPages() throws Exception + { + TransformationOptionLimits combined = limits.combine(limits); // may combine with itself + combined.setMaxPages(1); + } + + @Test(expected=UnsupportedOperationException.class) + public void testCombineSetPageLimit() throws Exception + { + TransformationOptionLimits combined = limits.combine(limits); // may combine with itself + combined.setPageLimit(1); + } + + @Test(expected=UnsupportedOperationException.class) + public void testCombineSetMap() throws Exception + { + TransformationOptionLimits combined = limits.combine(limits); // may combine with itself + combined.set(null); + } +} + diff --git a/source/java/org/alfresco/service/cmr/repository/TransformationOptionPair.java b/source/java/org/alfresco/service/cmr/repository/TransformationOptionPair.java new file mode 100644 index 0000000000..f355c888ef --- /dev/null +++ b/source/java/org/alfresco/service/cmr/repository/TransformationOptionPair.java @@ -0,0 +1,244 @@ +/* + * Copyright (C) 2005-2011 Alfresco Software Limited. + * + * This file is part of Alfresco + * + * Alfresco is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * Alfresco is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with Alfresco. If not, see . + */ +package org.alfresco.service.cmr.repository; + +import java.io.IOException; +import java.util.Map; + +import org.alfresco.repo.content.transform.TransformerDebug; +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; + +/** + * A pair of transformation options that specify + * A) a max value over which the source is not read (throws an Exception) or + * B) a limit over which no more of the source is read (returns EOF) + * + * Each pair represents a values such as an elapse time, KBytes read or number of pages read. + * It is only meaningful for a either the max or limit value to be set. + * + * There is one pair of values for each transformer and another pair passed in via the + * options parameter for each individual transformation. The later is for specific types of + * transformation, such as thumbnail generation. When this occurs values are combined, by + * using the lowest of the four values. + * + * @author Alan Davis + */ +public class TransformationOptionPair +{ + /** + * Action to take place for a given pair of values. + */ + public enum Action + { + THROW_EXCEPTION + { + public void throwIOExceptionIfRequired(String message, TransformerDebug transformerDebug) throws IOException + { + throw transformerDebug.setCause(new IOException(message)); + } + }, + RETURN_EOF + { + public void throwIOExceptionIfRequired(String message, TransformerDebug transformerDebug) throws IOException + { + if (transformerDebug.isEnabled()) + { + transformerDebug.debug(message + " Returning EOF"); + } + } + }; + + public abstract void throwIOExceptionIfRequired(String message, TransformerDebug transformerDebug) throws IOException; + }; + + private long max = -1; + private long limit = -1; + + public long getMax() + { + return max; + } + + public void setMax(long max, String exceptionMessage) + { + if (max >= 0 && limit >= 0) + { + throw new IllegalArgumentException(exceptionMessage); + } + this.max = max; + } + + public long getLimit() + { + return limit; + } + + public void setLimit(long limit, String exceptionMessage) + { + if (max >= 0 && limit >= 0) + { + throw new IllegalArgumentException(exceptionMessage); + } + this.limit = limit; + } + + public long getValue() + { + return minSet(getMax(), getLimit()); + } + + public Action getAction() + { + return + (getMax() >= 0) ? Action.THROW_EXCEPTION : + (getLimit() >= 0) ? Action.RETURN_EOF + : null; + } + + /** + * Returns the lower of the two value supplied, ignoring values less than + * 0 unless both are less than zero. + */ + private long minSet(long value1, long value2) + { + if (value1 < 0) + { + return value2; + } + else if (value2 < 0) + { + return value1; + } + return Math.min(value1, value2); + } + + public Map toMap(Map optionsMap, String optMaxKey, String optLimitKey) + { + optionsMap.put(optMaxKey, getMax()); + optionsMap.put(optLimitKey, getLimit()); + return optionsMap; + } + + public void set(Map optionsMap, String optMaxKey, String optLimitKey, + String exceptionMessage) + { + long max = nvl((Long)optionsMap.get(optMaxKey)); + long limit = nvl((Long)optionsMap.get(optLimitKey)); + if (max >= 0 && limit >= 0) + { + throw new IllegalArgumentException(exceptionMessage); + } + if (max >= 0 || limit >= 0) + { + this.limit = limit; + this.max = max; + } + } + + private long nvl(Long l) + { + return l == null ? -1 : l; + } + + /** + * Returns a TransformationOptionPair that has getter methods that combine the + * the values from the getter methods of this and the supplied TransformationOptionPair. + */ + public TransformationOptionPair combine(final TransformationOptionPair that) + { + return new TransformationOptionPair() + { + /** + * Combines max values of this TransformationOptionPair and the supplied + * one to return the max to be used in a transformation. The limit + * value is discarded (-1 is returned) if the combined limit value is lower. + */ + @Override + public long getMax() + { + long max = minSet(TransformationOptionPair.this.getMax(), that.getMax()); + long limit = minSet(TransformationOptionPair.this.getLimit(), that.getLimit()); + + return (max >= 0 && (limit < 0 || limit >= max)) + ? max + : -1; + } + + @Override + public void setMax(long max, String exceptionMessage) + { + throw new UnsupportedOperationException(); + } + + /** + * Combines limit values of this TransformationOptionPair and the supplied + * one to return the limit to be used in a transformation. The limit + * value is discarded (-1 is returned) if the combined max value is lower. + */ + @Override + public long getLimit() + { + long max = minSet(TransformationOptionPair.this.getMax(), that.getMax()); + long limit = minSet(TransformationOptionPair.this.getLimit(), that.getLimit()); + + return (limit >= 0 && (max < 0 || max >= limit)) + ? limit + : -1; + } + + @Override + public void setLimit(long limit, String exceptionMessage) + { + throw new UnsupportedOperationException(); + } + + @Override + public void set(Map optionsMap, String optMaxKey, String optLimitKey, + String exceptionMessage) + { + throw new UnsupportedOperationException(); + } + }; + } + + @Override + public int hashCode() + { + return (int) ((max > 0) ? max : limit); + } + + @Override + public boolean equals(Object obj) + { + if (this == obj) + { + return true; + } + else if (obj instanceof TransformationOptionPair) + { + TransformationOptionPair that = (TransformationOptionPair) obj; + return max == that.max && limit == that.limit; + } + else + { + return false; + } + } +} diff --git a/source/java/org/alfresco/service/cmr/repository/TransformationOptionPairTest.java b/source/java/org/alfresco/service/cmr/repository/TransformationOptionPairTest.java new file mode 100644 index 0000000000..8a39cf0075 --- /dev/null +++ b/source/java/org/alfresco/service/cmr/repository/TransformationOptionPairTest.java @@ -0,0 +1,332 @@ +/* + * Copyright (C) 2005-2011 Alfresco Software Limited. + * + * This file is part of Alfresco + * + * Alfresco is free software: you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * Alfresco is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with Alfresco. If not, see . + */ +package org.alfresco.service.cmr.repository; + + +import java.util.HashMap; +import java.util.Map; + +import org.alfresco.service.cmr.repository.TransformationOptionPair.Action; +import org.junit.Before; +import org.junit.Test; + +import static org.junit.Assert.assertEquals; + +/** + * Test TransformationOptionPair + */ +public class TransformationOptionPairTest +{ + TransformationOptionPair pair; + + @Before + public void setUp() throws Exception + { + pair = new TransformationOptionPair(); + } + + @Test + public void testUnset() throws Exception + { + long value = -1; + long actual = pair.getMax(); + + assertEquals("Getter did not return unset value", value, actual); + assertEquals("getValue() did not return set value", value, pair.getValue()); + assertEquals("Expected action not returned", null, pair.getAction()); + } + + @Test + public void testMax() throws Exception + { + long value = 1234; + pair.setMax(value, null); + long actual = pair.getMax(); + + assertEquals("Getter did not return set value", value, actual); + assertEquals("getValue() did not return set value", value, pair.getValue()); + assertEquals("Expected action not returned", Action.THROW_EXCEPTION, pair.getAction()); + } + + @Test + public void testLimit() throws Exception + { + long value = 1234; + pair.setLimit(value, null); + long actual = pair.getLimit(); + assertEquals("Getter did not return set value", value, actual); + assertEquals("getValue() did not return set value", value, pair.getValue()); + assertEquals("Expected action not returned", Action.RETURN_EOF, pair.getAction()); + } + + @Test + public void testMaxAlreadySet() throws Exception + { + String message = "Oh no the other value is set"; + String actual = null; + pair.setMax(1234, null); + try + { + pair.setLimit(111, message); + } + catch (IllegalArgumentException e) + { + actual = e.getMessage(); + } + assertEquals("Expected an IllegalArgumentException message", message, actual); + } + + @Test + public void testLimitAlreadySet() throws Exception + { + String message = "Oh no the other value is set"; + String actual = null; + pair.setLimit(1234, null); + try + { + pair.setMax(111, message); + } + catch (IllegalArgumentException e) + { + actual = e.getMessage(); + } + assertEquals("Expected an IllegalArgumentException message", message, actual); + } + + @Test + public void testSetMaxMultipleTimes() throws Exception + { + long value = 1234; + pair.setMax(1, null); + pair.setMax(2, null); // Should be no exception + pair.setMax(value, null); + long actual = pair.getMax(); + assertEquals("Getter did not return set value", value, actual); + } + + @Test + public void testSetLimitMultipleTimes() throws Exception + { + long value = 1234; + pair.setLimit(1, null); + pair.setLimit(2, null); // Should be no exception + pair.setLimit(value, null); + long actual = pair.getLimit(); + assertEquals("Getter did not return set value", value, actual); + } + + @Test + public void testSetClearSet() throws Exception + { + // Test there is no exception if we clear the other value first + String message = "Oh no the other value is set"; + pair.setLimit(1, message); + pair.setLimit(-1, message); + pair.setMax(1, message); + pair.setMax(-1, message); + pair.setLimit(1, message); + } + + @Test + public void testMapMax() throws Exception + { + String maxKey = "Max"; + String limitKey = "Limit"; + String message = "Oh no the other value is set"; + pair.setMax(123, null); + + Map optionsMap = new HashMap(); + pair.toMap(optionsMap, maxKey, limitKey); + + TransformationOptionPair actual = new TransformationOptionPair(); + actual.set(optionsMap, maxKey, limitKey, message); + + assertEquals("Did not match original values", pair, actual); + } + + @Test + public void testMapLimit() throws Exception + { + String maxKey = "Max"; + String limitKey = "Limit"; + String message = "Oh no the other value is set"; + pair.setLimit(123, null); + + Map optionsMap = new HashMap(); + pair.toMap(optionsMap, maxKey, limitKey); + + TransformationOptionPair actual = new TransformationOptionPair(); + actual.set(optionsMap, maxKey, limitKey, message); + + assertEquals("Did not match original values", pair, actual); + } + + @Test + public void testMapBothSet() throws Exception + { + String maxKey = "Max"; + String limitKey = "Limit"; + String message = "Oh no the other value is set"; + pair.setLimit(123, null); + + Map optionsMap = new HashMap(); + pair.toMap(optionsMap, maxKey, limitKey); + optionsMap.put(maxKey, 456L); // Introduce error + + String actual = null; + TransformationOptionPair pair2 = new TransformationOptionPair(); + try + { + pair2.set(optionsMap, maxKey, limitKey, message); + } + catch (IllegalArgumentException e) + { + actual = e.getMessage(); + } + assertEquals("Expected an IllegalArgumentException message", message, actual); + } + + @Test + public void testMapNeitherSet() throws Exception + { + // Original value should not be changed if keys don't exist + long value = 1234; + String maxKey = "Max"; + String limitKey = "Limit"; + String message = "Oh no the other value is set"; + pair.setLimit(value, null); + + Map optionsMap = new HashMap(); + optionsMap.put("AnotherKey", 456L); + + pair.set(optionsMap, maxKey, limitKey, message); + long actual = pair.getLimit(); + assertEquals("Original value should not be changed", value, actual); + } + + @Test + public void testCombineOrder() throws Exception + { + TransformationOptionPair second = new TransformationOptionPair(); + + pair.setMax(123, null); + second.setMax(12, null); + TransformationOptionPair combined = pair.combine(second); + + TransformationOptionPair combinedOtherWay = second.combine(pair); + assertEquals("The combine order should not matter", combined, combinedOtherWay); + } + + @Test + public void testCombineMax() throws Exception + { + TransformationOptionPair second = new TransformationOptionPair(); + + pair.setMax(123, null); + second.setMax(12, null); + TransformationOptionPair combined = pair.combine(second); + + assertEquals("Expected the lower value", 12, combined.getValue()); + assertEquals("Expected the lower value", 12, combined.getMax()); + } + + @Test + public void testCombineLimit() throws Exception + { + TransformationOptionPair second = new TransformationOptionPair(); + + pair.setLimit(123, null); + second.setLimit(12, null); + TransformationOptionPair combined = pair.combine(second); + + assertEquals("Expected the lower value", 12, combined.getValue()); + assertEquals("Expected the lower value", 12, combined.getLimit()); + } + + @Test + public void testCombineMaxWins() throws Exception + { + // Try both max and limit values where max is lower + TransformationOptionPair second = new TransformationOptionPair(); + + pair.setLimit(123, null); + second.setMax(12, null); + TransformationOptionPair combined = pair.combine(second); + + assertEquals("Expected the lower value", 12, combined.getValue()); + assertEquals("Expected the lower value", 12, combined.getMax()); + assertEquals("Expected unset value", -1, combined.getLimit()); + } + + @Test + public void testCombineLimitWins() throws Exception + { + // Try both max and limit values where limit is lower + TransformationOptionPair second = new TransformationOptionPair(); + + pair.setMax(123, null); + second.setLimit(12, null); + TransformationOptionPair combined = pair.combine(second); + + assertEquals("Expected the lower value", 12, combined.getValue()); + assertEquals("Expected the lower value", 12, combined.getLimit()); + assertEquals("Expected unset value", -1, combined.getMax()); + } + + @Test + public void testCombineDynamicChange() throws Exception + { + TransformationOptionPair second = new TransformationOptionPair(); + + pair.setMax(123, null); + second.setMax(1234, null); + TransformationOptionPair combined = pair.combine(second); + + // Test dynamic changes of value + pair.setMax(45, null); + assertEquals("Expected the lower value", 45, combined.getMax()); + assertEquals("Expected an unset value", -1, combined.getLimit()); + + second.setMax(-1, null); + second.setLimit(10, null); + assertEquals("Expected an unset value", -1, combined.getMax()); + assertEquals("Expected the lower value", 10, combined.getLimit()); + } + + @Test(expected=UnsupportedOperationException.class) + public void testCombineSetMax() throws Exception + { + TransformationOptionPair combined = pair.combine(pair); // may combine with itself + combined.setMax(1, null); + } + + @Test(expected=UnsupportedOperationException.class) + public void testCombineSetLimit() throws Exception + { + TransformationOptionPair combined = pair.combine(pair); // may combine with itself + combined.setLimit(1, null); + } + + @Test(expected=UnsupportedOperationException.class) + public void testCombineSetMap() throws Exception + { + TransformationOptionPair combined = pair.combine(pair); // may combine with itself + combined.set(null, null, null, null); + } +} diff --git a/source/java/org/alfresco/service/cmr/repository/TransformationOptions.java b/source/java/org/alfresco/service/cmr/repository/TransformationOptions.java index c68f3a2500..845417ad50 100644 --- a/source/java/org/alfresco/service/cmr/repository/TransformationOptions.java +++ b/source/java/org/alfresco/service/cmr/repository/TransformationOptions.java @@ -1,5 +1,5 @@ /* - * Copyright (C) 2005-2010 Alfresco Software Limited. + * Copyright (C) 2005-2011 Alfresco Software Limited. * * This file is part of Alfresco * @@ -58,6 +58,9 @@ public class TransformationOptions /** The include embedded resources yes/no */ private Boolean includeEmbedded; + + /** Time, KBytes and page limits */ + private TransformationOptionLimits limits = new TransformationOptionLimits(); /** * Default construtor @@ -91,12 +94,22 @@ public class TransformationOptions * @param optionsMap options map */ public TransformationOptions(Map optionsMap) + { + set(optionsMap); + } + + /** + * Sets options from the supplied map. + * @param optionsMap + */ + public void set(Map optionsMap) { this.sourceNodeRef = (NodeRef)optionsMap.get(OPT_SOURCE_NODEREF); this.sourceContentProperty = (QName)optionsMap.get(OPT_SOURCE_CONTENT_PROPERTY); this.targetNodeRef = (NodeRef)optionsMap.get(OPT_TARGET_NODEREF); this.targetContentProperty = (QName)optionsMap.get(OPT_TARGET_CONTENT_PROPERTY); this.includeEmbedded = (Boolean)optionsMap.get(OPT_INCLUDE_EMBEDDED); + limits.set(optionsMap); } /** @@ -207,7 +220,161 @@ public class TransformationOptions return includeEmbedded; } - /** + // --------------- Time --------------- + + /** + * Gets the timeout (ms) on the InputStream after which an IOExecption is thrown + * to terminate very slow transformations or a subprocess is terminated (killed). + * @return timeoutMs in milliseconds. If less than or equal to zero (the default) + * there is no timeout. + */ + public long getTimeoutMs() + { + return limits.getTimeoutMs(); + } + + /** + * Sets a timeout (ms) on the InputStream after which an IOExecption is thrown + * to terminate very slow transformations or to terminate (kill) a subprocess. + * @param timeoutMs in milliseconds. If less than or equal to zero (the default) + * there is no timeout. + * If greater than zero the {@code readLimitTimeMs} must not be set. + */ + public void setTimeoutMs(long timeoutMs) + { + limits.setTimeoutMs(timeoutMs); + } + + /** + * Gets the limit in terms of the amount of data read (by time) to limit transformations where + * only the start of the content is needed. After this limit is reached the InputStream reports + * end of file. + * @return readLimitBytes if less than or equal to zero (the default) there is no limit. + */ + public long getReadLimitTimeMs() + { + return limits.getReadLimitTimeMs(); + } + + // --------------- KBytes --------------- + + /** + * Sets a limit in terms of the amount of data read (by time) to limit transformations where + * only the start of the content is needed. After this limit is reached the InputStream reports + * end of file. + * @param readLimitBytes if less than or equal to zero (the default) there is no limit. + * If greater than zero the {@code timeoutMs} must not be set. + */ + public void setReadLimitTimeMs(long readLimitTimeMs) + { + limits.setReadLimitTimeMs(readLimitTimeMs); + } + + /** + * Gets the maximum source content size, to skip transformations where + * the source is just too large to expect it to perform. If the source is larger + * the transformer indicates it is not available. + * @return maxSourceSizeKBytes if less than or equal to zero (the default) there is no limit. + */ + public long getMaxSourceSizeKBytes() + { + return limits.getMaxSourceSizeKBytes(); + } + + /** + * Sets a maximum source content size, to skip transformations where + * the source is just too large to expect it to perform. If the source is larger + * the transformer indicates it is not available. + * @param maxSourceSizeKBytes if less than or equal to zero (the default) there is no limit. + * If greater than zero the {@code readLimitKBytes} must not be set. + */ + public void setMaxSourceSizeKBytes(long maxSourceSizeKBytes) + { + limits.setMaxSourceSizeKBytes(maxSourceSizeKBytes); + } + + /** + * Gets the limit in terms of the about of data read to limit transformations where + * only the start of the content is needed. After this limit is reached the InputStream reports + * end of file. + * @return readLimitKBytes if less than or equal to zero (the default) no limit should be applied. + */ + public long getReadLimitKBytes() + { + return limits.getReadLimitKBytes(); + } + + /** + * Sets a limit in terms of the about of data read to limit transformations where + * only the start of the content is needed. After this limit is reached the InputStream reports + * end of file. + * @param readLimitKBytes if less than or equal to zero (the default) there is no limit. + * If greater than zero the {@code maxSourceSizeKBytes} must not be set. + */ + public void setReadLimitKBytes(long readLimitKBytes) + { + limits.setReadLimitKBytes(readLimitKBytes); + } + + // --------------- Pages --------------- + + /** + * Get the maximum number of pages read before an exception is thrown. + * @return If less than or equal to zero (the default) no limit should be applied. + */ + public int getMaxPages() + { + return limits.getMaxPages(); + } + + /** + * Set the number of pages read from the source before an exception is thrown. + * + * @param maxPages the number of pages to be read from the source. If less than or equal to zero + * (the default) no limit is applied. + */ + public void setMaxPages(int maxPages) + { + limits.setMaxPages(maxPages); + } + + /** + * Get the page limit before returning EOF. + * @return If less than or equal to zero (the default) no limit should be applied. + */ + public int getPageLimit() + { + return limits.getPageLimit(); + } + + /** + * Set the number of pages read from the source before returning EOF. + * + * @param pageLimit the number of pages to be read from the source. If less + * than or equal to zero (the default) no limit is applied. + */ + public void setPageLimit(int pageLimit) + { + limits.setPageLimit(pageLimit); + } + + /** + * Returns max and limit values for time, size and pages in a single operation. + */ + public TransformationOptionLimits getLimits() + { + return limits; + } + + /** + * Sets max and limit values for time, size and pages in a single operation. + */ + public void setLimits(TransformationOptionLimits limits) + { + this.limits = limits; + } + + /** * Convert the transformation options into a map. *

* Basic options (optional) are: @@ -217,6 +384,12 @@ public class TransformationOptions *

  • {@link #OPT_TARGET_NODEREF}
    • *
  • {@link #OPT_TARGET_CONTENT_PROPERTY}
    • *
  • {@link #OPT_INCLUDE_EMBEDDED}
    • + *
  • {@link TransformationOptionLimits#OPT_TIMEOUT_MS}
    • + *
  • {@link TransformationOptionLimits#OPT_READ_LIMIT_TIME_MS}
    • + *
  • {@link TransformationOptionLimits#OPT_MAX_SOURCE_SIZE_K_BYTES = "maxSourceSizeKBytes"; + *
  • {@link TransformationOptionLimits#OPT_READ_LIMIT_K_BYTES}
    • + *
  • {@link TransformationOptionLimits#OPT_MAX_PAGES}
    • + *
  • {@link TransformationOptionLimits#OPT_PAGE_LIMIT}
    • * *

    * Override this method to append option values to the map. Derived classes should call @@ -230,9 +403,25 @@ public class TransformationOptions optionsMap.put(OPT_TARGET_NODEREF, targetNodeRef); optionsMap.put(OPT_TARGET_CONTENT_PROPERTY, targetContentProperty); optionsMap.put(OPT_INCLUDE_EMBEDDED, includeEmbedded); + limits.toMap(optionsMap); return optionsMap; } + public String toString(boolean includeLimits) + { + Map map = toMap(); + if (!includeLimits) + { + TransformationOptionLimits.removeFromMap(map); + } + return map.toString(); + } + + public String toString() + { + return toMap().toString(); + } + public static TypeConverter.Converter relaxedBooleanTypeConverter = new TypeConverter.Converter() { public Boolean convert(String source)