33119: Merge V3.4-BUG-FIX (3.4.8) to V4.0-BUG-FIX (4.0.1)
33099: ALF-10412 Nonreducing 100% CPU Uploading Large Files to Share Site Document Library
ALF-10976 Excel files bigger than 2mb cause soffice.exe to take 100% of one CPU for more than 2 minutes in previews.
- Polish TransformerDebug
- Better config for txt and xlsx to swf
33095: ALF-10412 Nonreducing 100% CPU Uploading Large Files to Share Site Document Library
ALF-10976 Excel files bigger than 2mb cause soffice.exe to take 100% of one CPU for more than 2 minutes in previews.
- Improvements to TransformerDebug so that calls to getTransformers use trace rather than debug level logging
allowing one to see the wood for the trees
33016: ALF-10412 Nonreducing 100% CPU Uploading Large Files to Share Site Document Library
ALF-10976 Excel files bigger than 2mb cause soffice.exe to take 100% of one CPU for more than 2 minutes in previews.
- fix build errors - may not get all of them as not tested on Linux
33005: ALF-10412 Nonreducing 100% CPU Uploading Large Files to Share Site Document Library
ALF-10976 Excel files bigger than 2mb cause soffice.exe to take 100% of one CPU for more than 2 minutes in previews.
- Disable transformers if the source txt or xlsx is too large - avoids transforms that don't finish
txt limit is 5MB
xlsx limit is 1MB
- Added a general 2 minute timeout added (ignored by JOD transformers - which already have a 2 minute timeout
and OpenOffice transformers - would require more work)
- Previous commit already limited txt -> pdf -> png so that only the 1st pdf page was created when creating a doclib icon
- Earlier commit already reduced the priority of the background Thread used for transformations so that user interaction
did not suffer.
33004: ALF-10412 Nonreducing 100% CPU Uploading Large Files to Share Site Document Library
ALF-10976 Excel files bigger than 2mb cause soffice.exe to take 100% of one CPU for more than 2 minutes in previews.
- Added time, size and page limits to transformer configuration to allow one to avoid
costly transformations and to stop them if they do occur. Limits avoid a transformer being
selected if the source is too large, or make it throw and Exception or discard data after a given
time, KBytes read or pages created.
- Page limits currently only used by TextToPdfContentTransformer for thumbnail (icon) creation.
- Time, Size and Page limits are currently ignored by JodContentTransformer and OpenOfficeContentTransformerWorker
once running but the max source size limits may be used to avoid the selection of the transformer in the first
place.
- TransformerDebug added to be able to see what is going on. A real eye opener!
log4j org.alfresco.repo.content.transform.TransformerDebug
32136: ALF-10412 Nonreducing 100% CPU Uploading Large Files to Share Site Document Library
Reducing the priority of the async thread pool that is used to perform the transformations so that normal activity (and even garbage collection) is not interrupted by transformations.
git-svn-id: https://svn.alfresco.com/repos/alfresco-enterprise/alfresco/HEAD/root@33223 c4b6b30b-aa2e-2d43-bbcb-ca4b014f7261
- Part 2 - refactor API + collateral damage
- Fix tracking tests to cope with the presence of background transactions - should fix Oracle build and other intermittent build failures
git-svn-id: https://svn.alfresco.com/repos/alfresco-enterprise/alfresco/HEAD/root@32824 c4b6b30b-aa2e-2d43-bbcb-ca4b014f7261
Activiti has one default transition "Next". If there is no transition then the model builder was not finding a translation for the task outcome (and just using the english word, e.g. "Approve"). Now it looks up the translation workflowtask.outcome.[wf:outcome property]
git-svn-id: https://svn.alfresco.com/repos/alfresco-enterprise/alfresco/HEAD/root@32724 c4b6b30b-aa2e-2d43-bbcb-ca4b014f7261
- Refix ALF-10665: Caches that use @@VALUE_NOT_FOUND@@ are not really immutable
- Add NodeBulkLoader.setCheckNodeConsistency
- Use in a transaction to ensure that the node cache views are consistent with
the database views.
- Increase size of contentDataCache and make it support equals checking
- Details:
32162: Read-through cache changes
32163: TransactionalCache changes to support more efficient consistency guarantees
- 'allowEqualsCheck' property allows cache to do a full equals check against changed shared cache values
- In-transaction option 'setDisableSharedCacheReadForTransaction'. Values are cache in-transaction
and written back at the end of the transaction (subject to collision rules) but the first read will
not go to the shared cache.
- Drop optimistic write-through in read-only transactions; they are equally likely to want to flush
stale data.
- Add simpler logic for mutable and allowEqualsCheck and make sure all conditions are covered by tests
32164: Cache node entity support TransactionalCache's allowEqualsCheck
32165: Add NodeDAO.setCheckNodeConsistency() method
- Allows code to request that node metadata is consistent with whatever view the DB is providing
- Incorporate into node concurrency tests without issue
- Only one cache is affected (nodesCache) and it is enhanced by having 'allowEqualsCheck' to prevent
massive flushing when multiple read transactions are all trying to push data into the shared caches,
particularly during (re)indexing operations.
- Further reduces the cache invalidation messages required in order to maintain consistency across
the cluster
32166: Make Lucene reindex work (trackers and FTS) use enforced node consistency
- bulkLoader.setCheckNodeConsistency() incorporated where 'isReadThrough' is on
32167: SOLR tracking uses NodeDAO.setCheckNodeConsistency() during node metadata retrieval
- Ensures that any stale node metadata does not find its way into indexed SOLR node metadata
32207: Fix ALF-11644: AVM cleanup jobs run when WCM is not installed
- Moved scheduled jobs to installable wcm-bootstrap-context.xml
- Also got rid of orphan reaper warnings when running in a cluster
32208: Better hashcode for NodeVersionKey
32209: RECORD ONLY
32210: RECORD ONLY
32212: Proper fix for ALF-10665: Immutable caches do not respond well to null (=> @@VALUE_NOT_FOUND@@)
- The following caches were incorrectly classed as 'immutable':
propertyValueCache
immutableEntityCache
rootNodesCache
allRootNodesCache
authorityCache
tagscopeSummaryCache
imapMessageCache
- The 'immutable' caches are:
node.aspectsCache
node.propertiesCache
node.parentAssocsCache
- The following caches support equals checks:
node.nodesCache
authorityLookupCache
32213: Fixed getNodeRefStatus(): nodesCache caches deleted entries as well.
git-svn-id: https://svn.alfresco.com/repos/alfresco-enterprise/alfresco/HEAD/root@32657 c4b6b30b-aa2e-2d43-bbcb-ca4b014f7261
- addition of index control aspect to surf-config objects - config xml files no longer indexed at all
- refactoring of My Profile user contents fts-alfresco queries to search specific known containers only rather than all
git-svn-id: https://svn.alfresco.com/repos/alfresco-enterprise/alfresco/HEAD/root@31288 c4b6b30b-aa2e-2d43-bbcb-ca4b014f7261
o performance improvements: disable/enable auditing behaviour per-transaction rather than per-node
o add "disable rules" checkbox to GUI and support disabling of rules during import
git-svn-id: https://svn.alfresco.com/repos/alfresco-enterprise/alfresco/HEAD/root@31243 c4b6b30b-aa2e-2d43-bbcb-ca4b014f7261
- Bulk Filesystem Importer
- Adapted from work by Peter Monks (see http://code.google.com/p/alfresco-bulk-filesystem-import) and Romain Guinot
- Refactored to limit repeated code
- Refactored to use the Bulk Processor
- Currently limited to one bulk import at a time (enforced by lock service)
- Unit tests added
- Some fixes applied
- Performance tests; tried different filesystem "walking" strategies
- Still to do: yui dependencies - are these necessary?
git-svn-id: https://svn.alfresco.com/repos/alfresco-enterprise/alfresco/HEAD/root@31100 c4b6b30b-aa2e-2d43-bbcb-ca4b014f7261
30947: ALF-10619: Merged PATCHES/V3.1.2 to V3.4-BUG-FIX
30884: ALF-10588: Another possible race condition resulting in out of sync transactions - found on SQL Server and JBoss in 3.1.2
- FTS could process updated and deleted nodes in the same transaction before the tracker got to them, leaving behind the correct transaction ID and deleted nodes but undeleted container docs!
- We now have to validate all deletions have been honoured when index tracking
30890: ALF-10588: Temporarily disable FTS during IndexCheckServiceImplTest
- Otherwise can get confused by intermediate FTS state of its own nodes!
30894: ALF-10588: Correction to deletion checking
- Only search for deleted nodes, not updated ones too!
30948: ALF-10619: Fixed merge issue
30982: - ALF-10503 60k Site Performance: Admin Console | Groups: search with a value that matches all 60 groups: maxClasuesCount=10000
- ALF-10511 60k Site Performance: Admin Console | Users | Edit User | Group Search with a value that matches all 60 groups: maxClauseCount=10000
- ALF-10608 60k Site Performance: Searching for a group to add to a site with a value that matches all 60 groups: maxClauseCount=10000
- ALF-10515 60k Site Performance: Edit Group Display Name: The first time, nothing appears to happen for 10 seconds after pressing [Save]
- ALF-10514 60k Site Performance: Admin Console | Groups | Search | Delete Group: no feedback to user for 20 seconds after clicking delete icon
30985: Increases in node, property and aspect caches.
30987: Merged DEV/TEMPORARY to V3.4-BUG-FIX
30984: ALF-9880 : ContentGet web script throws NullPointerException for nodes missing cm:modified property
The check for null was added for cm:modified property (similar to BaseDownloadContentServlet).
30995: Fix for ALF-9021
30996: ALF-10324 Cannot disable Home Folder Creation
- Bug introduced into V3.1 on the 8 March 2010
- ChainingUserRegistrySynchronizerTest enhanced to check for this
- Fix to PersonService: Home folder was not being created for 'missing' persons
- PersonService: Changed autoCreate parameters to more descriptive names (okay long) and updated Javadoc
30998: ALF-10512 60k Site Performance: Clicking on Sites (left hand side) in the Repository browser causes transactional limit to be reached
- Changed node, aspect, property and parentAssoc cache sizes (based on Derek's Skype message)
31006: ALF-10512 60k Site Performance: Clicking on Sites (left hand side) in the Repository browser causes a transactional limit to be reached
- Having changed cache sizes in previous commit, the nodeOwner and acl transactional caches were then blown with test case for ALF-10512
Changed to 20k from 10k. Tried 15k but it still had a problem.
31052: Fix for ALF-10520
Merged HEAD to V3.4-BUG-FIX
31051: Performance improvements for Share Repository browser queries.
DB with ~50,000 nodes under Company Home:
Before:
- I'm Editing - 16 secs, Favorites - 17 secs, Tag - 14 secs
After:
- I'm Editing - 1.5 secs, Favorites - 1.2 secs, Tag - 1.25 secs
31058: ALF-10324 Cannot disable Home Folder Creation
- ChainingUserRegistrySynchronizerTest check using personService with both eager and non eager home folder creation
31064: ALF-9360: Merged PATCHES/V3.4.4 to V3.4-BUG-FIX
30244: Merged DEV/DAVEW/IMAP_NEW to PATCHES/V3.4.4
29635: Rework of IMAP to use lightweight caching and correctly set UIDVALIDITY, NEXTUID and Marked / Unmarked state
29668: 1. Changed get AlfrescoImapFolder.getFullNameInternal to be dynamic for cache support
29692: 1. Reverts changed in AlfrescoImapServer to allow ImapHostManager to be a session key for folder.
2. getFlags relies on FileInfo.getProperties()
29741: 1. Changed AbstractMimeMessage.updateMessageId() to follow RFC2822 (3.6.4. Identification fields)
2. Changed ImapServiceImpl to handle absent folders and return "NO" reply to a client.
3. Changed ImapServiceImpl that behaviours don't fail when Alfresco is being first time bootstrapped with IMAP enabled.
4. Cleared AlfrescoImapFolder constructor.
5. Fixed SelectCommand's response to adhere RFC3501 (6.3.1. SELECT Command)
6. Fixed CommandParser to be able parse the flag which is not surrounded by braces (STORE 2:4 +FLAGS \Deleted)
30235: Completion of IMAP rework
- Scalable caching
- Proper transactional cache for assembled messages
- No more assumption that EHcache will always hold entire folder set at once (and perhaps it can't)
- Per session (TCP connection) cache of accessed folders
- Session cache validation via a 'change token' that is incremented on all significant events
- Folder status attributes evaluated once and reused until the change token changes
- Now only changed folders need to be queried on an IMAP sync and the server doesn't have to hold all folders in memory
- User's view is consistent with their security permissions
- Simplification / overhaul of ImapServiceImpl including efficient recursive path building and matching
- AlfrescoImapFolder immutable as it should be
- Greenmail fixes
- Fixed quoting of mailbox names
- Fixed hanging problem in ImapRequestLineReader - regression caused by our 8 bit encoding fix. Avoid using an InputStreamReader to read ISO-8859-1 bytes as it has an internal buffer.
30275: Fix failing IMAP tests broken by my recent refactor!
- Fixed greenmail conversion of ISO-8859-1 bytes to chars
- Transaction read write attributes on service
- Read only commands on AbstractImapFolder
- Imap aspect properties must be managed as SYSTEM user
- Restored persistence of new mail messages
- Avoid unit test txn rollback woes by making it possible to check for existence of a path with FileFolderService
30487: ALF-10268: Merged V3.4-BUG-FIX to PATCHES/V3.4.4
30264: ALF-10187: Merged V3.3 to V3.4-BUG-FIX
30003: ALF-9898: More defensive exception handling to avoid packet pool leaks and extra logging on packet pool exhaustion
30540: ALF-10257: Fixed logic error introduced into Greenmail ImapRequestLineReader
30988: ALF-9361: Merged DEV/DAVEW/IMAP_NEW to PATCHES/V3.4.4 (by Arseny)
30419: Remote test for generic client request sequence.
30547: 1. A bug with FetchCommand particularly with FETCH (BODY.PEEK[1]) with an error
1315912197.789640 1.5 NO FETCH failed. java.lang.String cannot be cast to javax.mail.internet.MimeMultipart
This happened while message content is being proceeded like MimeMultipart mp = (MimeMultipart) mimeMessage.getContent();, but javadoc of mimeMessage.getContent() says that this content can be a String in case of non-multipart message. Fixed FetchCommand accordingly to mimeMessage.getContent() javadoc.
2. A bug with RFC822MetadataExtracter
When mimeMessage.getHeader("received"); is used with the message with following header
Received: with ECARTIS (v1.0.0; list dovecot); Tue, 06 Aug 2002 13:01:17 +0300 (EEST)
It doesn't extract a date, because it uses lastReceived.indexOf(';') which returns the position IN the ECARTIS (v1.0.0; list dovecot) after v1.0.0,
So it should use lastReceived.lastIndexOf(';') to get the position after ECARTIS (v1.0.0; list dovecot).
31025: ALF-9361: IMAP Performance
- Introduced folder status MRU cache
- Keyed by user ID and change token so no need to cluster
- Now means we should get reuse across IMAP sessions
- Also fixed isMarked() implementation to only return true if there are recent or unseen mails
31038: ALF-9361: Prevent the starting of unnecessary transactions in AlfrescoImapFolder interface
- getFolderStatus regulates its own transaction
- Dropped all those *Internal methods from the abstract class
- getUnqualifiedMailboxPattern moved to AlfrescoImapHostManager
- Fixes to session folder cache validation / reuse
31039: ALF-9361: Repository tuning for IMAP performance
- Backed out ALF-5575 60 second timeout on node caches - Should be covered by ALF-8607 fix
- Also made TransactionalCache.NewCacheBucket save new values to the shared cache for 'mutable' caches. Previously it was only possibly to load into the node caches in a read only transaction!
- Also added fix to make AbstractNodeDAOImpl bulk load empty node aspect sets
- Result is a drastic speedup of full sync times as most items can be served from the cache`
31042: ALF-9361: Fix ImapServiceImplTest
31048: ALF-9361: Make ConcurrentNodeServiceTest work again, after relaxation of 'mutable' transactional caches
- aspect and property caches validated by node transaction ID, as per parent assocs in ALF-8607
31050: ALF-9361: Caching correction
Always use the cached mailbox reference if it is equivalent (because the session remembers the last selected mailbox)
31060: ALF-9361: Fix CacheTest, following back out of ALF-5575 behaviour
31061: ALF-9361: More caching fixes
31062: ALF-9361: Undo accidental changes to ConcurrentNodeServiceTest
31063: ALF-9361: Build fix: replaced assertSame with assertEquals
git-svn-id: https://svn.alfresco.com/repos/alfresco-enterprise/alfresco/HEAD/root@31079 c4b6b30b-aa2e-2d43-bbcb-ca4b014f7261
