Dave Ward e18ab791d7 Merged V3.4-BUG-FIX to HEAD
31072: Incremented version.revision for 3.4.7
   31082: Merged DEV to V3.4-BUG-FIX
      31077: ALF-9661: Adding or removing associations does not trigger 'update' rule.
             1. Unit test for replicating.
             2. SingleAssocRefPolicyRuleTrigger now handles trigger of the rules on the list of parents.
   31115: Merged DEV/TEMPORARY to V3.4-BUG-FIX
      30992: ALF-10029 : Webdav error when document type is missing
         Correcting directory listing. Display reader's mimetype if display label for mimetype couldn't be found.
   31116: Merged DEV/TEMPORARY to V3.4-BUG-FIX
      31114: ALF-10309 : CLONE -WebDAV - Canceling "save as" upload will create 0 byte content
         Logic was added to PutMethod that removes empty files if a save operation is cancelled or fails.
   31157: ALF-10757: Merged HEAD to V3.4-BUG-FIX
      31154: Synchronize file writes from CIFS. ALF-10721.
   31188: Merged DEV/TEMPORARY to V3.4-BUG-FIX
      31176: ALF-9979: Error thrown when adding user(s) to group in explorer if user already exists in group
         Implemented checking of the user presence in the particular group using getAuthService().getContainedAuthorities()
   31189: Merged DEV/TEMPORARY to V3.4-BUG-FIX
      31174: ALF-9684: CMIS AtomPub: The combination of includeRelationships=both and returnVersion=latestmajor returns an internal server error
         Try-catch block was added for nodeService.getSourceAssocs(node, RegexQNamePattern.MATCH_ALL) operation. This operation is not supported for a version store.
   31190: Merged DEV/TEMPORARY to V3.4-BUG-FIX
      20694: ALF-925: "title" property always gets populated with the name of a file when content without a file extension is added via Webdav.
         When user creates file through webdav, "title" property was populated with "name" property value.
         The code that was doing it has been removed. Now the title will be empty in such a case.
         This makes things consistent with CIFS and FTP, where you have to create a rule with action "Extract common metadata fields from content". 
   31217: ALF-10807: Share document download with SSL
   - Fix from Bitrock
   31226: ALF-10632: Added a sample configuration fragment for war-deployers-jboss-beans.xml because it's rather tricky to publish the correct configuration in the docs!
   31265: ALF-9999 Cannot search pdf files, which are written vertically.
      Sample configures PdfMiner as the pdf -> text and ??? -> pdf -> text transformer rather than PDFBox.
      PdfMiner is slower but does handle vertical text.
   31268: ALF-9093 Alfresco incapable of previewing text files including non Western European characters
      Sample file that configures OpenOffice as the default handler for text to PDF conversions
   31304: Merged BRANCHES/DEV/BELARUS/HEAD-2011_10_05 to BRANCHES/DEV/V3.4-BUG-FIX:
      31291: ALF-9460 : Archived discussion posts break display of the discussed node when users do not have access to the trash can
   31313: Fix for JBPM security issues with timers and async tasks.
   ALF-10776: Workflow timer fires a policy that causes a AuthenticationCredentialsNotFoundException
   ALF-6405: SecureContext issue if task node async property is set to "true"
   31314: Fixed ALF-10373 "There is no path to the folder for "Move" action inside the rule's description."
   31361: Fixed ALF-10930. Incorrect description of 'xam.archive.forceBackgroundStoreMove'
   31366: ALF-3756 - original fix didn't cope with existing MLText having one or more language variants. 
   ...upgrading to the fix would therefore not solve the problem properly. For example, if a property has en_GB text in it, then 'updating' that property with a locale of en_US will result in the addition of the en_US text rather than a true update (they're both English, and using two slightly differently configured browsers in this way leads to confusion).
   31387: ALF-10740: Flash upload - Working Copy label not removed on check in if non english locale is used in Share client   
   Two separate problems were fixed:   
   * Locale string as supplied by the flash uploader was being placed into the language portion of the Locale object in java, therefore no proper match was made when looking up the working copy label (e.g. "(Working Copy)") - fixed by using a proper locale string parser as supplied by Surf.   
   * Working copy label could still fail to be removed if document was checked out under one locale and checked in using another. Fix provided by recording the *original* working copy label against the cm:workingcopy aspect.   
   31395: ALF-3756: fixes broken FullNodeServiceTest
   31399: ALF-3756: fixing build failures from -c31366 (VersionServiceImplText.testRevert()) 
   A ConcurrentModificationException was being thrown, have changed the way that the hashmap is processed to avoid this.
   31401: Fix for ALF-2980
   31410: Fix for ALF-10961 - Share - Sites not displayed on dashboard
   31418: ALF-10768 Environment variables are being used to replace property placeholders in xml configuration files
      Modified ALL spring beans that are a PropertyPlaceholderConfigurer (including sub classes) and the 
      PropertyPlaceholderConfigurer created ChildApplicationContextFactory so that OS environment variables
      are not used in property place holder substitutions. Spring 3 started allowing OS environment variables
      to be used by default. This may explain why we have so many install problems.
   31420: ALF-3756 another compile error: This time StoreSelectorAspectContentStoreTest
   31425: ALF-10740: Fix from -c 31387 (compilation error)   
   This fix is in addition to -c31420. The compilation error was due to me not having the bug fix branch configured as an enterprise project.
   31458: Fix for ALF-10916
   31459: Fix for ALF-10915
   31482: ALF-10915
     - Fix build
   31491: Various oplock, access mode and CIFS protocol fixes ported from v4/HEAD. ALF-10331.
   31492: File open access mode checks ported from v4/HEAD. ALF-10331.
   31529: ALF-11144: NPE during incremental LDAP sync
   31530: Fixed ALF-11139: Audit concurrency exceptions do not trigger transaction retries
    - AuditMethodInterceptor allows audit exceptions to propagate out
   31606: ALF-10309 CLONE -WebDAV - Cancelling "save as" upload will create 0 byte content
       Changed name of aspect to be WebDAV specific (rather than using a CIFS value), so that it might be possible to tidy these up in future.
   31638: Fix for ALF-10597 - Share - Upload new version issue - OK button displayed during refresh
   31639: Fix for ALF-10297 JSF - Sort by First/Last Name in Groups Management not working
   31641: ALF-9776 - Fix for WCM - it allows users to submit contents to Staging with past Launch and Expiration Date
   31653: ALF-8906: Ensure entire IMAP service startup transaction runs as system, to allow for any auto-aspect adding / versioning during cache warming
   31654: Merged V3.4 to V3.4-BUG-FIX
      31327: ALF-10878 Merged HEAD to V3.4
           30225: ALF-10001 Update SiteService.listMembers to check group display names as well as group names, plus test
      31329: Merged V3.4-BUG-FIX (3.4.7) to V3.4 (3.4.6)
         30992: ALF-10029 : Webdav error when document type is missing
            Correcting directory listing. Display reader's mimetype if display label for mimetype couldn't be found.
      31462: ALF-10010 Document can not be found even after a full reindex.
         Sample configures xpdf as the pdf -> text and ??? -> pdf -> text transformer rather than PDFBox.
      31478: Merged BELARUS/V3.4-BUG-FIX-2011_10_13 to V3.4 (3.4.6)
         31460: ALF-10952: User is unable to edit own profile using Alfresco Share
            To compare two objects it needs to use equals() method instead of ==.
            Such a check was changed in UsageQuotaProtector.onUpdateProperties() method.
      31502: ALF-10324 Cannot disable Home Folder Creation
         2nd part to the problem. Folders were being created when "Show all" was pressed in the UI
      31504: Fix for ALF-9589 - Share - Advanced Search is not IE7 compatible for other themes
      31531: ALF-10324 Cannot disable Home Folder Creation
         There is a knock on effect to ContentUsageService when the user does not exist and we are not allowed to create missing users.
         Resulted in 2 DMDeploymentTargetTest failures
      31532: ALF-10955 No pooled task created for moderated site managers if any user requested to join
         Going with David Wards suggestion of still using getContainingAuthoritiesInZone with a null zone and a cut off at 100.
         The 100 might be too small, but probably is okay.
   31655: ALF-10187: Merged V3.3 to V3.4-BUG-FIX
      31180: ALF-9898: CIFS packet pool leaks
      - File state manager op lock break expiry thread was never getting started so some deferred packets were never getting released!
      - Synchronize around m_borrowed to prevent ConcurrentModificationExceptions in diagnostics
      - Synchronize around m_selector.keys() to prevent ConcurrentModificationExceptions during session reaping
      31244: Merged DEV/BELARUS/V3.4-BUG-FIX-2011_09_29_CIFS to V3.3
         ALF-10831: CIFS concurrency issues under load
            - Atomic synchronized write AND read operations in FileStateCache, TreeConnection, SrvSessionList, VirtualCircuit, VirtualCircuitList, ContentQuotaManager
      31245: Merged DEV/BELARUS/V3.4-BUG-FIX-2011_09_29_CIFS to V3.3
         ALF-9540: copy from drive to CIFS is slower than direct drive to drive copy by a factor of ~ 15
         - batchRemoveRequest added to ThreadRequestQueue, meaning that not all incoming packets are scatter-gunned across different worker threads and reducing context switching
         - Arseny's tests show that combined with 3.4.6 changes, the factor is now under 2!
      31246: ALF-9898: Merged HEAD to V3.3
         31154: Synchronize file writes from CIFS. ALF-10721.
         31233: Fixed a possible memory leak in the CIFS oplock timeout handler.
      31273: ALF-9540: Reversed batchRemoveRequest changes
      - Review by Gary. Thread pool may not be utilized properly when multiple virtual connections are being handled at once.
      31576: Changes to the CIFS idle session reaper so that it runs in the main socket event thread. ALF-9898.


git-svn-id: https://svn.alfresco.com/repos/alfresco-enterprise/alfresco/HEAD/root@31657 c4b6b30b-aa2e-2d43-bbcb-ca4b014f7261
2011-11-02 23:22:24 +00:00

555 lines
16 KiB
Java

/*
* Copyright (C) 2005-2010 Alfresco Software Limited.
*
* This file is part of Alfresco
*
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
*/
package org.alfresco.filesys.repo;
import java.io.IOException;
import java.util.HashMap;
import java.util.Iterator;
import java.util.Map;
import org.alfresco.jlan.server.SrvSession;
import org.alfresco.jlan.server.filesys.DiskDeviceContext;
import org.alfresco.jlan.server.filesys.DiskFullException;
import org.alfresco.jlan.server.filesys.DiskInterface;
import org.alfresco.jlan.server.filesys.NetworkFile;
import org.alfresco.jlan.server.filesys.TreeConnection;
import org.alfresco.jlan.server.filesys.quota.QuotaManager;
import org.alfresco.jlan.server.filesys.quota.QuotaManagerException;
import org.alfresco.jlan.util.MemorySize;
import org.alfresco.jlan.util.StringList;
import org.alfresco.repo.security.authentication.AuthenticationUtil;
import org.alfresco.service.cmr.repository.ContentService;
import org.alfresco.service.cmr.usage.ContentUsageService;
import org.alfresco.util.PropertyCheck;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
/**
* Content Quota Manager Class
*
* <p>Implementation of JLAN QuotaManager interface for the Alfresco repository.
* <p>Keeps an in memory quota for each active user. After a configurable length of
* time quotas are removed from memory.
*
* @author gkspencer
*
*/
public class ContentQuotaManager implements QuotaManager, Runnable {
// Debug logging
private static final Log logger = LogFactory.getLog(ContentQuotaManager.class);
// User details idle check interval
private static final long UserQuotaCheckInterval = 1 * 60 * 1000; // 1 minute
private static final long UserQuotaExpireInterval = 5 * 60 * 1000; // 5 minutes
// Associated filesystem driver
private DiskInterface m_filesys;
// Content usage service
private ContentUsageService m_usageService;
private ContentService contentService;
// Track live usage of users that are writing files
private Map<String, UserQuotaDetails> m_liveUsage;
private Object m_liveUsageLock = new Object();
// User details inactivity checker thread
private Thread m_thread;
private boolean m_shutdown;
public void init()
{
PropertyCheck.mandatory(this, "contentService", getContentService());
PropertyCheck.mandatory(this, "contentUsageService", m_usageService);
}
/**
* Get the usage service
*
* @return ContentUsageService
*/
public final ContentUsageService getUsageService() {
return m_usageService;
}
/**
* Set the usage service
*
* @param usageService ContentUsageService
*/
public final void setUsageService(ContentUsageService usageService) {
m_usageService = usageService;
}
/**
* Return the free space available in bytes
*
* @return long
*/
public long getAvailableFreeSpace() {
// Get the live free space value from the content store, if supported
long freeSpace = contentService.getStoreFreeSpace();
if ( freeSpace == -1L) {
// Content store does not support sizing, return a large dummy value
freeSpace = ContentDiskDriver.DiskFreeDefault;
}
return freeSpace;
}
/**
* Return the free space available to the specified user/session
*
* @param sess SrvSession
* @param tree TreeConnection
* @return long
*/
public long getUserFreeSpace(SrvSession sess, TreeConnection tree) {
// Check if content usage is enabled
if ( m_usageService.getEnabled() == false)
return 0L;
// Check if there is a live usage record for the user
UserQuotaDetails userQuota = getQuotaDetails(sess, true);
if ( userQuota != null) {
synchronized (userQuota) {
return userQuota.getAvailableSpace();
}
}
// No quota details available
return 0L;
}
/**
* Allocate space on the filesystem.
*
* @param sess SrvSession
* @param tree TreeConnection
* @param file NetworkFile
* @param alloc long requested allocation size
* @return long granted allocation size
* @exception IOException
*/
public long allocateSpace(SrvSession sess, TreeConnection tree, NetworkFile file, long alloc)
throws IOException {
// Check if content usage is enabled
if ( m_usageService.getEnabled() == false)
return alloc;
// Check if there is a live usage record for the user
UserQuotaDetails userQuota = getQuotaDetails(sess, true);
long allowedAlloc = 0L;
if ( userQuota != null) {
// Check if the user has a usage quota
synchronized ( userQuota) {
if ( userQuota.hasUserQuota()) {
// Check if the user has enough free space allocation
if ( alloc > 0 && userQuota.getAvailableSpace() >= alloc) {
userQuota.addToCurrentUsage( alloc);
allowedAlloc = alloc;
}
}
else {
// Update the live usage
userQuota.addToCurrentUsage( alloc);
allowedAlloc = alloc;
}
}
}
else if ( logger.isDebugEnabled())
logger.debug("Failed to allocate " + alloc + " bytes for sess " + sess.getUniqueId());
// Check if the allocation was allowed
if ( allowedAlloc < alloc) {
// DEBUG
if ( logger.isDebugEnabled())
logger.debug("Allocation failed userQuota=" + userQuota);
throw new DiskFullException();
}
else if ( logger.isDebugEnabled())
logger.debug("Allocated " + alloc + " bytes, userQuota=" + userQuota);
// Return the allocation size
return allowedAlloc;
}
/**
* Release space to the free space for the filesystem.
*
* @param sess SrvSession
* @param tree TreeConnection
* @param fid int
* @param path String
* @param alloc long
* @exception IOException
*/
public void releaseSpace(SrvSession sess, TreeConnection tree, int fid, String path, long alloc)
throws IOException {
// Check if content usage is enabled
if ( m_usageService.getEnabled() == false)
return;
// Check if there is a live usage record for the user
UserQuotaDetails userQuota = getQuotaDetails(sess, true);
if ( userQuota != null) {
synchronized ( userQuota) {
// Release the space from the live usage value
userQuota.subtractFromCurrentUsage( alloc);
}
// DEBUG
if ( logger.isDebugEnabled())
logger.debug("Released " + alloc + " bytes, userQuota=" + userQuota);
}
else if ( logger.isDebugEnabled())
logger.debug("Failed to release " + alloc + " bytes for sess " + sess.getUniqueId());
}
/**
* Start the quota manager.
*
* @param disk DiskInterface
* @param ctx DiskDeviceContext
* @exception QuotaManagerException
*/
public void startManager(DiskInterface disk, DiskDeviceContext ctx)
throws QuotaManagerException
{
if(logger.isDebugEnabled())
{
logger.debug("Start Quota Manager");
}
// Save the filesystem driver details
m_filesys = disk;
// Allocate the live usage table
m_liveUsage = new HashMap<String, UserQuotaDetails>();
// Create the inactivity checker thread
m_thread = new Thread(this);
m_thread.setDaemon(true);
m_thread.setName("ContentQuotaManagerChecker");
m_thread.start();
}
/**
* Stop the quota manager
*
* @param disk DiskInterface
* @param ctx DiskDeviceContext
* @exception QuotaManagerException
*/
public void stopManager(DiskInterface disk, DiskDeviceContext ctx)
throws QuotaManagerException
{
if(logger.isDebugEnabled())
{
logger.debug("Stop Quota Manager");
}
// Clear out the live usage details
synchronized (m_liveUsageLock)
{
m_liveUsage.clear();
m_shutdown = true;
}
// Shutdown the checker thread
m_thread.interrupt();
}
/**
* Get the usage details for the session/user
*
* @param sess SrvSession
* @param loadDetails boolean
* @return UserQuotaDetails or null
*/
private UserQuotaDetails getQuotaDetails(SrvSession sess, boolean loadDetails) {
synchronized (m_liveUsageLock) {
UserQuotaDetails userQuota = null;
String userName = AuthenticationUtil.getFullyAuthenticatedUser();
if ( sess != null && userName != null)
{
// Get the live usage values
userQuota = m_liveUsage.get(userName);
if ( userQuota == null && loadDetails == true)
{
// User is not in the live tracking table, load details for the user
try
{
logger.debug("user is not in cache - load details");
userQuota = loadUsageDetails(userName);
}
catch ( QuotaManagerException ex)
{
if ( logger.isDebugEnabled())
{
logger.debug("Unable to load usage details", ex);
}
}
}
}
// Return the user quota details
return userQuota;
}
}
/**
* Load the user quota details
*
* @param user - name of the user.
* @return UserQuotaDetails
* @throws QuotaManagerException
*/
private UserQuotaDetails loadUsageDetails(String userName)
throws QuotaManagerException {
// Check if the user name is available
UserQuotaDetails quotaDetails = null;
try
{
if ( userName == null || userName.length() == 0)
{
logger.debug("user name is null or empty - throw QuotaManagerException");
throw new QuotaManagerException("No user name for client");
}
// Get the usage quota and current usage values for the user
long userQuota = m_usageService.getUserQuota( userName);
long userUsage = m_usageService.getUserUsage( userName);
// Create the user quota details for live tracking
quotaDetails = new UserQuotaDetails( userName, userQuota);
if ( userUsage > 0L)
{
quotaDetails.setCurrentUsage( userUsage);
}
// Add the details to the live tracking table
// Check if another thread has added the details
UserQuotaDetails details = m_liveUsage.get( userName);
if ( details != null)
{
quotaDetails = details;
}
else
{
m_liveUsage.put( userName, quotaDetails);
}
// DEBUG
if ( logger.isDebugEnabled())
{
logger.debug( "Added live usage tracking " + quotaDetails);
}
}
catch ( Exception ex)
{
// Log the error
if ( logger.isDebugEnabled())
{
logger.debug("Failed to load usage for" + userName, ex);
}
// Failed to load usage details
throw new QuotaManagerException("Failed to load usage for " + userName + ", " + ex);
}
// Return the user usage details
return quotaDetails;
}
/**
* Inactivity checker, run in a seperate thread
*/
public void run() {
// DEBUG
if ( logger.isDebugEnabled())
logger.debug("Content quota manager checker thread starting");
// Loop forever
StringList removeNameList = new StringList();
m_shutdown = false;
while ( m_shutdown == false)
{
// Sleep for the required interval
try
{
Thread.sleep( UserQuotaCheckInterval);
}
catch (InterruptedException ex)
{
}
// Check for shutdown
if ( m_shutdown == true)
{
// Debug
if ( logger.isDebugEnabled())
logger.debug("Content quota manager checker thread closing");
return;
}
// Check if there are any user quota details to check
synchronized (m_liveUsageLock)
{
if ( m_liveUsage != null && m_liveUsage.size() > 0)
{
try
{
// Timestamp to check if the quota details is inactive
long checkTime = System.currentTimeMillis() - UserQuotaExpireInterval;
// Loop through the user quota details
removeNameList.remoteAllStrings();
Iterator<String> userNames = m_liveUsage.keySet().iterator();
while ( userNames.hasNext()) {
// Get the user quota details and check if it has been inactive in the last check interval
String userName = userNames.next();
UserQuotaDetails quotaDetails = m_liveUsage.get( userName);
synchronized (quotaDetails) {
if ( quotaDetails.getLastUpdated() < checkTime) {
// Add the user name to the remove list, inactive
removeNameList.addString( userName);
}
}
}
// Remove inactive records from the live quota tracking
while ( removeNameList.numberOfStrings() > 0)
{
// Get the current user name and remove the record
String userName = removeNameList.removeStringAt( 0);
UserQuotaDetails quotaDetails = m_liveUsage.remove( userName);
// DEBUG
if ( logger.isDebugEnabled())
logger.debug("Removed inactive usage tracking, " + quotaDetails);
}
}
catch (Exception ex)
{
// Log errors if not shutting down
if ( m_shutdown == false)
logger.debug(ex);
}
}
}
}
}
public void setContentService(ContentService contentService)
{
this.contentService = contentService;
}
public ContentService getContentService()
{
return contentService;
}
}