inteligr8/alfresco-community-repo
1
0
Fork 0
mirror of https://github.com/Alfresco/alfresco-community-repo.git synced 2025-08-07 17:49:17 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merged V3.2 to HEAD

Browse Source 
16662: LDAP sync: improved group association filtering, referential integrity checking, deletion strategy and performance tuning of batch sizes
   16648: ETHREEOH-2752: Improved ticket validation fix
      - Invalidate user's tickets during person deletion rather than validation or it can mess up chained validation
   16647: ETHREEOH-2534: Fixed Sharepoint NTLM authentication
      - user details were never getting cached in the session
   16579: Small improvement to LDAP error reporting
      - Committed errors counted before successes in a logging interval
   16515: LDAP sync performance
      - Improved full sync strategy - run differential queries to work out required updates/additions and full queries to work out required deletions. Saves updating unchanged nodes.
      - Use a TreeSet rather than a HashSet to gather group associations in an attempt to avoid blowing the heap size
   16498: More LDAP performance improvements
      - Uses thread pool with 4 worker threads and blocking queue to process returned results. The number of worker threads can be controlled by the synchronization.workerThreads property.
      - Switched LDAP connection pooling back on again
      - Group Associations processsed individually so that errors are collated and we get a better idea of their throughput
      - Fixed potential bug. Group membership resolution done with isolated LDAP context to avoid cookies from paging creeping in.
   16424: Try switching off LDAP connection pooling to see if it works better with our flaky server.
   16414: Further LDAP fault tolerance
      - Log causes of group member resolution failures where possible
   16413: More fault tolerance for LDAP sync
      - Always commit last sync times before overall sync is complete to avoid the 'forgetting' of differential sync information
      - DN comparisons should be case insensitive to avoid issues resolving DNs to user and group IDs
   16398: Improved monitoring and fault tolerance for LDAP sync
      - When the batch is complete a summary of the number of errors and the last error stack trace will be logged at ERROR level
      - Each individual error is logged at WARN level and progress information (including % complete) is collated and logged at INFO level after a configurable interval
      - In the Enterprise Edition all metrics can be monitored in real time through JMX
      - Sanity testing to be performed by Mike!
   16319: Merged HEAD to V3.2
      16316: ALFCOM-3397: JBoss 5 compatibility fix
         - Relative paths used by LDAP subsystem configuration weren't being resolved correctly
         - See also https://jira.jboss.org/jira/browse/JBAS-6548 and https://jira.springsource.org/browse/SPR-5120
   16272: ETHREEOH-2752: Once more with feeling!
   16261: ETHREEOH-2752: Correct exception propagation.
   16260: ETHREEOH-2752: Fix ticket validation
      - Current ticket was getting forgotten by previous fix
      - Person validation in CHECK mode now done AFTER the current user is set, so that the current ticket is remembered
   16243: ETHREEOH-2752: Improve ticket validation used by all authentication filters
      - Now takes into account whether person actually exists or not
      - Tickets for non-nonexistent persons are now considered invalid and cached session information is invalidated
      - New BaseAuthenticationFilter superclass for all authentication filters
      - Improved fix to ETHREEOH-2839: WebDAV user is cached consistently using a different session attribute from the Web Client
   16233: ETHREEOH-2754: Correction to previous checkin.
      - relogin for SSO authentication, logout for normal login page
      - logout is default
   16232: ETHREEOH-2754: Log Out Action outcome passed as a parameter
      - relogin for SSO authentication, login for normal login page
      - Means the log out link always leads to the correct place, even when the session has expired
      - Also lowered ticket validation error logging to DEBUG level to avoid unnecessary noise in the logs from expired sessions
   16220: ETHREEOH-2839: Fixed potential ClassCastExceptions when Alfresco accessed via WebDAV and Web Client links in same browser
      - WebDAV side no longer directly casts session user to a WebDAVUser
      - ContextListener no longer casts session user to web client user
      - Web client side will 'promote' session user to a web client User if necessary via AuthenticationHelper
      - All authentication filters made to use appropriate AuthenticationHelper methods
   16211: ETHREEOH-2835: LDAP sync batches user and group deletions as well as creations
      - Also improved logging of sync failures
   16197: ETHREEOH-2782: LDAP subsystems now support search-based user DN resolution
      - When ldap.authentication.userNameFormat isn't set (now the default) converts a user ID to a DN by running ldap.synchronization.personQuery with an extra condition tacked on the end to find the user by ID
      - Structured directories and authentication by attributes not in the DN such as email address now supported
   16189: ALFCOM-3283: Prevent errors when user accepts an invite when not logged in
      - new isGuest attribute propagated to user object
      - header component (used by accept-invite page) needs to avoid calling prefs and site webscripts for guest user
      - Conditional stuff in header template changed to use user.isGuest


git-svn-id: https://svn.alfresco.com/repos/alfresco-enterprise/alfresco/HEAD/root@16896 c4b6b30b-aa2e-2d43-bbcb-ca4b014f7261
This commit is contained in:
Dave Ward
2009-10-14 09:24:13 +00:00
parent e096d59076
commit 85c1b71826
24 changed files with 2372 additions and 809 deletions
Download Patch File Download Diff File Expand all files Collapse all files

105
source/java/org/alfresco/repo/security/sync/BatchMonitor.java Normal file
View File

@@ -0,0 +1,105 @@
/*
* Copyright (C) 2005-2009 Alfresco Software Limited.
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License
* as published by the Free Software Foundation; either version 2
* of the License, or (at your option) any later version.
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
* As a special exception to the terms and conditions of version 2.0 of
* the GPL, you may redistribute this Program in connection with Free/Libre
* and Open Source Software ("FLOSS") applications as described in Alfresco's
* FLOSS exception. You should have received a copy of the text describing
* the FLOSS exception, and it is also available here:
* http://www.alfresco.com/legal/licensing"
*/
package org.alfresco.repo.security.sync;
import java.util.Date;
/**
* An interface that allows the monitoring of metrics relating to a potentially long-running batch process.
*
* @author dward
*/
public interface BatchMonitor
{
/**
* Gets the process name.
*
* @return the process name
*/
public String getProcessName();
/**
* Gets the start time.
*
* @return the start time
*/
public Date getStartTime();
/**
* Gets the total number of results.
*
* @return the total number of results
*/
public int getTotalResults();
/**
* Gets the ID of the entry being processed
*
* @return the current entry id
*/
public String getCurrentEntryId();
/**
* Gets the number of successfully processed entries.
*
* @return the successfully processed entries
*/
public int getSuccessfullyProcessedEntries();
/**
* Gets the progress expressed as a percentage.
*
* @return the progress expressed as a percentage
*/
public String getPercentComplete();
/**
* Gets the total number of errors.
*
* @return the total number of errors
*/
public int getTotalErrors();
/**
* Gets the stack trace of the last error.
*
* @return the stack trace of the last error
*/
public String getLastError();
/**
* Gets the entry id that caused the last error.
*
* @return the last error entry id
*/
public String getLastErrorEntryId();
/**
* Gets the end time.
*
* @return the end time
*/
public Date getEndTime();
}

60
source/java/org/alfresco/repo/security/sync/BatchMonitorEvent.java Normal file
View File

@@ -0,0 +1,60 @@
/*
* Copyright (C) 2005-2009 Alfresco Software Limited.
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License
* as published by the Free Software Foundation; either version 2
* of the License, or (at your option) any later version.
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
* As a special exception to the terms and conditions of version 2.0 of
* the GPL, you may redistribute this Program in connection with Free/Libre
* and Open Source Software ("FLOSS") applications as described in Alfresco's
* FLOSS exception. You should have received a copy of the text describing
* the FLOSS exception, and it is also available here:
* http://www.alfresco.com/legal/licensing"
*/
package org.alfresco.repo.security.sync;
import org.springframework.context.ApplicationEvent;
/**
* An event alerting listeners to the existence of a new {@link BatchMonitor}.
*
* @author dward
*/
public class BatchMonitorEvent extends ApplicationEvent
{
private static final long serialVersionUID = -5787104103292355106L;
/**
* The Constructor.
*
* @param source
* the source of the event
*/
public BatchMonitorEvent(BatchMonitor source)
{
super(source);
}
/**
* Gets the source batch monitor.
*
* @return the batch monitor
*/
public BatchMonitor getBatchMonitor()
{
return (BatchMonitor) getSource();
}
}

608
source/java/org/alfresco/repo/security/sync/BatchProcessor.java Normal file
View File

@@ -0,0 +1,608 @@
/*
* Copyright (C) 2005-2009 Alfresco Software Limited.
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License
* as published by the Free Software Foundation; either version 2
* of the License, or (at your option) any later version.
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
* As a special exception to the terms and conditions of version 2.0 of
* the GPL, you may redistribute this Program in connection with Free/Libre
* and Open Source Software ("FLOSS") applications as described in Alfresco's
* FLOSS exception. You should have received a copy of the text describing
* the FLOSS exception, and it is also available here:
* http://www.alfresco.com/legal/licensing"
*/
package org.alfresco.repo.security.sync;
import java.io.PrintWriter;
import java.io.StringWriter;
import java.io.Writer;
import java.text.NumberFormat;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Date;
import java.util.Iterator;
import java.util.List;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
import org.alfresco.error.AlfrescoRuntimeException;
import org.alfresco.repo.transaction.RetryingTransactionHelper;
import org.alfresco.repo.transaction.RetryingTransactionHelper.RetryingTransactionCallback;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.springframework.context.ApplicationEventPublisher;
/**
* A <code>BatchProcessor</code> manages the running and monitoring of a potentially long-running transactional batch
* process. It iterates over a collection, and queues jobs that fire a worker on a batch of members. The queued jobs
* handle progress / error reporting, transaction delineation and retrying. They are processed in parallel by a pool of
* threads of a configurable size. The job processing is designed to be fault tolerant and will continue in the event of
* errors. When the batch is complete a summary of the number of errors and the last error stack trace will be logged at
* ERROR level. Each individual error is logged at WARN level and progress information is logged at INFO level. Through
* the {@link BatchMonitor} interface, it also supports the real-time monitoring of batch metrics (e.g. over JMX in the
* Enterprise Edition).
*
* @author dward
*/
public class BatchProcessor<T> implements BatchMonitor
{
/** Let's share the logger of ChainingUserRegistrySynchronizer. */
private static final Log logger = LogFactory.getLog(ChainingUserRegistrySynchronizer.class);
/** The retrying transaction helper. */
private RetryingTransactionHelper retryingTransactionHelper;
/** The collection. */
private Collection<T> collection;
/** The process name. */
private String processName;
/** The number of entries to process before reporting progress. */
private int loggingInterval;
/** The number of worker threads. */
private int workerThreads;
/** The number of entries we process at a time in a transaction *. */
private final int batchSize;
/** The current entry id. */
private String currentEntryId;
/** The last error. */
private Throwable lastError;
/** The last error entry id. */
private String lastErrorEntryId;
/** The total number of errors. */
private int totalErrors;
/** The number of successfully processed entries. */
private int successfullyProcessedEntries;
/** The start time. */
private Date startTime;
/** The end time. */
private Date endTime;
/**
* Instantiates a new batch processor.
*
* @param retryingTransactionHelper
* the retrying transaction helper
* @param collection
* the collection
* @param processName
* the process name
* @param loggingInterval
* the number of entries to process before reporting progress
* @param applicationEventPublisher
* the application event publisher
* @param workerThreads
* the number of worker threads
* @param batchSize
* the number of entries we process at a time in a transaction
*/
public BatchProcessor(RetryingTransactionHelper retryingTransactionHelper,
ApplicationEventPublisher applicationEventPublisher, Collection<T> collection, String processName,
int loggingInterval, int workerThreads, int batchSize)
{
this.retryingTransactionHelper = retryingTransactionHelper;
this.collection = collection;
this.processName = processName;
this.loggingInterval = loggingInterval;
this.workerThreads = workerThreads;
this.batchSize = batchSize;
// Let the (enterprise) monitoring side know of our presence
applicationEventPublisher.publishEvent(new BatchMonitorEvent(this));
}
/*
* (non-Javadoc)
* @see org.alfresco.repo.security.sync.BatchMonitor#getCurrentEntryId()
*/
public synchronized String getCurrentEntryId()
{
return this.currentEntryId;
}
/*
* (non-Javadoc)
* @see org.alfresco.repo.security.sync.BatchMonitor#getLastError()
*/
public synchronized String getLastError()
{
if (this.lastError == null)
{
return null;
}
Writer buff = new StringWriter(1024);
PrintWriter out = new PrintWriter(buff);
this.lastError.printStackTrace(out);
out.close();
return buff.toString();
}
/*
* (non-Javadoc)
* @see org.alfresco.repo.security.sync.BatchMonitor#getLastErrorEntryId()
*/
public synchronized String getLastErrorEntryId()
{
return this.lastErrorEntryId;
}
/*
* (non-Javadoc)
* @see org.alfresco.repo.security.sync.BatchMonitor#getBatchType()
*/
public synchronized String getProcessName()
{
return this.processName;
}
/*
* (non-Javadoc)
* @see org.alfresco.repo.security.sync.BatchMonitor#getSuccessfullyProcessedResults()
*/
public synchronized int getSuccessfullyProcessedEntries()
{
return this.successfullyProcessedEntries;
}
/*
* (non-Javadoc)
* @see org.alfresco.repo.security.sync.BatchMonitor#getPercentComplete()
*/
public synchronized String getPercentComplete()
{
int totalResults = this.collection.size();
int processed = this.successfullyProcessedEntries + this.totalErrors;
return processed <= totalResults ? NumberFormat.getPercentInstance().format(
totalResults == 0 ? 1.0F : (float) processed / totalResults) : "Unknown";
}
/*
* (non-Javadoc)
* @see org.alfresco.repo.security.sync.BatchMonitor#getTotalErrors()
*/
public synchronized int getTotalErrors()
{
return this.totalErrors;
}
/*
* (non-Javadoc)
* @see org.alfresco.repo.security.sync.BatchMonitor#getTotalResults()
*/
public int getTotalResults()
{
return this.collection.size();
}
/*
* (non-Javadoc)
* @see org.alfresco.repo.security.sync.BatchMonitor#getEndTime()
*/
public synchronized Date getEndTime()
{
return this.endTime;
}
/*
* (non-Javadoc)
* @see org.alfresco.repo.security.sync.BatchMonitor#getStartTime()
*/
public synchronized Date getStartTime()
{
return this.startTime;
}
/**
* Invokes the worker for each entry in the collection, managing transactions and collating success / failure
* information.
*
* @param worker
* the worker
* @param splitTxns
* Can the modifications to Alfresco be split across multiple transactions for maximum performance? If
* <code>true</code>, worker invocations are isolated in separate transactions in batches of 10 for
* increased performance. If <code>false</code>, all invocations are performed in the current
* transaction. This is required if calling synchronously (e.g. in response to an authentication event in
* the same transaction).
* @return the number of invocations
*/
@SuppressWarnings("serial")
public int process(final Worker<T> worker, final boolean splitTxns)
{
int count = this.collection.size();
synchronized (this)
{
this.startTime = new Date();
if (BatchProcessor.logger.isInfoEnabled())
{
if (count >= 0)
{
BatchProcessor.logger.info(getProcessName() + ": Commencing batch of " + count + " entries");
}
else
{
BatchProcessor.logger.info(getProcessName() + ": Commencing batch");
}
}
}
// Create a thread pool executor with the specified number of threads and a finite blocking queue of jobs
ExecutorService executorService = splitTxns && this.workerThreads > 1 ? new ThreadPoolExecutor(
this.workerThreads, this.workerThreads, 0L, TimeUnit.MILLISECONDS, new ArrayBlockingQueue<Runnable>(
this.workerThreads * 10)
{
// Add blocking behaviour to work queue
@Override
public boolean offer(Runnable o)
{
try
{
put(o);
}
catch (InterruptedException e)
{
return false;
}
return true;
}
}) : null;
try
{
Iterator<T> iterator = this.collection.iterator();
List<T> batch = new ArrayList<T>(this.batchSize);
while (iterator.hasNext())
{
batch.add(iterator.next());
boolean hasNext = iterator.hasNext();
if (batch.size() >= this.batchSize || !hasNext)
{
final TxnCallback callback = new TxnCallback(worker, batch, splitTxns);
if (hasNext)
{
batch = new ArrayList<T>(this.batchSize);
}
if (executorService == null)
{
callback.run();
}
else
{
executorService.execute(callback);
}
}
}
return count;
}
finally
{
if (executorService != null)
{
executorService.shutdown();
try
{
executorService.awaitTermination(Long.MAX_VALUE, TimeUnit.SECONDS);
}
catch (InterruptedException e)
{
}
}
synchronized (this)
{
reportProgress(true);
this.endTime = new Date();
if (BatchProcessor.logger.isInfoEnabled())
{
if (count >= 0)
{
BatchProcessor.logger.info(getProcessName() + ": Completed batch of " + count + " entries");
}
else
{
BatchProcessor.logger.info(getProcessName() + ": Completed batch");
}
}
if (this.totalErrors > 0 && BatchProcessor.logger.isErrorEnabled())
{
BatchProcessor.logger.error(getProcessName() + ": " + this.totalErrors
+ " error(s) detected. Last error from entry \"" + this.lastErrorEntryId + "\"",
this.lastError);
}
}
}
}
/**
* Reports the current progress.
*
* @param last
* Have all jobs been processed? If <code>false</code> then progress is only reported after the number of
* entries indicated by {@link #loggingInterval}. If <code>true</code> then progress is reported if this
* is not one of the entries indicated by {@link #loggingInterval}.
*/
private synchronized void reportProgress(boolean last)
{
int processed = this.successfullyProcessedEntries + this.totalErrors;
if (processed % this.loggingInterval == 0 ^ last)
{
StringBuilder message = new StringBuilder(100).append(getProcessName()).append(": Processed ").append(
processed).append(" entries");
int totalResults = this.collection.size();
if (totalResults >= processed)
{
message.append(" out of ").append(totalResults).append(". ").append(
NumberFormat.getPercentInstance().format(
totalResults == 0 ? 1.0F : (float) processed / totalResults)).append(" complete");
}
long duration = System.currentTimeMillis() - startTime.getTime();
if (duration > 0)
{
message.append(". Rate: ").append(processed * 1000 / duration).append(" per second");
}
message.append(". " + this.totalErrors + " failures detected.");
BatchProcessor.logger.info(message);
}
}
/**
* An interface for workers to be invoked by the {@link BatchProcessor}.
*/
public interface Worker<T>
{
/**
* Gets an identifier for the given entry (for monitoring / logging purposes).
*
* @param entry
* the entry
* @return the identifier
*/
public String getIdentifier(T entry);
/**
* Processes the given entry.
*
* @param entry
* the entry
* @throws Throwable
* on any error
*/
public void process(T entry) throws Throwable;
}
/**
* A callback that invokes a worker on a batch, optionally in a new transaction.
*/
class TxnCallback implements RetryingTransactionCallback<Object>, Runnable
{
/**
* Instantiates a new callback.
*
* @param worker
* the worker
* @param batch
* the batch to process
* @param splitTxns
* If <code>true</code>, the worker invocation is made in a new transaction.
*/
public TxnCallback(Worker<T> worker, List<T> batch, boolean splitTxns)
{
this.worker = worker;
this.batch = batch;
this.splitTxns = splitTxns;
}
/** The worker. */
private final Worker<T> worker;
/** The batch. */
private final List<T> batch;
/** If <code>true</code>, the worker invocation is made in a new transaction. */
private final boolean splitTxns;
/** The total number of errors. */
private int txnErrors;
/** The number of successfully processed entries. */
private int txnSuccesses;
/** The last error. */
private Throwable txnLastError;
/** The last error entry id. */
private String txnLastErrorEntryId;
/*
* (non-Javadoc)
* @see org.alfresco.repo.transaction.RetryingTransactionHelper.RetryingTransactionCallback#execute ()
*/
public Object execute() throws Throwable
{
reset();
for (T entry : this.batch)
{
String txnEntryId = this.worker.getIdentifier(entry);
synchronized (BatchProcessor.this)
{
BatchProcessor.this.currentEntryId = txnEntryId;
}
try
{
this.worker.process(entry);
this.txnSuccesses++;
}
catch (Throwable t)
{
this.txnLastError = t;
this.txnLastErrorEntryId = txnEntryId;
this.txnErrors++;
if (RetryingTransactionHelper.extractRetryCause(t) == null)
{
if (BatchProcessor.logger.isWarnEnabled())
{
BatchProcessor.logger.warn(getProcessName() + ": Failed to process entry \"" + txnEntryId
+ "\".", t);
}
}
else
{
throw t;
}
}
}
return null;
}
/*
* (non-Javadoc)
* @see java.lang.Runnable#run()
*/
public void run()
{
try
{
BatchProcessor.this.retryingTransactionHelper.doInTransaction(this, false, this.splitTxns);
}
catch (Throwable t)
{
// If the callback was in its own transaction, it must have run out of retries
if (this.splitTxns)
{
if (BatchProcessor.logger.isWarnEnabled())
{
BatchProcessor.logger.warn(getProcessName() + ": Failed to process entry \""
+ BatchProcessor.this.currentEntryId + "\".", t);
}
}
// Otherwise, we have a retryable exception that we should propagate
else
{
if (t instanceof RuntimeException)
{
throw (RuntimeException) t;
}
if (t instanceof Error)
{
throw (Error) t;
}
throw new AlfrescoRuntimeException("Transactional error during " + getProcessName(), t);
}
}
commitProgress();
}
/**
* Resets the callback state for a retry.
*/
private void reset()
{
this.txnLastError = null;
this.txnLastErrorEntryId = null;
this.txnSuccesses = this.txnErrors = 0;
}
/**
* Commits progress from this transaction after a successful commit.
*/
private void commitProgress()
{
synchronized (BatchProcessor.this)
{
if (this.txnErrors > 0)
{
int processed = BatchProcessor.this.successfullyProcessedEntries + BatchProcessor.this.totalErrors;
int currentIncrement = processed % BatchProcessor.this.loggingInterval;
int newErrors = BatchProcessor.this.totalErrors + this.txnErrors;
// Work out the number of logging intervals we will cross and report them
int intervals = (this.txnErrors + currentIncrement) / BatchProcessor.this.loggingInterval;
if (intervals > 0)
{
BatchProcessor.this.totalErrors += BatchProcessor.this.loggingInterval - currentIncrement;
reportProgress(false);
while (--intervals > 0)
{
BatchProcessor.this.totalErrors += BatchProcessor.this.loggingInterval;
reportProgress(false);
}
}
BatchProcessor.this.totalErrors = newErrors;
}
if (this.txnSuccesses > 0)
{
int processed = BatchProcessor.this.successfullyProcessedEntries + BatchProcessor.this.totalErrors;
int currentIncrement = processed % BatchProcessor.this.loggingInterval;
int newSuccess = BatchProcessor.this.successfullyProcessedEntries + this.txnSuccesses;
// Work out the number of logging intervals we will cross and report them
int intervals = (this.txnSuccesses + currentIncrement) / BatchProcessor.this.loggingInterval;
if (intervals > 0)
{
BatchProcessor.this.successfullyProcessedEntries += BatchProcessor.this.loggingInterval
- currentIncrement;
reportProgress(false);
while (--intervals > 0)
{
BatchProcessor.this.successfullyProcessedEntries += BatchProcessor.this.loggingInterval;
reportProgress(false);
}
}
BatchProcessor.this.successfullyProcessedEntries = newSuccess;
}
if (this.txnLastError != null)
{
BatchProcessor.this.lastError = this.txnLastError;
BatchProcessor.this.lastErrorEntryId = this.txnLastErrorEntryId;
}
reset();
}
}
}
}

828
source/java/org/alfresco/repo/security/sync/ChainingUserRegistrySynchronizer.java
View File

File diff suppressed because it is too large Load Diff

243
source/java/org/alfresco/repo/security/sync/ChainingUserRegistrySynchronizerTest.java
View File

@@ -24,6 +24,8 @@
*/
package org.alfresco.repo.security.sync;
import java.util.AbstractCollection;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.Collections;
@@ -31,6 +33,7 @@ import java.util.Date;
import java.util.HashSet;
import java.util.Iterator;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
@@ -46,6 +49,7 @@ import org.alfresco.service.cmr.repository.NodeService;
import org.alfresco.service.cmr.security.AuthorityService;
import org.alfresco.service.cmr.security.AuthorityType;
import org.alfresco.service.cmr.security.PersonService;
import org.alfresco.util.GUID;
import org.alfresco.util.PropertyMap;
import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;
@@ -65,7 +69,7 @@ public class ChainingUserRegistrySynchronizerTest extends TestCase
"classpath:alfresco/application-context.xml", "classpath:sync-test-context.xml"
};
/** The Spring application context */
/** The Spring application context. */
private static ApplicationContext context = new ClassPathXmlApplicationContext(
ChainingUserRegistrySynchronizerTest.CONFIG_LOCATIONS);
@@ -90,6 +94,10 @@ public class ChainingUserRegistrySynchronizerTest extends TestCase
/** The retrying transaction helper. */
private RetryingTransactionHelper retryingTransactionHelper;
/*
* (non-Javadoc)
* @see junit.framework.TestCase#setUp()
*/
@Override
protected void setUp() throws Exception
{
@@ -110,6 +118,10 @@ public class ChainingUserRegistrySynchronizerTest extends TestCase
.getBean("retryingTransactionHelper");
}
/*
* (non-Javadoc)
* @see junit.framework.TestCase#tearDown()
*/
@Override
protected void tearDown() throws Exception
{
@@ -191,6 +203,12 @@ public class ChainingUserRegistrySynchronizerTest extends TestCase
});
}
/**
* Tear down test users and groups.
*
* @throws Exception
* the exception
*/
private void tearDownTestUsersAndGroups() throws Exception
{
// Wipe out everything that was in Z1 and Z2
@@ -301,7 +319,7 @@ public class ChainingUserRegistrySynchronizerTest extends TestCase
* <pre>
* Z1
* G1 - U6
* G2 -
* G2 -
* G3 - U2, G5 - U6
* G6 - u3
*
@@ -365,7 +383,30 @@ public class ChainingUserRegistrySynchronizerTest extends TestCase
}
/**
* Constructs a description of a test group
* Tests synchronization with a zone with a larger volume of authorities.
*
* @throws Exception
* the exception
*/
public void testVolume() throws Exception
{
List<NodeDescription> persons = new ArrayList<NodeDescription>(new RandomPersonCollection(100));
List<NodeDescription> groups = new ArrayList<NodeDescription>(new RandomGroupCollection(100, persons));
this.applicationContextManager.setUserRegistries(new MockUserRegistry("Z0", persons, groups));
this.retryingTransactionHelper.doInTransaction(new RetryingTransactionCallback<Object>()
{
public Object execute() throws Throwable
{
ChainingUserRegistrySynchronizerTest.this.synchronizer.synchronize(true, true);
return null;
}
});
tearDownTestUsersAndGroups();
}
/**
* Constructs a description of a test group.
*
* @param name
* the name
@@ -375,9 +416,10 @@ public class ChainingUserRegistrySynchronizerTest extends TestCase
*/
private NodeDescription newGroup(String name, String... members)
{
NodeDescription group = new NodeDescription();
String longName = longName(name);
NodeDescription group = new NodeDescription(longName);
PropertyMap properties = group.getProperties();
properties.put(ContentModel.PROP_AUTHORITY_NAME, longName(name));
properties.put(ContentModel.PROP_AUTHORITY_NAME, longName);
if (members.length > 0)
{
Set<String> assocs = group.getChildAssociations();
@@ -413,7 +455,7 @@ public class ChainingUserRegistrySynchronizerTest extends TestCase
*/
private NodeDescription newPerson(String userName, String email)
{
NodeDescription person = new NodeDescription();
NodeDescription person = new NodeDescription(userName);
PropertyMap properties = person.getProperties();
properties.put(ContentModel.PROP_USERNAME, userName);
properties.put(ContentModel.PROP_FIRSTNAME, userName + "F");
@@ -481,7 +523,7 @@ public class ChainingUserRegistrySynchronizerTest extends TestCase
}
/**
* Asserts that a person's email has the expected value
* Asserts that a person's email has the expected value.
*
* @param personName
* the person name
@@ -518,10 +560,27 @@ public class ChainingUserRegistrySynchronizerTest extends TestCase
private String zoneId;
/** The persons. */
private NodeDescription[] persons;
private Collection<NodeDescription> persons;
/** The groups. */
private NodeDescription[] groups;
private Collection<NodeDescription> groups;
/**
* Instantiates a new mock user registry.
*
* @param zoneId
* the zone id
* @param persons
* the persons
* @param groups
* the groups
*/
public MockUserRegistry(String zoneId, Collection<NodeDescription> persons, Collection<NodeDescription> groups)
{
this.zoneId = zoneId;
this.persons = persons;
this.groups = groups;
}
/**
* Instantiates a new mock user registry.
@@ -535,9 +594,7 @@ public class ChainingUserRegistrySynchronizerTest extends TestCase
*/
public MockUserRegistry(String zoneId, NodeDescription[] persons, NodeDescription[] groups)
{
this.zoneId = zoneId;
this.persons = persons;
this.groups = groups;
this(zoneId, Arrays.asList(persons), Arrays.asList(groups));
}
/**
@@ -552,20 +609,32 @@ public class ChainingUserRegistrySynchronizerTest extends TestCase
/*
* (non-Javadoc)
* @see org.alfresco.repo.security.sync.UserRegistry#getGroups(java.util.Date)
* @see org.alfresco.repo.security.sync.UserRegistry#getGroups(java.util.Date, java.util.Set, boolean)
*/
public Iterator<NodeDescription> getGroups(Date modifiedSince)
public Collection<NodeDescription> getGroups(Date modifiedSince, Set<String> candidateAuthoritiesForDeletion,
boolean prune)
{
return Arrays.asList(this.groups).iterator();
if (prune)
{
for (NodeDescription person : this.persons)
{
candidateAuthoritiesForDeletion.remove(person.getProperties().get(ContentModel.PROP_USERNAME));
}
for (NodeDescription group : this.groups)
{
candidateAuthoritiesForDeletion.remove(group.getProperties().get(ContentModel.PROP_AUTHORITY_NAME));
}
}
return this.groups;
}
/*
* (non-Javadoc)
* @see org.alfresco.repo.security.sync.UserRegistry#getPersons(java.util.Date)
*/
public Iterator<NodeDescription> getPersons(Date modifiedSince)
public Collection<NodeDescription> getPersons(Date modifiedSince)
{
return Arrays.asList(this.persons).iterator();
return this.persons;
}
}
@@ -615,4 +684,144 @@ public class ChainingUserRegistrySynchronizerTest extends TestCase
return this.contexts.keySet();
}
}
/**
* A collection whose iterator returns randomly generated persons.
*/
public class RandomPersonCollection extends AbstractCollection<NodeDescription>
{
/** The collection size. */
private final int size;
/**
* The Constructor.
*
* @param size
* the collection size
*/
public RandomPersonCollection(int size)
{
this.size = size;
}
/*
* (non-Javadoc)
* @see java.util.AbstractCollection#iterator()
*/
@Override
public Iterator<NodeDescription> iterator()
{
return new Iterator<NodeDescription>()
{
private int pos;
public boolean hasNext()
{
return pos < size;
}
public NodeDescription next()
{
pos++;
return newPerson("U" + GUID.generate());
}
public void remove()
{
throw new UnsupportedOperationException();
}
};
}
/*
* (non-Javadoc)
* @see java.util.AbstractCollection#size()
*/
@Override
public int size()
{
return size;
}
}
/**
* A collection whose iterator returns randomly generated groups with random associations to a given list of
* persons.
*/
public class RandomGroupCollection extends AbstractCollection<NodeDescription>
{
/** The collection size. */
private final int size;
/** The persons. */
private final List<NodeDescription> persons;
/**
* The Constructor.
*
* @param size
* the collection size
* @param persons
* the persons
*/
public RandomGroupCollection(int size, List<NodeDescription> persons)
{
this.size = size;
this.persons = persons;
}
/*
* (non-Javadoc)
* @see java.util.AbstractCollection#iterator()
*/
@Override
public Iterator<NodeDescription> iterator()
{
return new Iterator<NodeDescription>()
{
private int pos;
public boolean hasNext()
{
return pos < size;
}
public NodeDescription next()
{
pos++;
String[] personNames = new String[10];
for (int i = 0; i < personNames.length; i++)
{
personNames[i] = (String) persons.get((int) (Math.random() * (double) (persons.size() - 1)))
.getProperties().get(ContentModel.PROP_USERNAME);
}
return newGroup("G" + GUID.generate(), personNames);
}
public void remove()
{
throw new UnsupportedOperationException();
}
};
}
/*
* (non-Javadoc)
* @see java.util.AbstractCollection#size()
*/
@Override
public int size()
{
return size;
}
}
}

26
source/java/org/alfresco/repo/security/sync/NodeDescription.java
View File

@@ -37,6 +37,11 @@ import org.alfresco.util.PropertyMap;
*/
public class NodeDescription
{
/**
* An identifier for the node for monitoring purposes. Should help trace where the node originated from.
*/
private String sourceId;
/** The properties. */
private final PropertyMap properties = new PropertyMap(19);
@@ -46,6 +51,27 @@ public class NodeDescription
/** The last modification date. */
private Date lastModified;
/**
* Instantiates a new node description.
*
* @param sourceId
* An identifier for the node for monitoring purposes. Should help trace where the node originated from.
*/
public NodeDescription(String sourceId)
{
this.sourceId = sourceId;
}
/**
* Gets an identifier for the node for monitoring purposes. Should help trace where the node originated from.
*
* @return an identifier for the node for monitoring purposes
*/
public String getSourceId()
{
return sourceId;
}
/**
* Gets the last modification date.
*

21
source/java/org/alfresco/repo/security/sync/UserRegistry.java
View File

@@ -24,8 +24,9 @@
*/
package org.alfresco.repo.security.sync;
import java.util.Collection;
import java.util.Date;
import java.util.Iterator;
import java.util.Set;
/**
* A <code>UserRegistry</code> is an encapsulation of an external registry from which user and group information can be
@@ -42,21 +43,29 @@ public interface UserRegistry
* @param modifiedSince
* if non-null, then only descriptions of users modified since this date should be returned; if
* <code>null</code> then descriptions of all users should be returned.
* @return a {@link Iterator} over {@link NodeDescription}s of all the persons (users) in the user registry or all
* @return a {@link Collection} of {@link NodeDescription}s of all the persons (users) in the user registry or all
* those changed since a certain date. The description properties should correspond to those of an Alfresco
* person node.
*/
public Iterator<NodeDescription> getPersons(Date modifiedSince);
public Collection<NodeDescription> getPersons(Date modifiedSince);
/**
* Gets descriptions of all the groups in the user registry or all those changed since a certain date.
* Gets descriptions of all the groups in the user registry or all those changed since a certain date. Group
* associations should be restricted to those in the given set of known authorities. Optionally this set is 'pruned'
* to contain only those authorities that no longer exist in the user registry, i.e. the deletion candidates.
*
* @param modifiedSince
* if non-null, then only descriptions of groups modified since this date should be returned; if
* <code>null</code> then descriptions of all groups should be returned.
* @return a {@link Iterator} over {@link NodeDescription}s of all the groups in the user registry or all those
* @param knownAuthorities
* the current set of known authorities
* @param prune
* should this set be 'pruned' so that it contains only those authorities that do not exist in the
* registry, i.e. the deletion candidates?
* @return a {@link Collection} of {@link NodeDescription}s of all the groups in the user registry or all those
* changed since a certain date. The description properties should correspond to those of an Alfresco
* authority node.
*/
public Iterator<NodeDescription> getGroups(Date modifiedSince);
public Collection<NodeDescription> getGroups(Date modifiedSince, Set<String> knownAuthorities, boolean prune);
}

47
source/java/org/alfresco/repo/security/sync/ldap/LDAPNameResolver.java Normal file
View File

@@ -0,0 +1,47 @@
/*
* Copyright (C) 2005-2009 Alfresco Software Limited.
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License
* as published by the Free Software Foundation; either version 2
* of the License, or (at your option) any later version.
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
* As a special exception to the terms and conditions of version 2.0 of
* the GPL, you may redistribute this Program in connection with Free/Libre
* and Open Source Software ("FLOSS") applications as described in Alfresco's
* FLOSS exception. You should have received a copy of the text describing
* the FLOSS exception, and it is also available here:
* http://www.alfresco.com/legal/licensing"
*/
package org.alfresco.repo.security.sync.ldap;
import org.alfresco.repo.security.authentication.AuthenticationException;
/**
* An interface for objects capable of resolving user IDs to full LDAP Distinguished Names (DNs).
*
* @author dward
*/
public interface LDAPNameResolver
{
/**
* Resolves a user ID to a distinguished name.
*
* @param userId
* the user id
* @return the DN
* @throws AuthenticationException
* if the user ID cannot be resolved
*/
public String resolveDistinguishedName(String userId) throws AuthenticationException;
}

998
source/java/org/alfresco/repo/security/sync/ldap/LDAPUserRegistry.java
View File

File diff suppressed because it is too large Load Diff