Merged V3.2 to HEAD

16662: LDAP sync: improved group association filtering, referential integrity checking, deletion strategy and performance tuning of batch sizes
   16648: ETHREEOH-2752: Improved ticket validation fix
      - Invalidate user's tickets during person deletion rather than validation or it can mess up chained validation
   16647: ETHREEOH-2534: Fixed Sharepoint NTLM authentication
      - user details were never getting cached in the session
   16579: Small improvement to LDAP error reporting
      - Committed errors counted before successes in a logging interval
   16515: LDAP sync performance
      - Improved full sync strategy - run differential queries to work out required updates/additions and full queries to work out required deletions. Saves updating unchanged nodes.
      - Use a TreeSet rather than a HashSet to gather group associations in an attempt to avoid blowing the heap size
   16498: More LDAP performance improvements
      - Uses thread pool with 4 worker threads and blocking queue to process returned results. The number of worker threads can be controlled by the synchronization.workerThreads property.
      - Switched LDAP connection pooling back on again
      - Group Associations processsed individually so that errors are collated and we get a better idea of their throughput
      - Fixed potential bug. Group membership resolution done with isolated LDAP context to avoid cookies from paging creeping in.
   16424: Try switching off LDAP connection pooling to see if it works better with our flaky server.
   16414: Further LDAP fault tolerance
      - Log causes of group member resolution failures where possible
   16413: More fault tolerance for LDAP sync
      - Always commit last sync times before overall sync is complete to avoid the 'forgetting' of differential sync information
      - DN comparisons should be case insensitive to avoid issues resolving DNs to user and group IDs
   16398: Improved monitoring and fault tolerance for LDAP sync
      - When the batch is complete a summary of the number of errors and the last error stack trace will be logged at ERROR level
      - Each individual error is logged at WARN level and progress information (including % complete) is collated and logged at INFO level after a configurable interval
      - In the Enterprise Edition all metrics can be monitored in real time through JMX
      - Sanity testing to be performed by Mike!
   16319: Merged HEAD to V3.2
      16316: ALFCOM-3397: JBoss 5 compatibility fix
         - Relative paths used by LDAP subsystem configuration weren't being resolved correctly
         - See also https://jira.jboss.org/jira/browse/JBAS-6548 and https://jira.springsource.org/browse/SPR-5120
   16272: ETHREEOH-2752: Once more with feeling!
   16261: ETHREEOH-2752: Correct exception propagation.
   16260: ETHREEOH-2752: Fix ticket validation
      - Current ticket was getting forgotten by previous fix
      - Person validation in CHECK mode now done AFTER the current user is set, so that the current ticket is remembered
   16243: ETHREEOH-2752: Improve ticket validation used by all authentication filters
      - Now takes into account whether person actually exists or not
      - Tickets for non-nonexistent persons are now considered invalid and cached session information is invalidated
      - New BaseAuthenticationFilter superclass for all authentication filters
      - Improved fix to ETHREEOH-2839: WebDAV user is cached consistently using a different session attribute from the Web Client
   16233: ETHREEOH-2754: Correction to previous checkin.
      - relogin for SSO authentication, logout for normal login page
      - logout is default
   16232: ETHREEOH-2754: Log Out Action outcome passed as a parameter
      - relogin for SSO authentication, login for normal login page
      - Means the log out link always leads to the correct place, even when the session has expired
      - Also lowered ticket validation error logging to DEBUG level to avoid unnecessary noise in the logs from expired sessions
   16220: ETHREEOH-2839: Fixed potential ClassCastExceptions when Alfresco accessed via WebDAV and Web Client links in same browser
      - WebDAV side no longer directly casts session user to a WebDAVUser
      - ContextListener no longer casts session user to web client user
      - Web client side will 'promote' session user to a web client User if necessary via AuthenticationHelper
      - All authentication filters made to use appropriate AuthenticationHelper methods
   16211: ETHREEOH-2835: LDAP sync batches user and group deletions as well as creations
      - Also improved logging of sync failures
   16197: ETHREEOH-2782: LDAP subsystems now support search-based user DN resolution
      - When ldap.authentication.userNameFormat isn't set (now the default) converts a user ID to a DN by running ldap.synchronization.personQuery with an extra condition tacked on the end to find the user by ID
      - Structured directories and authentication by attributes not in the DN such as email address now supported
   16189: ALFCOM-3283: Prevent errors when user accepts an invite when not logged in
      - new isGuest attribute propagated to user object
      - header component (used by accept-invite page) needs to avoid calling prefs and site webscripts for guest user
      - Conditional stuff in header template changed to use user.isGuest


git-svn-id: https://svn.alfresco.com/repos/alfresco-enterprise/alfresco/HEAD/root@16896 c4b6b30b-aa2e-2d43-bbcb-ca4b014f7261

config/alfresco/public-services-security-context.xml

@@ -702,6 +702,7 @@
             <value>
                 org.alfresco.service.cmr.security.AuthorityService.hasAdminAuthority=ACL_ALLOW
                 org.alfresco.service.cmr.security.AuthorityService.isAdminAuthority=ACL_ALLOW
+                org.alfresco.service.cmr.security.AuthorityService.isGuestAuthority=ACL_ALLOW
                 org.alfresco.service.cmr.security.AuthorityService.getAuthorities=ACL_ALLOW
 				org.alfresco.service.cmr.security.AuthorityService.getAuthoritiesForUser=ACL_METHOD.ROLE_ADMINISTRATOR
                 org.alfresco.service.cmr.security.AuthorityService.getAllAuthorities=ACL_ALLOW

config/alfresco/subsystems/Authentication/common-ldap-context.xml

@@ -35,17 +35,11 @@
          <ref bean="ldapInitialDirContextFactory" />
       </property>
       <property name="userNameFormat">
-         <!--
-
-            This maps between what the user types in and what is passed through to the underlying LDAP authentication.
-
-            "%s" - the user id is passed through without modification. Used for LDAP authentication such as DIGEST-MD5,
-            anything that is not "simple". "cn=%s,ou=London,dc=company,dc=com" - If the user types in "Joe Bloggs" the
-            authenticate as "cn=Joe Bloggs,ou=London,dc=company,dc=com" Usually for simple authentication. Simple
-            authentication always uses the DN for the user.
-         -->
          <value>${ldap.authentication.userNameFormat}</value>
       </property>
+      <property name="ldapNameResolver">
+         <ref bean="userRegistry" />
+      </property>
       <property name="nodeService">
          <ref bean="nodeService" />
       </property>
@@ -341,6 +335,10 @@
             </map>
         </property>
     
+        <property name="enableProgressEstimation">
+            <value>${ldap.synchronization.enableProgressEstimation}</value>
+        </property>
+
         <!-- Services -->
         <property name="LDAPInitialDirContextFactory">
             <ref bean="ldapInitialDirContextFactory"/>

config/alfresco/subsystems/Authentication/ldap-ad/ldap-ad-authentication.properties

@@ -104,3 +104,6 @@ ldap.synchronization.personType=user
 
 # The attribute in LDAP on group objects that defines the DN for its members
 ldap.synchronization.groupMemberAttributeName=member
+
+# If true progress estimation is enabled. When enabled, the user query has to be run twice in order to count entries.
+ldap.synchronization.enableProgressEstimation=true

config/alfresco/subsystems/Authentication/ldap/ldap-authentication.properties

@@ -7,14 +7,17 @@ ldap.authentication.active=true
 # This properties file brings together the common options for LDAP authentication rather than editing the bean definitions
 #
 ldap.authentication.allowGuestLogin=true
-# How to map the user id entered by the user to taht passed through to LDAP
+# How to map the user id entered by the user to that passed through to LDAP
 # - simple 
 #    - this must be a DN and would be something like
 #      uid=%s,ou=People,dc=company,dc=com
 # - digest
 #    - usually pass through what is entered
 #      %s
-ldap.authentication.userNameFormat=uid\=%s,ou\=People,dc\=company,dc\=com
+# If not set, an LDAP query involving ldap.synchronization.personQuery and ldap.synchronization.userIdAttributeName will 
+# be performed to resolve the DN dynamically. This allows directories to be structured and doesn't require the user ID to
+# appear in the DN.
+ldap.authentication.userNameFormat=
 
 # The LDAP context factory to use
 ldap.authentication.java.naming.factory.initial=com.sun.jndi.ldap.LdapCtxFactory
@@ -107,3 +110,6 @@ ldap.synchronization.personType=inetOrgPerson
 
 # The attribute in LDAP on group objects that defines the DN for its members
 ldap.synchronization.groupMemberAttributeName=member
+
+# If true progress estimation is enabled. When enabled, the user query has to be run twice in order to count entries.
+ldap.synchronization.enableProgressEstimation=true

config/alfresco/subsystems/Synchronization/default/default-synchronization-context.xml

@@ -65,6 +65,12 @@
         <property name="sourceBeanName">
             <value>userRegistry</value>
         </property>
+        <property name="loggingInterval">
+            <value>${synchronization.loggingInterval}</value>
+        </property>
+        <property name="workerThreads">
+            <value>${synchronization.workerThreads}</value>
+        </property>
     </bean>
 
 

config/alfresco/subsystems/Synchronization/default/default-synchronization.properties

@@ -20,3 +20,9 @@ synchronization.syncOnStartup=true
 
 # Should we auto create a missing person on log in?
 synchronization.autoCreatePeopleOnLogin=true
+
+# The number of entries to process before logging progress
+synchronization.loggingInterval=100
+
+# The number of threads to use when doing a batch (scheduled or startup) sync
+synchronization.workerThreads=2

source/java/org/alfresco/repo/jscript/People.java

@@ -712,6 +712,19 @@ public final class People extends BaseScopableProcessorExtension
         return this.authorityService.isAdminAuthority((String)person.getProperties().get(ContentModel.PROP_USERNAME));
     }
 
+    /**
+     * Return true if the specified user is an guest authority.
+     * 
+     * @param person to test
+     * 
+     * @return true if an admin, false otherwise
+     */
+    public boolean isGuest(ScriptNode person)
+    {
+        ParameterCheck.mandatory("Person", person);
+        return this.authorityService.isGuestAuthority((String) person.getProperties().get(ContentModel.PROP_USERNAME));
+    }
+
     /**
      * Get Contained Authorities
      * 

source/java/org/alfresco/repo/security/authentication/AbstractAuthenticationComponent.java

@@ -191,19 +191,12 @@ public abstract class AbstractAuthenticationComponent implements AuthenticationC
         throw new UnsupportedOperationException();
     }
 
-    public Authentication setCurrentUser(String userName, UserNameValidationMode validationMode)
+    public Authentication setCurrentUser(final String userName) throws AuthenticationException
     {
-        switch (validationMode)
-        {
-        case NONE:
-            return setCurrentUserImpl(userName);
-        case CHECK_AND_FIX:
-        default:
-            return setCurrentUser(userName);
-        }
+        return setCurrentUser(userName, UserNameValidationMode.CHECK_AND_FIX);
     }
 
-    public Authentication setCurrentUser(final String userName) throws AuthenticationException
+    public Authentication setCurrentUser(String userName, UserNameValidationMode validationMode)
     {
         if (isSystemUserName(userName))
         {
@@ -211,29 +204,33 @@ public abstract class AbstractAuthenticationComponent implements AuthenticationC
         }
         else
         {
-            SetCurrentUserCallback callback = new SetCurrentUserCallback(userName);
-            Authentication auth;
-            // If the repository is read only, we have to settle for a read only transaction. Auto user creation will
-            // not be possible.
+            CurrentUserCallback callback = validationMode == UserNameValidationMode.CHECK_AND_FIX ? new FixCurrentUserCallback(
+                    userName)
+                    : new CheckCurrentUserCallback(userName);
+            Authentication authentication;
+            // If the repository is read only, we have to settle for a read only transaction. Auto user creation
+            // will not be possible.
             if (transactionService.isReadOnly())
             {
-                auth = transactionService.getRetryingTransactionHelper().doInTransaction(callback, true, false);
+                authentication = transactionService.getRetryingTransactionHelper().doInTransaction(callback, true,
+                        false);
             }
             // Otherwise, we want a writeable transaction, so if the current transaction is read only we set the
             // requiresNew flag to true
             else
             {
-                auth = transactionService.getRetryingTransactionHelper().doInTransaction(callback, false,
+                authentication = transactionService.getRetryingTransactionHelper().doInTransaction(callback, false,
                         AlfrescoTransactionSupport.getTransactionReadState() == TxnReadState.TXN_READ_ONLY);
             }
-            if ((auth == null) || (callback.ae != null))
+            if ((authentication == null) || (callback.ae != null))
             {
                 throw callback.ae;
             }
-            return auth;
+            return authentication;
         }
     }
     
+
     /**
      * Explicitly set the current user to be authenticated.
      * 
@@ -451,22 +448,72 @@ public abstract class AbstractAuthenticationComponent implements AuthenticationC
         authenticationContext.clearCurrentSecurityContext();
     }
 
-    class SetCurrentUserCallback implements RetryingTransactionHelper.RetryingTransactionCallback<Authentication>
+    abstract class CurrentUserCallback implements RetryingTransactionHelper.RetryingTransactionCallback<Authentication>
     {
         AuthenticationException ae = null;
 
         String userName;
 
-        SetCurrentUserCallback(String userName)
+        CurrentUserCallback(String userName)
         {
             this.userName = userName;
         }
+    }
+
+    class CheckCurrentUserCallback extends CurrentUserCallback
+    {
+
+        CheckCurrentUserCallback(String userName)
+        {
+            super(userName);
+        }
 
         public Authentication execute() throws Throwable
         {
             try
             {
-                String name = AuthenticationUtil.runAs(new RunAsWork<String>()
+                // We must set full authentication before calling runAs in order to retain tickets
+                Authentication authentication = setCurrentUserImpl(userName);
+                AuthenticationUtil.runAs(new RunAsWork<Object>()
+                {
+                    public Object doWork() throws Exception
+                    {
+                        if (!personService.personExists(userName)
+                                || !nodeService.getProperty(personService.getPerson(userName),
+                                        ContentModel.PROP_USERNAME).equals(userName))
+                        {
+                            if (logger.isDebugEnabled())
+                            {
+                                logger.debug("User \"" + userName
+                                        + "\" does not exist in Alfresco. Failing validation.");
+                            }
+                            throw new AuthenticationException("User \"" + userName + "\" does not exist in Alfresco");
+                        }
+                        return null;
+                    }
+                }, getSystemUserName(getUserDomain(userName)));
+                return authentication;
+            }
+            catch (AuthenticationException ae)
+            {
+                this.ae = ae;
+                return null;
+            }
+        }
+    }
+    
+    class FixCurrentUserCallback extends CurrentUserCallback
+    {
+        FixCurrentUserCallback(String userName)
+        {
+            super(userName);
+        }
+
+        public Authentication execute() throws Throwable
+        {
+            try
+            {
+                return setCurrentUserImpl(AuthenticationUtil.runAs(new RunAsWork<String>()
                 {
                     public String doWork() throws Exception
                     {
@@ -492,9 +539,7 @@ public abstract class AbstractAuthenticationComponent implements AuthenticationC
                         // checks
                         return (String) nodeService.getProperty(userNode, ContentModel.PROP_USERNAME);
                     }
-                }, getSystemUserName(getUserDomain(userName)));
-
-                return setCurrentUserImpl(name);
+                }, getSystemUserName(getUserDomain(userName))));
             }
             catch (AuthenticationException ae)
             {

source/java/org/alfresco/repo/security/authentication/AuthenticationComponent.java

@@ -32,7 +32,7 @@ public interface AuthenticationComponent extends AuthenticationContext
 {
     public enum UserNameValidationMode
     {
-        NONE, CHECK_AND_FIX;
+        CHECK, CHECK_AND_FIX;
     }
     
     /**

source/java/org/alfresco/repo/security/authentication/AuthenticationServiceImpl.java

@@ -174,14 +174,16 @@ public class AuthenticationServiceImpl extends AbstractAuthenticationService imp
 
     public void validate(String ticket) throws AuthenticationException
     {
+        String currentUser = null;
         try
         {
 
             // clear context - to avoid MT concurrency issue (causing domain mismatch) - see also 'authenticate' above
             clearCurrentSecurityContext();
-           authenticationComponent.setCurrentUser(ticketComponent.validateTicket(ticket), UserNameValidationMode.NONE);
+            currentUser = ticketComponent.validateTicket(ticket);
+            authenticationComponent.setCurrentUser(currentUser, UserNameValidationMode.CHECK);
         }
-        catch(AuthenticationException ae)
+        catch (AuthenticationException ae)
         {
             clearCurrentSecurityContext();
             throw ae;

source/java/org/alfresco/repo/security/authentication/ldap/LDAPAuthenticationComponentImpl.java

@@ -30,13 +30,17 @@ import javax.naming.directory.InitialDirContext;
 import org.alfresco.repo.management.subsystems.ActivateableBean;
 import org.alfresco.repo.security.authentication.AbstractAuthenticationComponent;
 import org.alfresco.repo.security.authentication.AuthenticationException;
+import org.alfresco.repo.security.sync.ldap.LDAPNameResolver;
+import org.springframework.beans.factory.InitializingBean;
 
 /**
- * Currently expects the cn name of the user which is in a fixed location.
+ * Authenticates a user by LDAP. To convert the user name to an LDAP DN, it uses the fixed format in
+ * <code>userNameFormat</code> if set, or calls the {@link LDAPNameResolver} otherwise.
  * 
  * @author Andy Hind
  */
-public class LDAPAuthenticationComponentImpl extends AbstractAuthenticationComponent implements ActivateableBean
+public class LDAPAuthenticationComponentImpl extends AbstractAuthenticationComponent implements InitializingBean,
+        ActivateableBean
 {
     private boolean escapeCommasInBind = false;
     
@@ -46,6 +50,8 @@ public class LDAPAuthenticationComponentImpl extends AbstractAuthenticationCompo
 
     private String userNameFormat;
     
+    private LDAPNameResolver ldapNameResolver;
+
     private LDAPInitialDirContextFactory ldapInitialContextFactory;
 
     public LDAPAuthenticationComponentImpl()
@@ -60,7 +66,12 @@ public class LDAPAuthenticationComponentImpl extends AbstractAuthenticationCompo
 
     public void setUserNameFormat(String userNameFormat)
     {
-        this.userNameFormat = userNameFormat;
+        this.userNameFormat = userNameFormat == null || userNameFormat.length() == 0 ? null : userNameFormat;
+    }
+        
+    public void setLdapNameResolver(LDAPNameResolver ldapNameResolver)
+    {
+        this.ldapNameResolver = ldapNameResolver;
     }
 
     public void setEscapeCommasInBind(boolean escapeCommasInBind)
@@ -87,15 +98,34 @@ public class LDAPAuthenticationComponentImpl extends AbstractAuthenticationCompo
         return this.active;
     }        
 
+    /*
+     * (non-Javadoc)
+     * @see org.springframework.beans.factory.InitializingBean#afterPropertiesSet()
+     */
+    public void afterPropertiesSet() throws Exception
+    {
+        if (this.ldapNameResolver == null && this.userNameFormat == null)
+        {
+            throw new IllegalStateException("At least one of ldapNameResolver and userNameFormat must be set");
+        }
+    }
+
     /**
      * Implement the authentication method
      */
     protected void authenticateImpl(String userName, char[] password) throws AuthenticationException
     {
+        // If we aren't using a fixed name format, do a search to resolve the user DN
+        String userDN = userNameFormat == null ? ldapNameResolver.resolveDistinguishedName(userName) : String.format(
+                userNameFormat, new Object[]
+                {
+                    escapeUserName(userName, escapeCommasInBind)
+                });
+
         InitialDirContext ctx = null;
         try
         {
-            ctx = ldapInitialContextFactory.getInitialDirContext(String.format(userNameFormat, new Object[] { escapeUserName(userName, escapeCommasInBind) }), new String(password));
+            ctx = ldapInitialContextFactory.getInitialDirContext(userDN, new String(password));
 
             // Authentication has been successful.
             // Set the current user, they are now authenticated.

source/java/org/alfresco/repo/security/authentication/ldap/LDAPInitialDirContextFactoryImpl.java

@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2005-2007 Alfresco Software Limited.
+ * Copyright (C) 2005-2009 Alfresco Software Limited.
  *
  * This program is free software; you can redistribute it and/or
  * modify it under the terms of the GNU General Public License
@@ -87,6 +87,7 @@ public class LDAPInitialDirContextFactoryImpl implements LDAPInitialDirContextFa
         Hashtable<String, String> env = new Hashtable<String, String>(initialDirContextEnvironment.size());
         env.putAll(initialDirContextEnvironment);
         env.put("javax.security.auth.useSubjectCredsOnly", "false");
+        env.put("com.sun.jndi.ldap.connect.pool", "true"); // Pool the default connection
         return buildInitialDirContext(env, pageSize);
     }
 

source/java/org/alfresco/repo/security/person/PersonServiceImpl.java

@@ -671,6 +671,16 @@ public class PersonServiceImpl extends TransactionListenerAdapter implements Per
             // Ignore this - externally authenticated user
         }
         
+        // Invalidate all that user's tickets
+        try
+        {
+            authenticationService.invalidateUserSession(userName);
+        }
+        catch (AuthenticationException e)
+        {
+            // Ignore this
+        }
+
         // remove user from any containing authorities
         Set<String> containerAuthorities = authorityService.getContainingAuthorities(null, userName, true);
         for (String containerAuthority : containerAuthorities)

source/java/org/alfresco/repo/security/sync/BatchMonitor.java

@@ -0,0 +1,105 @@
+/*
+ * Copyright (C) 2005-2009 Alfresco Software Limited.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version 2
+ * of the License, or (at your option) any later version.
+
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+
+ * As a special exception to the terms and conditions of version 2.0 of 
+ * the GPL, you may redistribute this Program in connection with Free/Libre 
+ * and Open Source Software ("FLOSS") applications as described in Alfresco's 
+ * FLOSS exception.  You should have received a copy of the text describing 
+ * the FLOSS exception, and it is also available here: 
+ * http://www.alfresco.com/legal/licensing"
+ */
+package org.alfresco.repo.security.sync;
+
+import java.util.Date;
+
+/**
+ * An interface that allows the monitoring of metrics relating to a potentially long-running batch process.
+ * 
+ * @author dward
+ */
+public interface BatchMonitor
+{
+    /**
+     * Gets the process name.
+     * 
+     * @return the process name
+     */
+    public String getProcessName();
+
+    /**
+     * Gets the start time.
+     * 
+     * @return the start time
+     */
+    public Date getStartTime();
+
+    /**
+     * Gets the total number of results.
+     * 
+     * @return the total number of results
+     */
+    public int getTotalResults();
+
+    /**
+     * Gets the ID of the entry being processed
+     * 
+     * @return the current entry id
+     */
+    public String getCurrentEntryId();
+
+    /**
+     * Gets the number of successfully processed entries.
+     * 
+     * @return the successfully processed entries
+     */
+    public int getSuccessfullyProcessedEntries();
+
+    /**
+     * Gets the progress expressed as a percentage.
+     * 
+     * @return the progress expressed as a percentage
+     */
+    public String getPercentComplete();
+
+    /**
+     * Gets the total number of errors.
+     * 
+     * @return the total number of errors
+     */
+    public int getTotalErrors();
+
+    /**
+     * Gets the stack trace of the last error.
+     * 
+     * @return the stack trace of the last error
+     */
+    public String getLastError();
+
+    /**
+     * Gets the entry id that caused the last error.
+     * 
+     * @return the last error entry id
+     */
+    public String getLastErrorEntryId();
+
+    /**
+     * Gets the end time.
+     * 
+     * @return the end time
+     */
+    public Date getEndTime();
+}

source/java/org/alfresco/repo/security/sync/BatchMonitorEvent.java

@@ -0,0 +1,60 @@
+/*
+ * Copyright (C) 2005-2009 Alfresco Software Limited.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version 2
+ * of the License, or (at your option) any later version.
+
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+
+ * As a special exception to the terms and conditions of version 2.0 of 
+ * the GPL, you may redistribute this Program in connection with Free/Libre 
+ * and Open Source Software ("FLOSS") applications as described in Alfresco's 
+ * FLOSS exception.  You should have received a copy of the text describing 
+ * the FLOSS exception, and it is also available here: 
+ * http://www.alfresco.com/legal/licensing"
+ */
+package org.alfresco.repo.security.sync;
+
+import org.springframework.context.ApplicationEvent;
+
+/**
+ * An event alerting listeners to the existence of a new {@link BatchMonitor}.
+ * 
+ * @author dward
+ */
+public class BatchMonitorEvent extends ApplicationEvent
+{
+
+    private static final long serialVersionUID = -5787104103292355106L;
+
+    /**
+     * The Constructor.
+     * 
+     * @param source
+     *            the source of the event
+     */
+    public BatchMonitorEvent(BatchMonitor source)
+    {
+        super(source);
+    }
+
+    /**
+     * Gets the source batch monitor.
+     * 
+     * @return the batch monitor
+     */
+    public BatchMonitor getBatchMonitor()
+    {
+        return (BatchMonitor) getSource();
+    }
+
+}

source/java/org/alfresco/repo/security/sync/BatchProcessor.java

@@ -0,0 +1,608 @@
+/*
+ * Copyright (C) 2005-2009 Alfresco Software Limited.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version 2
+ * of the License, or (at your option) any later version.
+
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+
+ * As a special exception to the terms and conditions of version 2.0 of 
+ * the GPL, you may redistribute this Program in connection with Free/Libre 
+ * and Open Source Software ("FLOSS") applications as described in Alfresco's 
+ * FLOSS exception.  You should have received a copy of the text describing 
+ * the FLOSS exception, and it is also available here: 
+ * http://www.alfresco.com/legal/licensing"
+ */
+package org.alfresco.repo.security.sync;
+
+import java.io.PrintWriter;
+import java.io.StringWriter;
+import java.io.Writer;
+import java.text.NumberFormat;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Date;
+import java.util.Iterator;
+import java.util.List;
+import java.util.concurrent.ArrayBlockingQueue;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.ThreadPoolExecutor;
+import java.util.concurrent.TimeUnit;
+
+import org.alfresco.error.AlfrescoRuntimeException;
+import org.alfresco.repo.transaction.RetryingTransactionHelper;
+import org.alfresco.repo.transaction.RetryingTransactionHelper.RetryingTransactionCallback;
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+import org.springframework.context.ApplicationEventPublisher;
+
+/**
+ * A <code>BatchProcessor</code> manages the running and monitoring of a potentially long-running transactional batch
+ * process. It iterates over a collection, and queues jobs that fire a worker on a batch of members. The queued jobs
+ * handle progress / error reporting, transaction delineation and retrying. They are processed in parallel by a pool of
+ * threads of a configurable size. The job processing is designed to be fault tolerant and will continue in the event of
+ * errors. When the batch is complete a summary of the number of errors and the last error stack trace will be logged at
+ * ERROR level. Each individual error is logged at WARN level and progress information is logged at INFO level. Through
+ * the {@link BatchMonitor} interface, it also supports the real-time monitoring of batch metrics (e.g. over JMX in the
+ * Enterprise Edition).
+ * 
+ * @author dward
+ */
+public class BatchProcessor<T> implements BatchMonitor
+{
+    /** Let's share the logger of ChainingUserRegistrySynchronizer. */
+    private static final Log logger = LogFactory.getLog(ChainingUserRegistrySynchronizer.class);
+
+    /** The retrying transaction helper. */
+    private RetryingTransactionHelper retryingTransactionHelper;
+
+    /** The collection. */
+    private Collection<T> collection;
+
+    /** The process name. */
+    private String processName;
+
+    /** The number of entries to process before reporting progress. */
+    private int loggingInterval;
+
+    /** The number of worker threads. */
+    private int workerThreads;
+
+    /** The number of entries we process at a time in a transaction *. */
+    private final int batchSize;
+
+    /** The current entry id. */
+    private String currentEntryId;
+
+    /** The last error. */
+    private Throwable lastError;
+
+    /** The last error entry id. */
+    private String lastErrorEntryId;
+
+    /** The total number of errors. */
+    private int totalErrors;
+
+    /** The number of successfully processed entries. */
+    private int successfullyProcessedEntries;
+
+    /** The start time. */
+    private Date startTime;
+
+    /** The end time. */
+    private Date endTime;
+
+    /**
+     * Instantiates a new batch processor.
+     * 
+     * @param retryingTransactionHelper
+     *            the retrying transaction helper
+     * @param collection
+     *            the collection
+     * @param processName
+     *            the process name
+     * @param loggingInterval
+     *            the number of entries to process before reporting progress
+     * @param applicationEventPublisher
+     *            the application event publisher
+     * @param workerThreads
+     *            the number of worker threads
+     * @param batchSize
+     *            the number of entries we process at a time in a transaction
+     */
+    public BatchProcessor(RetryingTransactionHelper retryingTransactionHelper,
+            ApplicationEventPublisher applicationEventPublisher, Collection<T> collection, String processName,
+            int loggingInterval, int workerThreads, int batchSize)
+    {
+        this.retryingTransactionHelper = retryingTransactionHelper;
+        this.collection = collection;
+        this.processName = processName;
+        this.loggingInterval = loggingInterval;
+        this.workerThreads = workerThreads;
+        this.batchSize = batchSize;
+        // Let the (enterprise) monitoring side know of our presence
+        applicationEventPublisher.publishEvent(new BatchMonitorEvent(this));
+    }
+
+    /*
+     * (non-Javadoc)
+     * @see org.alfresco.repo.security.sync.BatchMonitor#getCurrentEntryId()
+     */
+    public synchronized String getCurrentEntryId()
+    {
+        return this.currentEntryId;
+    }
+
+    /*
+     * (non-Javadoc)
+     * @see org.alfresco.repo.security.sync.BatchMonitor#getLastError()
+     */
+    public synchronized String getLastError()
+    {
+        if (this.lastError == null)
+        {
+            return null;
+        }
+        Writer buff = new StringWriter(1024);
+        PrintWriter out = new PrintWriter(buff);
+        this.lastError.printStackTrace(out);
+        out.close();
+        return buff.toString();
+    }
+
+    /*
+     * (non-Javadoc)
+     * @see org.alfresco.repo.security.sync.BatchMonitor#getLastErrorEntryId()
+     */
+    public synchronized String getLastErrorEntryId()
+    {
+        return this.lastErrorEntryId;
+    }
+
+    /*
+     * (non-Javadoc)
+     * @see org.alfresco.repo.security.sync.BatchMonitor#getBatchType()
+     */
+    public synchronized String getProcessName()
+    {
+        return this.processName;
+    }
+
+    /*
+     * (non-Javadoc)
+     * @see org.alfresco.repo.security.sync.BatchMonitor#getSuccessfullyProcessedResults()
+     */
+    public synchronized int getSuccessfullyProcessedEntries()
+    {
+        return this.successfullyProcessedEntries;
+    }
+
+    /*
+     * (non-Javadoc)
+     * @see org.alfresco.repo.security.sync.BatchMonitor#getPercentComplete()
+     */
+    public synchronized String getPercentComplete()
+    {
+        int totalResults = this.collection.size();
+        int processed = this.successfullyProcessedEntries + this.totalErrors;
+        return processed <= totalResults ? NumberFormat.getPercentInstance().format(
+                totalResults == 0 ? 1.0F : (float) processed / totalResults) : "Unknown";
+    }
+
+    /*
+     * (non-Javadoc)
+     * @see org.alfresco.repo.security.sync.BatchMonitor#getTotalErrors()
+     */
+    public synchronized int getTotalErrors()
+    {
+        return this.totalErrors;
+    }
+
+    /*
+     * (non-Javadoc)
+     * @see org.alfresco.repo.security.sync.BatchMonitor#getTotalResults()
+     */
+    public int getTotalResults()
+    {
+        return this.collection.size();
+    }
+
+    /*
+     * (non-Javadoc)
+     * @see org.alfresco.repo.security.sync.BatchMonitor#getEndTime()
+     */
+    public synchronized Date getEndTime()
+    {
+        return this.endTime;
+    }
+
+    /*
+     * (non-Javadoc)
+     * @see org.alfresco.repo.security.sync.BatchMonitor#getStartTime()
+     */
+    public synchronized Date getStartTime()
+    {
+        return this.startTime;
+    }
+
+    /**
+     * Invokes the worker for each entry in the collection, managing transactions and collating success / failure
+     * information.
+     * 
+     * @param worker
+     *            the worker
+     * @param splitTxns
+     *            Can the modifications to Alfresco be split across multiple transactions for maximum performance? If
+     *            <code>true</code>, worker invocations are isolated in separate transactions in batches of 10 for
+     *            increased performance. If <code>false</code>, all invocations are performed in the current
+     *            transaction. This is required if calling synchronously (e.g. in response to an authentication event in
+     *            the same transaction).
+     * @return the number of invocations
+     */
+    @SuppressWarnings("serial")
+    public int process(final Worker<T> worker, final boolean splitTxns)
+    {
+        int count = this.collection.size();
+        synchronized (this)
+        {
+            this.startTime = new Date();
+            if (BatchProcessor.logger.isInfoEnabled())
+            {
+                if (count >= 0)
+                {
+                    BatchProcessor.logger.info(getProcessName() + ": Commencing batch of " + count + " entries");
+                }
+                else
+                {
+                    BatchProcessor.logger.info(getProcessName() + ": Commencing batch");
+
+                }
+            }
+        }
+
+        // Create a thread pool executor with the specified number of threads and a finite blocking queue of jobs
+        ExecutorService executorService = splitTxns && this.workerThreads > 1 ? new ThreadPoolExecutor(
+                this.workerThreads, this.workerThreads, 0L, TimeUnit.MILLISECONDS, new ArrayBlockingQueue<Runnable>(
+                        this.workerThreads * 10)
+                {
+                    // Add blocking behaviour to work queue
+                    @Override
+                    public boolean offer(Runnable o)
+                    {
+                        try
+                        {
+                            put(o);
+                        }
+                        catch (InterruptedException e)
+                        {
+                            return false;
+                        }
+                        return true;
+                    }
+
+                }) : null;
+        try
+        {
+            Iterator<T> iterator = this.collection.iterator();
+            List<T> batch = new ArrayList<T>(this.batchSize);
+            while (iterator.hasNext())
+            {
+                batch.add(iterator.next());
+                boolean hasNext = iterator.hasNext();
+                if (batch.size() >= this.batchSize || !hasNext)
+                {
+                    final TxnCallback callback = new TxnCallback(worker, batch, splitTxns);
+                    if (hasNext)
+                    {
+                        batch = new ArrayList<T>(this.batchSize);
+                    }
+                    if (executorService == null)
+                    {
+                        callback.run();
+                    }
+                    else
+                    {
+                        executorService.execute(callback);
+                    }
+                }
+            }
+            return count;
+        }
+        finally
+        {
+            if (executorService != null)
+            {
+                executorService.shutdown();
+                try
+                {
+                    executorService.awaitTermination(Long.MAX_VALUE, TimeUnit.SECONDS);
+                }
+                catch (InterruptedException e)
+                {
+                }
+            }
+            synchronized (this)
+            {
+                reportProgress(true);
+                this.endTime = new Date();
+                if (BatchProcessor.logger.isInfoEnabled())
+                {
+                    if (count >= 0)
+                    {
+                        BatchProcessor.logger.info(getProcessName() + ": Completed batch of " + count + " entries");
+                    }
+                    else
+                    {
+                        BatchProcessor.logger.info(getProcessName() + ": Completed batch");
+
+                    }
+                }
+                if (this.totalErrors > 0 && BatchProcessor.logger.isErrorEnabled())
+                {
+                    BatchProcessor.logger.error(getProcessName() + ": " + this.totalErrors
+                            + " error(s) detected. Last error from entry \"" + this.lastErrorEntryId + "\"",
+                            this.lastError);
+                }
+            }
+        }
+    }
+
+    /**
+     * Reports the current progress.
+     * 
+     * @param last
+     *            Have all jobs been processed? If <code>false</code> then progress is only reported after the number of
+     *            entries indicated by {@link #loggingInterval}. If <code>true</code> then progress is reported if this
+     *            is not one of the entries indicated by {@link #loggingInterval}.
+     */
+    private synchronized void reportProgress(boolean last)
+    {
+        int processed = this.successfullyProcessedEntries + this.totalErrors;
+        if (processed % this.loggingInterval == 0 ^ last)
+        {
+            StringBuilder message = new StringBuilder(100).append(getProcessName()).append(": Processed ").append(
+                    processed).append(" entries");
+            int totalResults = this.collection.size();
+            if (totalResults >= processed)
+            {
+                message.append(" out of ").append(totalResults).append(". ").append(
+                        NumberFormat.getPercentInstance().format(
+                                totalResults == 0 ? 1.0F : (float) processed / totalResults)).append(" complete");
+            }
+            long duration = System.currentTimeMillis() - startTime.getTime();
+            if (duration > 0)
+            {
+                message.append(". Rate: ").append(processed * 1000 / duration).append(" per second");
+            }
+            message.append(". " + this.totalErrors + " failures detected.");
+            BatchProcessor.logger.info(message);
+        }
+    }
+
+    /**
+     * An interface for workers to be invoked by the {@link BatchProcessor}.
+     */
+    public interface Worker<T>
+    {
+
+        /**
+         * Gets an identifier for the given entry (for monitoring / logging purposes).
+         * 
+         * @param entry
+         *            the entry
+         * @return the identifier
+         */
+        public String getIdentifier(T entry);
+
+        /**
+         * Processes the given entry.
+         * 
+         * @param entry
+         *            the entry
+         * @throws Throwable
+         *             on any error
+         */
+        public void process(T entry) throws Throwable;
+    }
+
+    /**
+     * A callback that invokes a worker on a batch, optionally in a new transaction.
+     */
+    class TxnCallback implements RetryingTransactionCallback<Object>, Runnable
+    {
+
+        /**
+         * Instantiates a new callback.
+         * 
+         * @param worker
+         *            the worker
+         * @param batch
+         *            the batch to process
+         * @param splitTxns
+         *            If <code>true</code>, the worker invocation is made in a new transaction.
+         */
+        public TxnCallback(Worker<T> worker, List<T> batch, boolean splitTxns)
+        {
+            this.worker = worker;
+            this.batch = batch;
+            this.splitTxns = splitTxns;
+        }
+
+        /** The worker. */
+        private final Worker<T> worker;
+
+        /** The batch. */
+        private final List<T> batch;
+
+        /** If <code>true</code>, the worker invocation is made in a new transaction. */
+        private final boolean splitTxns;
+
+        /** The total number of errors. */
+        private int txnErrors;
+
+        /** The number of successfully processed entries. */
+        private int txnSuccesses;
+
+        /** The last error. */
+        private Throwable txnLastError;
+
+        /** The last error entry id. */
+        private String txnLastErrorEntryId;
+
+        /*
+         * (non-Javadoc)
+         * @see org.alfresco.repo.transaction.RetryingTransactionHelper.RetryingTransactionCallback#execute ()
+         */
+        public Object execute() throws Throwable
+        {
+            reset();
+            for (T entry : this.batch)
+            {
+                String txnEntryId = this.worker.getIdentifier(entry);
+                synchronized (BatchProcessor.this)
+                {
+                    BatchProcessor.this.currentEntryId = txnEntryId;
+                }
+                try
+                {
+                    this.worker.process(entry);
+                    this.txnSuccesses++;
+                }
+                catch (Throwable t)
+                {
+                    this.txnLastError = t;
+                    this.txnLastErrorEntryId = txnEntryId;
+                    this.txnErrors++;
+                    if (RetryingTransactionHelper.extractRetryCause(t) == null)
+                    {
+                        if (BatchProcessor.logger.isWarnEnabled())
+                        {
+                            BatchProcessor.logger.warn(getProcessName() + ": Failed to process entry \"" + txnEntryId
+                                    + "\".", t);
+                        }
+                    }
+                    else
+                    {
+                        throw t;
+                    }
+                }
+            }
+            return null;
+        }
+
+        /*
+         * (non-Javadoc)
+         * @see java.lang.Runnable#run()
+         */
+        public void run()
+        {
+            try
+            {
+                BatchProcessor.this.retryingTransactionHelper.doInTransaction(this, false, this.splitTxns);
+            }
+            catch (Throwable t)
+            {
+                // If the callback was in its own transaction, it must have run out of retries
+                if (this.splitTxns)
+                {
+                    if (BatchProcessor.logger.isWarnEnabled())
+                    {
+                        BatchProcessor.logger.warn(getProcessName() + ": Failed to process entry \""
+                                + BatchProcessor.this.currentEntryId + "\".", t);
+                    }
+                }
+                // Otherwise, we have a retryable exception that we should propagate
+                else
+                {
+                    if (t instanceof RuntimeException)
+                    {
+                        throw (RuntimeException) t;
+                    }
+                    if (t instanceof Error)
+                    {
+                        throw (Error) t;
+                    }
+                    throw new AlfrescoRuntimeException("Transactional error during " + getProcessName(), t);
+                }
+            }
+            commitProgress();
+        }
+
+        /**
+         * Resets the callback state for a retry.
+         */
+        private void reset()
+        {
+            this.txnLastError = null;
+            this.txnLastErrorEntryId = null;
+            this.txnSuccesses = this.txnErrors = 0;
+        }
+
+        /**
+         * Commits progress from this transaction after a successful commit.
+         */
+        private void commitProgress()
+        {
+            synchronized (BatchProcessor.this)
+            {
+                if (this.txnErrors > 0)
+                {
+                    int processed = BatchProcessor.this.successfullyProcessedEntries + BatchProcessor.this.totalErrors;
+                    int currentIncrement = processed % BatchProcessor.this.loggingInterval;
+                    int newErrors = BatchProcessor.this.totalErrors + this.txnErrors;
+                    // Work out the number of logging intervals we will cross and report them
+                    int intervals = (this.txnErrors + currentIncrement) / BatchProcessor.this.loggingInterval;
+                    if (intervals > 0)
+                    {
+                        BatchProcessor.this.totalErrors += BatchProcessor.this.loggingInterval - currentIncrement;
+                        reportProgress(false);
+                        while (--intervals > 0)
+                        {
+                            BatchProcessor.this.totalErrors += BatchProcessor.this.loggingInterval;
+                            reportProgress(false);
+                        }
+                    }
+                    BatchProcessor.this.totalErrors = newErrors;
+                }
+
+                if (this.txnSuccesses > 0)
+                {
+                    int processed = BatchProcessor.this.successfullyProcessedEntries + BatchProcessor.this.totalErrors;
+                    int currentIncrement = processed % BatchProcessor.this.loggingInterval;
+                    int newSuccess = BatchProcessor.this.successfullyProcessedEntries + this.txnSuccesses;
+                    // Work out the number of logging intervals we will cross and report them
+                    int intervals = (this.txnSuccesses + currentIncrement) / BatchProcessor.this.loggingInterval;
+                    if (intervals > 0)
+                    {
+                        BatchProcessor.this.successfullyProcessedEntries += BatchProcessor.this.loggingInterval
+                                - currentIncrement;
+                        reportProgress(false);
+                        while (--intervals > 0)
+                        {
+                            BatchProcessor.this.successfullyProcessedEntries += BatchProcessor.this.loggingInterval;
+                            reportProgress(false);
+                        }
+                    }
+                    BatchProcessor.this.successfullyProcessedEntries = newSuccess;
+                }
+
+                if (this.txnLastError != null)
+                {
+                    BatchProcessor.this.lastError = this.txnLastError;
+                    BatchProcessor.this.lastErrorEntryId = this.txnLastErrorEntryId;
+                }
+                reset();
+            }
+        }
+    }
+
+}

source/java/org/alfresco/repo/security/sync/ChainingUserRegistrySynchronizer.java

@@ -26,12 +26,11 @@ package org.alfresco.repo.security.sync;
 
 import java.text.DateFormat;
 import java.util.Collection;
+import java.util.Collections;
+import java.util.Comparator;
 import java.util.Date;
 import java.util.HashSet;
-import java.util.Iterator;
-import java.util.Map;
 import java.util.Set;
-import java.util.TreeMap;
 import java.util.TreeSet;
 
 import org.alfresco.model.ContentModel;
@@ -44,6 +43,7 @@ import org.alfresco.repo.management.subsystems.ActivateableBean;
 import org.alfresco.repo.management.subsystems.ChildApplicationContextManager;
 import org.alfresco.repo.security.authentication.AuthenticationUtil;
 import org.alfresco.repo.security.authentication.AuthenticationUtil.RunAsWork;
+import org.alfresco.repo.security.sync.BatchProcessor.Worker;
 import org.alfresco.repo.transaction.AlfrescoTransactionSupport;
 import org.alfresco.repo.transaction.RetryingTransactionHelper;
 import org.alfresco.repo.transaction.AlfrescoTransactionSupport.TxnReadState;
@@ -55,12 +55,15 @@ import org.alfresco.service.cmr.security.PersonService;
 import org.alfresco.service.namespace.NamespaceService;
 import org.alfresco.service.namespace.QName;
 import org.alfresco.util.AbstractLifecycleBean;
+import org.alfresco.util.Pair;
 import org.alfresco.util.PropertyMap;
 import org.apache.commons.logging.Log;
 import org.apache.commons.logging.LogFactory;
 import org.springframework.beans.factory.NoSuchBeanDefinitionException;
 import org.springframework.context.ApplicationContext;
 import org.springframework.context.ApplicationEvent;
+import org.springframework.context.ApplicationEventPublisher;
+import org.springframework.context.ApplicationEventPublisherAware;
 
 /**
  * A <code>ChainingUserRegistrySynchronizer</code> is responsible for synchronizing Alfresco's local user (person) and
@@ -87,12 +90,10 @@ import org.springframework.context.ApplicationEvent;
  * 
  * @author dward
  */
-public class ChainingUserRegistrySynchronizer extends AbstractLifecycleBean implements UserRegistrySynchronizer
+public class ChainingUserRegistrySynchronizer extends AbstractLifecycleBean implements UserRegistrySynchronizer,
+        ApplicationEventPublisherAware
 {
 
-    /** The number of users / groups we add at a time in a transaction *. */
-    private static final int BATCH_SIZE = 10;
-
     /** The logger. */
     private static final Log logger = LogFactory.getLog(ChainingUserRegistrySynchronizer.class);
 
@@ -133,6 +134,9 @@ public class ChainingUserRegistrySynchronizer extends AbstractLifecycleBean impl
     /** The job lock service. */
     private JobLockService jobLockService;
 
+    /** The application event publisher. */
+    private ApplicationEventPublisher applicationEventPublisher;
+
     /** Should we trigger a differential sync when missing people log in?. */
     private boolean syncWhenMissingPeopleLogIn = true;
 
@@ -142,6 +146,12 @@ public class ChainingUserRegistrySynchronizer extends AbstractLifecycleBean impl
     /** Should we auto create a missing person on log in?. */
     private boolean autoCreatePeopleOnLogin = true;
 
+    /** The number of entries to process before reporting progress. */
+    private int loggingInterval = 100;
+
+    /** The number of worker threads. */
+    private int workerThreads = 2;
+
     /**
      * Sets the application context manager.
      * 
@@ -219,6 +229,17 @@ public class ChainingUserRegistrySynchronizer extends AbstractLifecycleBean impl
         this.jobLockService = jobLockService;
     }
 
+    /*
+     * (non-Javadoc)
+     * @see
+     * org.springframework.context.ApplicationEventPublisherAware#setApplicationEventPublisher(org.springframework.context
+     * .ApplicationEventPublisher)
+     */
+    public void setApplicationEventPublisher(ApplicationEventPublisher applicationEventPublisher)
+    {
+        this.applicationEventPublisher = applicationEventPublisher;
+    }
+
     /**
      * Controls whether we auto create a missing person on log in.
      * 
@@ -252,11 +273,36 @@ public class ChainingUserRegistrySynchronizer extends AbstractLifecycleBean impl
         this.syncOnStartup = syncOnStartup;
     }
 
+    /**
+     * Sets the number of entries to process before reporting progress.
+     * 
+     * @param loggingInterval
+     *            the number of entries to process before reporting progress or zero to disable progress reporting.
+     */
+    public void setLoggingInterval(int loggingInterval)
+    {
+        this.loggingInterval = loggingInterval;
+    }
+
+    /**
+     * Sets the number of worker threads.
+     * 
+     * @param workerThreads
+     *            the number of worker threads
+     */
+    public void setWorkerThreads(int workerThreads)
+    {
+        this.workerThreads = workerThreads;
+    }
+
     /*
      * (non-Javadoc)
      * @see org.alfresco.repo.security.sync.UserRegistrySynchronizer#synchronize(boolean, boolean)
      */
     public void synchronize(boolean force, boolean splitTxns)
+    {
+        // Let's ensure all exceptions get logged
+        try
         {
             // First, try to obtain a lock to ensure we are the only node trying to run this job
             try
@@ -287,8 +333,7 @@ public class ChainingUserRegistrySynchronizer extends AbstractLifecycleBean impl
             Collection<String> instanceIds = this.applicationContextManager.getInstanceIds();
 
             // Work out the set of all zone IDs in the authentication chain so that we can decide which users / groups
-        // need
-        // 're-zoning'
+            // need 're-zoning'
             Set<String> allZoneIds = new TreeSet<String>();
             for (String id : instanceIds)
             {
@@ -320,7 +365,7 @@ public class ChainingUserRegistrySynchronizer extends AbstractLifecycleBean impl
                         boolean requiresNew = splitTxns
                                 || AlfrescoTransactionSupport.getTransactionReadState() == TxnReadState.TXN_READ_ONLY;
 
-                    int personsProcessed = syncPersonsWithPlugin(id, plugin, force, requiresNew, visitedZoneIds,
+                        int personsProcessed = syncPersonsWithPlugin(id, plugin, requiresNew, visitedZoneIds,
                                 allZoneIds);
                         int groupsProcessed = syncGroupsWithPlugin(id, plugin, force, requiresNew, visitedZoneIds,
                                 allZoneIds);
@@ -339,6 +384,12 @@ public class ChainingUserRegistrySynchronizer extends AbstractLifecycleBean impl
                 }
             }
         }
+        catch (RuntimeException e)
+        {
+            ChainingUserRegistrySynchronizer.logger.error("Synchronization aborted due to error", e);
+            throw e;
+        }
+    }
 
     /*
      * (non-Javadoc)
@@ -380,16 +431,13 @@ public class ChainingUserRegistrySynchronizer extends AbstractLifecycleBean impl
     }
 
     /**
-     * Synchronizes local users (persons) with a {@link UserRegistry} for a particular zone.
+     * Synchronizes changes only to local users (persons) with a {@link UserRegistry} for a particular zone.
      * 
      * @param zone
      *            the zone id. This identifier is used to tag all created users, so that in the future we can tell those
      *            that have been deleted from the registry.
      * @param userRegistry
      *            the user registry for the zone.
-     * @param force
-     *            <code>true</code> if all persons are to be queried. <code>false</code> if only those changed since the
-     *            most recent queried user should be queried.
      * @param splitTxns
      *            Can the modifications to Alfresco be split across multiple transactions for maximum performance? If
      *            <code>true</code>, users and groups are created/updated in batches of 10 for increased performance. If
@@ -405,7 +453,7 @@ public class ChainingUserRegistrySynchronizer extends AbstractLifecycleBean impl
      *            be 're-zoned'.
      * @return the number of users processed
      */
-    private int syncPersonsWithPlugin(final String zone, UserRegistry userRegistry, boolean force, boolean splitTxns,
+    private int syncPersonsWithPlugin(final String zone, UserRegistry userRegistry, boolean splitTxns,
             final Set<String> visitedZoneIds, final Set<String> allZoneIds)
     {
         // Create a prefixed zone ID for use with the authority service
@@ -414,7 +462,7 @@ public class ChainingUserRegistrySynchronizer extends AbstractLifecycleBean impl
         // The set of zones we associate with new objects (default plus registry specific)
         final Set<String> zoneSet = getZones(zoneId);
 
-        final long lastModifiedMillis = force ? -1L : getMostRecentUpdateTime(
+        final long lastModifiedMillis = getMostRecentUpdateTime(
                 ChainingUserRegistrySynchronizer.PERSON_LAST_MODIFIED_ATTRIBUTE, zoneId);
         final Date lastModified = lastModifiedMillis == -1 ? null : new Date(lastModifiedMillis);
         if (ChainingUserRegistrySynchronizer.logger.isInfoEnabled())
@@ -429,10 +477,10 @@ public class ChainingUserRegistrySynchronizer extends AbstractLifecycleBean impl
                         + DateFormat.getDateTimeInstance().format(lastModified) + " from user registry '" + zone + "'");
             }
         }
-        final Iterator<NodeDescription> persons = userRegistry.getPersons(lastModified);
-        final Set<String> personsCreated = new TreeSet<String>();
-
-        class CreationWorker implements RetryingTransactionCallback<Integer>
+        final BatchProcessor<NodeDescription> personProcessor = new BatchProcessor<NodeDescription>(
+                this.retryingTransactionHelper, this.applicationEventPublisher, userRegistry.getPersons(lastModified),
+                zone + " User Creation", this.loggingInterval, this.workerThreads, 10);
+        class CreationWorker implements Worker<NodeDescription>
         {
             private long latestTime = lastModifiedMillis;
 
@@ -441,36 +489,32 @@ public class ChainingUserRegistrySynchronizer extends AbstractLifecycleBean impl
                 return this.latestTime;
             }
 
-            /*
-             * (non-Javadoc)
-             * @see org.alfresco.repo.transaction.RetryingTransactionHelper.RetryingTransactionCallback#execute()
-             */
-            public Integer execute() throws Throwable
+            public String getIdentifier(NodeDescription entry)
             {
-                int processedCount = 0;
-                do
+                return entry.getSourceId();
+            }
+
+            public void process(NodeDescription person) throws Throwable
             {
-                    NodeDescription person = persons.next();
                 PropertyMap personProperties = person.getProperties();
                 String personName = (String) personProperties.get(ContentModel.PROP_USERNAME);
-
                 Set<String> zones = ChainingUserRegistrySynchronizer.this.authorityService
                         .getAuthorityZones(personName);
                 if (zones == null)
                 {
                     // The person did not exist at all
-                        if (ChainingUserRegistrySynchronizer.logger.isInfoEnabled())
+                    if (ChainingUserRegistrySynchronizer.logger.isDebugEnabled())
                     {
-                            ChainingUserRegistrySynchronizer.logger.info("Creating user '" + personName + "'");
+                        ChainingUserRegistrySynchronizer.logger.debug("Creating user '" + personName + "'");
                     }
                     ChainingUserRegistrySynchronizer.this.personService.createPerson(personProperties, zoneSet);
                 }
                 else if (zones.contains(zoneId))
                 {
                     // The person already existed in this zone: update the person
-                        if (ChainingUserRegistrySynchronizer.logger.isInfoEnabled())
+                    if (ChainingUserRegistrySynchronizer.logger.isDebugEnabled())
                     {
-                            ChainingUserRegistrySynchronizer.logger.info("Updating user '" + personName + "'");
+                        ChainingUserRegistrySynchronizer.logger.debug("Updating user '" + personName + "'");
                     }
                     ChainingUserRegistrySynchronizer.this.personService.setPersonProperties(personName,
                             personProperties);
@@ -482,18 +526,17 @@ public class ChainingUserRegistrySynchronizer extends AbstractLifecycleBean impl
                     intersection.retainAll(allZoneIds);
                     if (intersection.size() == 0)
                     {
-                            // The person exists, but not in a zone that's in the authentication chain. May be due to
-                            // upgrade or zone changes. Let's re-zone them
+                        // The person exists, but not in a zone that's in the authentication chain. May be due
+                        // to upgrade or zone changes. Let's re-zone them
                         if (ChainingUserRegistrySynchronizer.logger.isWarnEnabled())
                         {
                             ChainingUserRegistrySynchronizer.logger.warn("Updating user '" + personName
-                                        + "'. This user will in future be assumed to originate from user registry '"
-                                        + zone + "'.");
+                                    + "'. This user will in future be assumed to originate from user registry '" + zone
+                                    + "'.");
                         }
                         ChainingUserRegistrySynchronizer.this.authorityService.removeAuthorityFromZones(personName,
                                 zones);
-                            ChainingUserRegistrySynchronizer.this.authorityService.addAuthorityToZones(personName,
-                                    zoneSet);
+                        ChainingUserRegistrySynchronizer.this.authorityService.addAuthorityToZones(personName, zoneSet);
                         ChainingUserRegistrySynchronizer.this.personService.setPersonProperties(personName,
                                 personProperties);
                     }
@@ -504,7 +547,7 @@ public class ChainingUserRegistrySynchronizer extends AbstractLifecycleBean impl
                         if (intersection.size() > 0)
                         {
                             // A person that exists in a different zone with higher precedence - ignore
-                                continue;
+                            return;
                         }
 
                         // The person existed, but in a zone with lower precedence
@@ -519,10 +562,8 @@ public class ChainingUserRegistrySynchronizer extends AbstractLifecycleBean impl
                         ChainingUserRegistrySynchronizer.this.personService.createPerson(personProperties, zoneSet);
                     }
                 }
-                    // Increment the count of processed people
-                    personsCreated.add(personName);
-                    processedCount++;
-
+                synchronized (this)
+                {
                     // Maintain the last modified date
                     Date personLastModified = person.getLastModified();
                     if (personLastModified != null)
@@ -530,54 +571,16 @@ public class ChainingUserRegistrySynchronizer extends AbstractLifecycleBean impl
                         this.latestTime = Math.max(this.latestTime, personLastModified.getTime());
                     }
                 }
-                while (persons.hasNext() && processedCount < ChainingUserRegistrySynchronizer.BATCH_SIZE);
-                return processedCount;
             }
         }
 
         CreationWorker creations = new CreationWorker();
-        int processedCount = 0;
-        while (persons.hasNext())
-        {
-            processedCount += this.retryingTransactionHelper.doInTransaction(creations, false, splitTxns);
-        }
+        int processedCount = personProcessor.process(creations, splitTxns);
         long latestTime = creations.getLatestTime();
         if (latestTime != -1)
         {
-            setMostRecentUpdateTime(ChainingUserRegistrySynchronizer.PERSON_LAST_MODIFIED_ATTRIBUTE, zoneId, latestTime);
-        }
-
-        // Handle deletions if we are doing a full sync
-        if (force)
-        {
-            class DeletionWorker implements RetryingTransactionCallback<Integer>
-            {
-                /*
-                 * (non-Javadoc)
-                 * @see org.alfresco.repo.transaction.RetryingTransactionHelper.RetryingTransactionCallback#execute()
-                 */
-                public Integer execute() throws Throwable
-                {
-                    int processedCount = 0;
-                    Set<String> personsToDelete = ChainingUserRegistrySynchronizer.this.authorityService
-                            .getAllAuthoritiesInZone(zoneId, AuthorityType.USER);
-                    personsToDelete.removeAll(personsCreated);
-                    for (String personName : personsToDelete)
-                    {
-                        if (ChainingUserRegistrySynchronizer.logger.isWarnEnabled())
-                        {
-                            ChainingUserRegistrySynchronizer.logger.warn("Deleting user '" + personName + "'");
-                        }
-                        ChainingUserRegistrySynchronizer.this.personService.deletePerson(personName);
-                        processedCount++;
-                    }
-                    return processedCount;
-                }
-            }
-
-            // Just use a single transaction
-            DeletionWorker deletions = new DeletionWorker();
-            processedCount += this.retryingTransactionHelper.doInTransaction(deletions, false, splitTxns);
+            setMostRecentUpdateTime(ChainingUserRegistrySynchronizer.PERSON_LAST_MODIFIED_ATTRIBUTE, zoneId,
+                    latestTime, splitTxns);
         }
 
         // Remember we have visited this zone
@@ -587,7 +590,8 @@ public class ChainingUserRegistrySynchronizer extends AbstractLifecycleBean impl
     }
 
     /**
-     * Synchronizes local groups (authorities) with a {@link UserRegistry} for a particular zone.
+     * Synchronizes local groups with a {@link UserRegistry} for a particular zone and also handles deletions of local
+     * groups and users.
      * 
      * @param zone
      *            the zone id. This identifier is used to tag all created groups, so that in the future we can tell
@@ -595,8 +599,7 @@ public class ChainingUserRegistrySynchronizer extends AbstractLifecycleBean impl
      * @param userRegistry
      *            the user registry for the zone.
      * @param force
-     *            <code>true</code> if all groups are to be queried. <code>false</code> if only those changed since the
-     *            most recent queried group should be queried.
+     *            <code>true</code> if user and group deletions are to be processed.
      * @param splitTxns
      *            Can the modifications to Alfresco be split across multiple transactions for maximum performance? If
      *            <code>true</code>, users and groups are created/updated in batches of 10 for increased performance. If
@@ -621,7 +624,7 @@ public class ChainingUserRegistrySynchronizer extends AbstractLifecycleBean impl
         // The set of zones we associate with new objects (default plus registry specific)
         final Set<String> zoneSet = getZones(zoneId);
 
-        final long lastModifiedMillis = force ? -1L : getMostRecentUpdateTime(
+        final long lastModifiedMillis = getMostRecentUpdateTime(
                 ChainingUserRegistrySynchronizer.GROUP_LAST_MODIFIED_ATTRIBUTE, zoneId);
         final Date lastModified = lastModifiedMillis == -1 ? null : new Date(lastModifiedMillis);
 
@@ -638,12 +641,37 @@ public class ChainingUserRegistrySynchronizer extends AbstractLifecycleBean impl
             }
         }
 
-        final Iterator<NodeDescription> groups = userRegistry.getGroups(lastModified);
-        final Map<String, Set<String>> groupAssocsToCreate = new TreeMap<String, Set<String>>();
-        final Set<String> groupsCreated = new TreeSet<String>();
-
-        class CreationWorker implements RetryingTransactionCallback<Integer>
+        // Get current set of known authorities
+        Set<String> deletionCandidates = this.retryingTransactionHelper.doInTransaction(
+                new RetryingTransactionCallback<Set<String>>()
                 {
+                    public Set<String> execute() throws Throwable
+                    {
+                        return ChainingUserRegistrySynchronizer.this.authorityService.getAllAuthoritiesInZone(zoneId,
+                                null);
+                    }
+                }, false, splitTxns);
+
+        final BatchProcessor<NodeDescription> groupProcessor = new BatchProcessor<NodeDescription>(
+                this.retryingTransactionHelper, this.applicationEventPublisher, userRegistry.getGroups(lastModified,
+                        deletionCandidates, force), zone + " Group Creation", this.loggingInterval, this.workerThreads,
+                20);
+        class CreationWorker implements Worker<NodeDescription>
+        {
+            private final Set<Pair<String, String>> groupAssocsToCreate = new TreeSet<Pair<String, String>>(
+                    new Comparator<Pair<String, String>>()
+                    {
+
+                        public int compare(Pair<String, String> o1, Pair<String, String> o2)
+                        {
+                            int result = o1.getFirst().compareTo(o2.getFirst());
+                            if (result == 0)
+                            {
+                                return o1.getSecond().compareTo(o2.getSecond());
+                            }
+                            return result;
+                        }
+                    });
             private long latestTime = lastModifiedMillis;
 
             public long getLatestTime()
@@ -651,29 +679,38 @@ public class ChainingUserRegistrySynchronizer extends AbstractLifecycleBean impl
                 return this.latestTime;
             }
 
+            public Set<Pair<String, String>> getGroupAssocsToCreate()
+            {
+                return this.groupAssocsToCreate;
+            }
+
             /*
              * (non-Javadoc)
-             * @see org.alfresco.repo.transaction.RetryingTransactionHelper.RetryingTransactionCallback#execute()
+             * @see org.alfresco.repo.security.sync.BatchProcessor.Worker#getIdentifier(java.lang.Object)
              */
-            public Integer execute() throws Throwable
+            public String getIdentifier(NodeDescription entry)
             {
-                int processedCount = 0;
-                do
+                return entry.getSourceId();
+            }
+
+            /*
+             * (non-Javadoc)
+             * @see org.alfresco.repo.security.sync.BatchProcessor.Worker#process(java.lang.Object)
+             */
+            public void process(NodeDescription group) throws Throwable
             {
-                    NodeDescription group = groups.next();
                 PropertyMap groupProperties = group.getProperties();
                 String groupName = (String) groupProperties.get(ContentModel.PROP_AUTHORITY_NAME);
-                    String groupShortName = ChainingUserRegistrySynchronizer.this.authorityService
-                            .getShortName(groupName);
+                String groupShortName = ChainingUserRegistrySynchronizer.this.authorityService.getShortName(groupName);
                 Set<String> groupZones = ChainingUserRegistrySynchronizer.this.authorityService
                         .getAuthorityZones(groupName);
 
                 if (groupZones == null)
                 {
                     // The group did not exist at all
-                        if (ChainingUserRegistrySynchronizer.logger.isInfoEnabled())
+                    if (ChainingUserRegistrySynchronizer.logger.isDebugEnabled())
                     {
-                            ChainingUserRegistrySynchronizer.logger.info("Creating group '" + groupShortName + "'");
+                        ChainingUserRegistrySynchronizer.logger.debug("Creating group '" + groupShortName + "'");
                     }
                     // create the group
                     ChainingUserRegistrySynchronizer.this.authorityService.createAuthority(AuthorityType
@@ -682,7 +719,13 @@ public class ChainingUserRegistrySynchronizer extends AbstractLifecycleBean impl
                     Set<String> children = group.getChildAssociations();
                     if (!children.isEmpty())
                     {
-                            groupAssocsToCreate.put(groupName, children);
+                        synchronized (this)
+                        {
+                            for (String child : children)
+                            {
+                                this.groupAssocsToCreate.add(new Pair<String, String>(groupName, child));
+                            }
+                        }
                     }
                 }
                 else
@@ -702,8 +745,7 @@ public class ChainingUserRegistrySynchronizer extends AbstractLifecycleBean impl
                         }
                         ChainingUserRegistrySynchronizer.this.authorityService.removeAuthorityFromZones(groupName,
                                 groupZones);
-                            ChainingUserRegistrySynchronizer.this.authorityService.addAuthorityToZones(groupName,
-                                    zoneSet);
+                        ChainingUserRegistrySynchronizer.this.authorityService.addAuthorityToZones(groupName, zoneSet);
                     }
                     if (groupZones.contains(zoneId) || intersection.isEmpty())
                     {
@@ -717,18 +759,23 @@ public class ChainingUserRegistrySynchronizer extends AbstractLifecycleBean impl
                         toAdd.removeAll(oldChildren);
                         if (!toAdd.isEmpty())
                         {
-                                groupAssocsToCreate.put(groupName, toAdd);
+                            synchronized (this)
+                            {
+                                for (String child : toAdd)
+                                {
+                                    this.groupAssocsToCreate.add(new Pair<String, String>(groupName, child));
+                                }
+                            }
                         }
                         for (String child : toDelete)
                         {
-                                if (ChainingUserRegistrySynchronizer.logger.isInfoEnabled())
+                            if (ChainingUserRegistrySynchronizer.logger.isDebugEnabled())
                             {
-                                    ChainingUserRegistrySynchronizer.logger.info("Removing '"
-                                            + ChainingUserRegistrySynchronizer.this.authorityService
-                                                    .getShortName(child) + "' from group '" + groupShortName + "'");
+                                ChainingUserRegistrySynchronizer.logger.debug("Removing '"
+                                        + ChainingUserRegistrySynchronizer.this.authorityService.getShortName(child)
+                                        + "' from group '" + groupShortName + "'");
                             }
-                                ChainingUserRegistrySynchronizer.this.authorityService
-                                        .removeAuthority(groupName, child);
+                            ChainingUserRegistrySynchronizer.this.authorityService.removeAuthority(groupName, child);
                         }
                     }
                     else
@@ -738,7 +785,7 @@ public class ChainingUserRegistrySynchronizer extends AbstractLifecycleBean impl
                         if (!intersection.isEmpty())
                         {
                             // A group that exists in a different zone with higher precedence
-                                continue;
+                            return;
                         }
                         // The group existed, but in a zone with lower precedence
                         if (ChainingUserRegistrySynchronizer.logger.isWarnEnabled())
@@ -756,130 +803,101 @@ public class ChainingUserRegistrySynchronizer extends AbstractLifecycleBean impl
                         Set<String> children = group.getChildAssociations();
                         if (!children.isEmpty())
                         {
-                                groupAssocsToCreate.put(groupName, children);
+                            synchronized (this)
+                            {
+                                for (String child : children)
+                                {
+                                    this.groupAssocsToCreate.add(new Pair<String, String>(groupName, child));
+                                }
+                            }
                         }
                     }
                 }
-                    // Increment the count of processed groups
-                    processedCount++;
-                    groupsCreated.add(groupName);
 
+                synchronized (this)
+                {
                     // Maintain the last modified date
                     Date groupLastModified = group.getLastModified();
                     if (groupLastModified != null)
                     {
                         this.latestTime = Math.max(this.latestTime, groupLastModified.getTime());
                     }
-
                 }
-                while (groups.hasNext() && processedCount < ChainingUserRegistrySynchronizer.BATCH_SIZE);
-                return processedCount;
             }
         }
 
         CreationWorker creations = new CreationWorker();
-        int processedCount = 0;
-        while (groups.hasNext())
-        {
-            processedCount += this.retryingTransactionHelper.doInTransaction(creations, false, splitTxns);
-        }
+        int processedCount = groupProcessor.process(creations, splitTxns);
         long latestTime = creations.getLatestTime();
         if (latestTime != -1)
         {
-            setMostRecentUpdateTime(ChainingUserRegistrySynchronizer.GROUP_LAST_MODIFIED_ATTRIBUTE, zoneId, latestTime);
+            setMostRecentUpdateTime(ChainingUserRegistrySynchronizer.GROUP_LAST_MODIFIED_ATTRIBUTE, zoneId, latestTime,
+                    splitTxns);
         }
 
         // Add the new associations, now that we have created everything
-
-        final Iterator<Map.Entry<String, Set<String>>> groupAssocs = groupAssocsToCreate.entrySet().iterator();
-        class AssocWorker implements RetryingTransactionCallback<Integer>
-        {
-            /*
-             * (non-Javadoc)
-             * @see org.alfresco.repo.transaction.RetryingTransactionHelper.RetryingTransactionCallback#execute()
-             */
-            public Integer execute() throws Throwable
-            {
-                int processedCount = 0;
-                do
+        BatchProcessor<Pair<String, String>> groupAssocProcessor = new BatchProcessor<Pair<String, String>>(
+                this.retryingTransactionHelper, this.applicationEventPublisher, creations.getGroupAssocsToCreate(),
+                zone + " Group Association Creation", this.loggingInterval, this.workerThreads, 20);
+        groupAssocProcessor.process(new Worker<Pair<String, String>>()
         {
 
-                    Map.Entry<String, Set<String>> entry = groupAssocs.next();
-                    for (String child : entry.getValue())
+            public String getIdentifier(Pair<String, String> entry)
             {
-                        String groupName = entry.getKey();
-                        if (ChainingUserRegistrySynchronizer.logger.isInfoEnabled())
+                return entry.toString();
+            }
+
+            public void process(Pair<String, String> entry) throws Throwable
             {
-                            ChainingUserRegistrySynchronizer.logger.info("Adding '"
+                String groupName = entry.getFirst();
+                String child = entry.getSecond();
+                if (ChainingUserRegistrySynchronizer.logger.isDebugEnabled())
+                {
+                    ChainingUserRegistrySynchronizer.logger.debug("Adding '"
                             + ChainingUserRegistrySynchronizer.this.authorityService.getShortName(child)
                             + "' to group '"
-                                    + ChainingUserRegistrySynchronizer.this.authorityService.getShortName(groupName)
-                                    + "'");
+                            + ChainingUserRegistrySynchronizer.this.authorityService.getShortName(groupName) + "'");
                 }
-                        try
-                        {
                 ChainingUserRegistrySynchronizer.this.authorityService.addAuthority(groupName, child);
             }
-                        catch (Exception e)
-                        {
-                            // Let's not allow referential integrity problems (dangling references) kill the whole
-                            // process
-                            if (ChainingUserRegistrySynchronizer.logger.isWarnEnabled())
-                            {
-                                ChainingUserRegistrySynchronizer.logger.warn("Failed to add '"
-                                        + ChainingUserRegistrySynchronizer.this.authorityService.getShortName(child)
-                                        + "' to group '"
-                                        + ChainingUserRegistrySynchronizer.this.authorityService
-                                                .getShortName(groupName) + "'", e);
-                            }
-                        }
+        }, splitTxns);
 
-                    }
-                }
-                while (groupAssocs.hasNext() && processedCount < ChainingUserRegistrySynchronizer.BATCH_SIZE);
-                return processedCount;
-
-            }
-        }
-
-        AssocWorker assocs = new AssocWorker();
-        while (groupAssocs.hasNext())
-        {
-            this.retryingTransactionHelper.doInTransaction(assocs, false, splitTxns);
-        }
-
-        // Delete groups if we have complete information for the zone
+        // Delete authorities if we have complete information for the zone
         if (force)
         {
-            class DeletionWorker implements RetryingTransactionCallback<Integer>
+            BatchProcessor<String> authorityDeletionProcessor = new BatchProcessor<String>(
+                    this.retryingTransactionHelper, this.applicationEventPublisher, deletionCandidates, zone
+                            + " Authority Deletion", this.loggingInterval, this.workerThreads, 10);
+            processedCount += authorityDeletionProcessor.process(new Worker<String>()
             {
-                /*
-                 * (non-Javadoc)
-                 * @see org.alfresco.repo.transaction.RetryingTransactionHelper.RetryingTransactionCallback#execute()
-                 */
-                public Integer execute() throws Throwable
+
+                public String getIdentifier(String entry)
                 {
-                    int processedCount = 0;
-                    Set<String> groupsToDelete = ChainingUserRegistrySynchronizer.this.authorityService
-                            .getAllAuthoritiesInZone(zoneId, AuthorityType.GROUP);
-                    groupsToDelete.removeAll(groupsCreated);
-                    for (String group : groupsToDelete)
-                    {
-                        if (ChainingUserRegistrySynchronizer.logger.isWarnEnabled())
-                        {
-                            ChainingUserRegistrySynchronizer.logger.warn("Deleting group '"
-                                    + ChainingUserRegistrySynchronizer.this.authorityService.getShortName(group) + "'");
-                        }
-                        ChainingUserRegistrySynchronizer.this.authorityService.deleteAuthority(group);
-                        processedCount++;
-                    }
-                    return processedCount;
-                }
+                    return entry;
                 }
 
-            // Just use a single transaction
-            DeletionWorker deletions = new DeletionWorker();
-            processedCount += this.retryingTransactionHelper.doInTransaction(deletions, false, splitTxns);
+                public void process(String authority) throws Throwable
+                {
+                    if (AuthorityType.getAuthorityType(authority) == AuthorityType.USER)
+                    {
+                        if (ChainingUserRegistrySynchronizer.logger.isDebugEnabled())
+                        {
+                            ChainingUserRegistrySynchronizer.logger.debug("Deleting user '" + authority + "'");
+                        }
+                        ChainingUserRegistrySynchronizer.this.personService.deletePerson(authority);
+                    }
+                    else
+                    {
+                        if (ChainingUserRegistrySynchronizer.logger.isDebugEnabled())
+                        {
+                            ChainingUserRegistrySynchronizer.logger.debug("Deleting group '"
+                                    + ChainingUserRegistrySynchronizer.this.authorityService.getShortName(authority)
+                                    + "'");
+                        }
+                        ChainingUserRegistrySynchronizer.this.authorityService.deleteAuthority(authority);
+                    }
+                }
+            }, splitTxns);
         }
 
         // Remember we have visited this zone
@@ -913,21 +931,38 @@ public class ChainingUserRegistrySynchronizer extends AbstractLifecycleBean impl
      *            the zone id
      * @param lastModifiedMillis
      *            the update time in milliseconds
+     * @param splitTxns
+     *            Can the modifications to Alfresco be split across multiple transactions for maximum performance? If
+     *            <code>true</code>, the attribute is persisted in a new transaction for increased performance and
+     *            reliability.
      */
-    private void setMostRecentUpdateTime(String label, String zoneId, long lastModifiedMillis)
+    private void setMostRecentUpdateTime(final String label, final String zoneId, final long lastModifiedMillis,
+            boolean splitTxns)
     {
-        String path = ChainingUserRegistrySynchronizer.ROOT_ATTRIBUTE_PATH + '/' + label;
-        if (!this.attributeService.exists(path))
+        final String path = ChainingUserRegistrySynchronizer.ROOT_ATTRIBUTE_PATH + '/' + label;
+        this.retryingTransactionHelper.doInTransaction(
+                new RetryingTransactionHelper.RetryingTransactionCallback<Object>()
                 {
-            if (!this.attributeService.exists(ChainingUserRegistrySynchronizer.ROOT_ATTRIBUTE_PATH))
+
+                    public Object execute() throws Throwable
                     {
-                this.attributeService.setAttribute("", ChainingUserRegistrySynchronizer.ROOT_ATTRIBUTE_PATH,
+                        if (!ChainingUserRegistrySynchronizer.this.attributeService.exists(path))
+                        {
+                            if (!ChainingUserRegistrySynchronizer.this.attributeService
+                                    .exists(ChainingUserRegistrySynchronizer.ROOT_ATTRIBUTE_PATH))
+                            {
+                                ChainingUserRegistrySynchronizer.this.attributeService.setAttribute("",
+                                        ChainingUserRegistrySynchronizer.ROOT_ATTRIBUTE_PATH, new MapAttributeValue());
+                            }
+                            ChainingUserRegistrySynchronizer.this.attributeService.setAttribute(
+                                    ChainingUserRegistrySynchronizer.ROOT_ATTRIBUTE_PATH, label,
                                     new MapAttributeValue());
                         }
-            this.attributeService.setAttribute(ChainingUserRegistrySynchronizer.ROOT_ATTRIBUTE_PATH, label,
-                    new MapAttributeValue());
+                        ChainingUserRegistrySynchronizer.this.attributeService.setAttribute(path, zoneId,
+                                new LongAttributeValue(lastModifiedMillis));
+                        return null;
                     }
-        this.attributeService.setAttribute(path, zoneId, new LongAttributeValue(lastModifiedMillis));
+                }, false, splitTxns);
     }
 
     /**
@@ -941,7 +976,7 @@ public class ChainingUserRegistrySynchronizer extends AbstractLifecycleBean impl
      */
     private Set<String> getZones(String zoneId)
     {
-        Set<String> zones = new HashSet<String>(2, 1.0f);
+        Set<String> zones = new HashSet<String>(5);
         zones.add(AuthorityService.ZONE_APP_DEFAULT);
         zones.add(zoneId);
         return zones;
@@ -993,5 +1028,4 @@ public class ChainingUserRegistrySynchronizer extends AbstractLifecycleBean impl
     protected void onShutdown(ApplicationEvent event)
     {
     }
-
 }

source/java/org/alfresco/repo/security/sync/ChainingUserRegistrySynchronizerTest.java

@@ -24,6 +24,8 @@
  */
 package org.alfresco.repo.security.sync;
 
+import java.util.AbstractCollection;
+import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.Collection;
 import java.util.Collections;
@@ -31,6 +33,7 @@ import java.util.Date;
 import java.util.HashSet;
 import java.util.Iterator;
 import java.util.LinkedHashMap;
+import java.util.List;
 import java.util.Map;
 import java.util.Set;
 
@@ -46,6 +49,7 @@ import org.alfresco.service.cmr.repository.NodeService;
 import org.alfresco.service.cmr.security.AuthorityService;
 import org.alfresco.service.cmr.security.AuthorityType;
 import org.alfresco.service.cmr.security.PersonService;
+import org.alfresco.util.GUID;
 import org.alfresco.util.PropertyMap;
 import org.springframework.context.ApplicationContext;
 import org.springframework.context.support.ClassPathXmlApplicationContext;
@@ -65,7 +69,7 @@ public class ChainingUserRegistrySynchronizerTest extends TestCase
         "classpath:alfresco/application-context.xml", "classpath:sync-test-context.xml"
     };
 
-    /** The Spring application context */
+    /** The Spring application context. */
     private static ApplicationContext context = new ClassPathXmlApplicationContext(
             ChainingUserRegistrySynchronizerTest.CONFIG_LOCATIONS);
 
@@ -90,6 +94,10 @@ public class ChainingUserRegistrySynchronizerTest extends TestCase
     /** The retrying transaction helper. */
     private RetryingTransactionHelper retryingTransactionHelper;
 
+    /*
+     * (non-Javadoc)
+     * @see junit.framework.TestCase#setUp()
+     */
     @Override
     protected void setUp() throws Exception
     {
@@ -110,6 +118,10 @@ public class ChainingUserRegistrySynchronizerTest extends TestCase
                 .getBean("retryingTransactionHelper");
     }
 
+    /*
+     * (non-Javadoc)
+     * @see junit.framework.TestCase#tearDown()
+     */
     @Override
     protected void tearDown() throws Exception
     {
@@ -191,6 +203,12 @@ public class ChainingUserRegistrySynchronizerTest extends TestCase
         });
     }
 
+    /**
+     * Tear down test users and groups.
+     * 
+     * @throws Exception
+     *             the exception
+     */
     private void tearDownTestUsersAndGroups() throws Exception
     {
         // Wipe out everything that was in Z1 and Z2
@@ -365,7 +383,30 @@ public class ChainingUserRegistrySynchronizerTest extends TestCase
     }
 
     /**
-     * Constructs a description of a test group
+     * Tests synchronization with a zone with a larger volume of authorities.
+     * 
+     * @throws Exception
+     *             the exception
+     */
+    public void testVolume() throws Exception
+    {
+        List<NodeDescription> persons = new ArrayList<NodeDescription>(new RandomPersonCollection(100));
+        List<NodeDescription> groups = new ArrayList<NodeDescription>(new RandomGroupCollection(100, persons));
+        this.applicationContextManager.setUserRegistries(new MockUserRegistry("Z0", persons, groups));
+        this.retryingTransactionHelper.doInTransaction(new RetryingTransactionCallback<Object>()
+        {
+
+            public Object execute() throws Throwable
+            {
+                ChainingUserRegistrySynchronizerTest.this.synchronizer.synchronize(true, true);
+                return null;
+            }
+        });
+        tearDownTestUsersAndGroups();
+    }
+
+    /**
+     * Constructs a description of a test group.
      * 
      * @param name
      *            the name
@@ -375,9 +416,10 @@ public class ChainingUserRegistrySynchronizerTest extends TestCase
      */
     private NodeDescription newGroup(String name, String... members)
     {
-        NodeDescription group = new NodeDescription();
+        String longName = longName(name);
+        NodeDescription group = new NodeDescription(longName);
         PropertyMap properties = group.getProperties();
-        properties.put(ContentModel.PROP_AUTHORITY_NAME, longName(name));
+        properties.put(ContentModel.PROP_AUTHORITY_NAME, longName);
         if (members.length > 0)
         {
             Set<String> assocs = group.getChildAssociations();
@@ -413,7 +455,7 @@ public class ChainingUserRegistrySynchronizerTest extends TestCase
      */
     private NodeDescription newPerson(String userName, String email)
     {
-        NodeDescription person = new NodeDescription();
+        NodeDescription person = new NodeDescription(userName);
         PropertyMap properties = person.getProperties();
         properties.put(ContentModel.PROP_USERNAME, userName);
         properties.put(ContentModel.PROP_FIRSTNAME, userName + "F");
@@ -481,7 +523,7 @@ public class ChainingUserRegistrySynchronizerTest extends TestCase
     }
 
     /**
-     * Asserts that a person's email has the expected value
+     * Asserts that a person's email has the expected value.
      * 
      * @param personName
      *            the person name
@@ -518,10 +560,27 @@ public class ChainingUserRegistrySynchronizerTest extends TestCase
         private String zoneId;
 
         /** The persons. */
-        private NodeDescription[] persons;
+        private Collection<NodeDescription> persons;
 
         /** The groups. */
-        private NodeDescription[] groups;
+        private Collection<NodeDescription> groups;
+
+        /**
+         * Instantiates a new mock user registry.
+         * 
+         * @param zoneId
+         *            the zone id
+         * @param persons
+         *            the persons
+         * @param groups
+         *            the groups
+         */
+        public MockUserRegistry(String zoneId, Collection<NodeDescription> persons, Collection<NodeDescription> groups)
+        {
+            this.zoneId = zoneId;
+            this.persons = persons;
+            this.groups = groups;
+        }
 
         /**
          * Instantiates a new mock user registry.
@@ -535,9 +594,7 @@ public class ChainingUserRegistrySynchronizerTest extends TestCase
          */
         public MockUserRegistry(String zoneId, NodeDescription[] persons, NodeDescription[] groups)
         {
-            this.zoneId = zoneId;
-            this.persons = persons;
-            this.groups = groups;
+            this(zoneId, Arrays.asList(persons), Arrays.asList(groups));
         }
 
         /**
@@ -552,20 +609,32 @@ public class ChainingUserRegistrySynchronizerTest extends TestCase
 
         /*
          * (non-Javadoc)
-         * @see org.alfresco.repo.security.sync.UserRegistry#getGroups(java.util.Date)
+         * @see org.alfresco.repo.security.sync.UserRegistry#getGroups(java.util.Date, java.util.Set, boolean)
          */
-        public Iterator<NodeDescription> getGroups(Date modifiedSince)
+        public Collection<NodeDescription> getGroups(Date modifiedSince, Set<String> candidateAuthoritiesForDeletion,
+                boolean prune)
         {
-            return Arrays.asList(this.groups).iterator();
+            if (prune)
+            {
+                for (NodeDescription person : this.persons)
+                {
+                    candidateAuthoritiesForDeletion.remove(person.getProperties().get(ContentModel.PROP_USERNAME));
+                }
+                for (NodeDescription group : this.groups)
+                {
+                    candidateAuthoritiesForDeletion.remove(group.getProperties().get(ContentModel.PROP_AUTHORITY_NAME));
+                }
+            }
+            return this.groups;
         }
 
         /*
          * (non-Javadoc)
          * @see org.alfresco.repo.security.sync.UserRegistry#getPersons(java.util.Date)
          */
-        public Iterator<NodeDescription> getPersons(Date modifiedSince)
+        public Collection<NodeDescription> getPersons(Date modifiedSince)
         {
-            return Arrays.asList(this.persons).iterator();
+            return this.persons;
         }
     }
 
@@ -615,4 +684,144 @@ public class ChainingUserRegistrySynchronizerTest extends TestCase
             return this.contexts.keySet();
         }
     }
+
+    /**
+     * A collection whose iterator returns randomly generated persons.
+     */
+    public class RandomPersonCollection extends AbstractCollection<NodeDescription>
+    {
+
+        /** The collection size. */
+        private final int size;
+
+        /**
+         * The Constructor.
+         * 
+         * @param size
+         *            the collection size
+         */
+        public RandomPersonCollection(int size)
+        {
+            this.size = size;
+        }
+
+        /*
+         * (non-Javadoc)
+         * @see java.util.AbstractCollection#iterator()
+         */
+        @Override
+        public Iterator<NodeDescription> iterator()
+        {
+            return new Iterator<NodeDescription>()
+            {
+
+                private int pos;
+
+                public boolean hasNext()
+                {
+                    return pos < size;
+                }
+
+                public NodeDescription next()
+                {
+                    pos++;
+                    return newPerson("U" + GUID.generate());
+                }
+
+                public void remove()
+                {
+                    throw new UnsupportedOperationException();
+                }
+            };
+
+        }
+
+        /*
+         * (non-Javadoc)
+         * @see java.util.AbstractCollection#size()
+         */
+        @Override
+        public int size()
+        {
+            return size;
+        }
+
+    }
+
+    /**
+     * A collection whose iterator returns randomly generated groups with random associations to a given list of
+     * persons.
+     */
+    public class RandomGroupCollection extends AbstractCollection<NodeDescription>
+    {
+
+        /** The collection size. */
+        private final int size;
+
+        /** The persons. */
+        private final List<NodeDescription> persons;
+
+        /**
+         * The Constructor.
+         * 
+         * @param size
+         *            the collection size
+         * @param persons
+         *            the persons
+         */
+        public RandomGroupCollection(int size, List<NodeDescription> persons)
+        {
+            this.size = size;
+            this.persons = persons;
+        }
+
+        /*
+         * (non-Javadoc)
+         * @see java.util.AbstractCollection#iterator()
+         */
+        @Override
+        public Iterator<NodeDescription> iterator()
+        {
+            return new Iterator<NodeDescription>()
+            {
+
+                private int pos;
+
+                public boolean hasNext()
+                {
+                    return pos < size;
+                }
+
+                public NodeDescription next()
+                {
+                    pos++;
+                    String[] personNames = new String[10];
+                    for (int i = 0; i < personNames.length; i++)
+                    {
+                        personNames[i] = (String) persons.get((int) (Math.random() * (double) (persons.size() - 1)))
+                                .getProperties().get(ContentModel.PROP_USERNAME);
+                    }
+                    return newGroup("G" + GUID.generate(), personNames);
+                }
+
+                public void remove()
+                {
+                    throw new UnsupportedOperationException();
+                }
+            };
+
+        }
+
+        /*
+         * (non-Javadoc)
+         * @see java.util.AbstractCollection#size()
+         */
+        @Override
+        public int size()
+        {
+            return size;
+        }
+
+    }
+
 }

source/java/org/alfresco/repo/security/sync/NodeDescription.java

@@ -37,6 +37,11 @@ import org.alfresco.util.PropertyMap;
  */
 public class NodeDescription
 {
+    /**
+     * An identifier for the node for monitoring purposes. Should help trace where the node originated from.
+     */
+    private String sourceId;
+
     /** The properties. */
     private final PropertyMap properties = new PropertyMap(19);
 
@@ -46,6 +51,27 @@ public class NodeDescription
     /** The last modification date. */
     private Date lastModified;
 
+    /**
+     * Instantiates a new node description.
+     * 
+     * @param sourceId
+     *            An identifier for the node for monitoring purposes. Should help trace where the node originated from.
+     */
+    public NodeDescription(String sourceId)
+    {
+        this.sourceId = sourceId;
+    }        
+
+    /**
+     * Gets an identifier for the node for monitoring purposes. Should help trace where the node originated from.
+     * 
+     * @return an identifier for the node for monitoring purposes
+     */
+    public String getSourceId()
+    {
+        return sourceId;
+    }
+
     /**
      * Gets the last modification date.
      * 

source/java/org/alfresco/repo/security/sync/UserRegistry.java

@@ -24,8 +24,9 @@
  */
 package org.alfresco.repo.security.sync;
 
+import java.util.Collection;
 import java.util.Date;
-import java.util.Iterator;
+import java.util.Set;
 
 /**
  * A <code>UserRegistry</code> is an encapsulation of an external registry from which user and group information can be
@@ -42,21 +43,29 @@ public interface UserRegistry
      * @param modifiedSince
      *            if non-null, then only descriptions of users modified since this date should be returned; if
      *            <code>null</code> then descriptions of all users should be returned.
-     * @return a {@link Iterator} over {@link NodeDescription}s of all the persons (users) in the user registry or all
+     * @return a {@link Collection} of {@link NodeDescription}s of all the persons (users) in the user registry or all
      *         those changed since a certain date. The description properties should correspond to those of an Alfresco
      *         person node.
      */
-    public Iterator<NodeDescription> getPersons(Date modifiedSince);
+    public Collection<NodeDescription> getPersons(Date modifiedSince);
 
     /**
-     * Gets descriptions of all the groups in the user registry or all those changed since a certain date.
+     * Gets descriptions of all the groups in the user registry or all those changed since a certain date. Group
+     * associations should be restricted to those in the given set of known authorities. Optionally this set is 'pruned'
+     * to contain only those authorities that no longer exist in the user registry, i.e. the deletion candidates.
      * 
      * @param modifiedSince
      *            if non-null, then only descriptions of groups modified since this date should be returned; if
      *            <code>null</code> then descriptions of all groups should be returned.
-     * @return a {@link Iterator} over {@link NodeDescription}s of all the groups in the user registry or all those
+     * @param knownAuthorities
+     *            the current set of known authorities
+     * @param prune
+     *            should this set be 'pruned' so that it contains only those authorities that do not exist in the
+     *            registry, i.e. the deletion candidates?
+     * @return a {@link Collection} of {@link NodeDescription}s of all the groups in the user registry or all those
      *         changed since a certain date. The description properties should correspond to those of an Alfresco
      *         authority node.
      */
-    public Iterator<NodeDescription> getGroups(Date modifiedSince);
+    public Collection<NodeDescription> getGroups(Date modifiedSince, Set<String> knownAuthorities, boolean prune);
+
 }

source/java/org/alfresco/repo/security/sync/ldap/LDAPNameResolver.java

@@ -0,0 +1,47 @@
+/*
+ * Copyright (C) 2005-2009 Alfresco Software Limited.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version 2
+ * of the License, or (at your option) any later version.
+
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+
+ * As a special exception to the terms and conditions of version 2.0 of 
+ * the GPL, you may redistribute this Program in connection with Free/Libre 
+ * and Open Source Software ("FLOSS") applications as described in Alfresco's 
+ * FLOSS exception.  You should have received a copy of the text describing 
+ * the FLOSS exception, and it is also available here: 
+ * http://www.alfresco.com/legal/licensing"
+ */
+package org.alfresco.repo.security.sync.ldap;
+
+import org.alfresco.repo.security.authentication.AuthenticationException;
+
+/**
+ * An interface for objects capable of resolving user IDs to full LDAP Distinguished Names (DNs).
+ * 
+ * @author dward
+ */
+public interface LDAPNameResolver
+{
+    
+    /**
+     * Resolves a user ID to a distinguished name.
+     * 
+     * @param userId
+     *            the user id
+     * @return the DN
+     * @throws AuthenticationException
+     *             if the user ID cannot be resolved
+     */
+    public String resolveDistinguishedName(String userId) throws AuthenticationException;
+}

source/java/org/alfresco/repo/security/sync/ldap/LDAPUserRegistry.java

@@ -25,11 +25,15 @@
 package org.alfresco.repo.security.sync.ldap;
 
 import java.text.DateFormat;
+import java.text.MessageFormat;
 import java.text.ParseException;
 import java.text.SimpleDateFormat;
+import java.util.AbstractCollection;
+import java.util.Collection;
 import java.util.Date;
 import java.util.HashSet;
 import java.util.Iterator;
+import java.util.Locale;
 import java.util.Map;
 import java.util.Set;
 import java.util.TimeZone;
@@ -41,6 +45,7 @@ import javax.naming.NamingEnumeration;
 import javax.naming.NamingException;
 import javax.naming.directory.Attribute;
 import javax.naming.directory.Attributes;
+import javax.naming.directory.DirContext;
 import javax.naming.directory.InitialDirContext;
 import javax.naming.directory.SearchControls;
 import javax.naming.directory.SearchResult;
@@ -49,6 +54,7 @@ import javax.naming.ldap.LdapName;
 import org.alfresco.error.AlfrescoRuntimeException;
 import org.alfresco.model.ContentModel;
 import org.alfresco.repo.management.subsystems.ActivateableBean;
+import org.alfresco.repo.security.authentication.AuthenticationException;
 import org.alfresco.repo.security.authentication.ldap.LDAPInitialDirContextFactory;
 import org.alfresco.repo.security.sync.NodeDescription;
 import org.alfresco.repo.security.sync.UserRegistry;
@@ -65,14 +71,18 @@ import org.springframework.beans.factory.InitializingBean;
  * 
  * @author dward
  */
-public class LDAPUserRegistry implements UserRegistry, InitializingBean, ActivateableBean
+public class LDAPUserRegistry implements UserRegistry, LDAPNameResolver, InitializingBean, ActivateableBean
 {
+
     /** The logger. */
     private static Log logger = LogFactory.getLog(LDAPUserRegistry.class);
 
     /** Is this bean active? I.e. should this part of the subsystem be used? */
     private boolean active = true;
 
+    /** Enable progress estimation? When enabled, the user query has to be run twice in order to count entries. */
+    private boolean enableProgressEstimation = true;
+
     /** The group query. */
     private String groupQuery = "(objectclass=groupOfNames)";
 
@@ -127,27 +137,30 @@ public class LDAPUserRegistry implements UserRegistry, InitializingBean, Activat
      */
     private int queryBatchSize;
 
-    /** Should we error on missing group members? */
+    /** Should we error on missing group members?. */
     private boolean errorOnMissingMembers;
 
-    /** Should we error on duplicate group IDs? */
+    /** Should we error on duplicate group IDs?. */
     private boolean errorOnDuplicateGID;
 
-    /** Should we error on missing group IDs? */
+    /** Should we error on missing group IDs?. */
     private boolean errorOnMissingGID = false;
 
-    /** Should we error on missing user IDs? */
+    /** Should we error on missing user IDs?. */
     private boolean errorOnMissingUID = false;
 
-    /** An array of all LDAP attributes to be queried from users */
+    /** An array of all LDAP attributes to be queried from users. */
     private String[] userAttributeNames;
 
-    /** An array of all LDAP attributes to be queried from groups */
+    /** An array of all LDAP attributes to be queried from groups. */
     private String[] groupAttributeNames;
 
     /** The LDAP generalized time format. */
     private DateFormat timestampFormat;
 
+    /**
+     * Instantiates a new lDAP user registry.
+     */
     public LDAPUserRegistry()
     {
         // Default to official LDAP generalized time format (unfortunately not used by Active Directory)
@@ -155,7 +168,7 @@ public class LDAPUserRegistry implements UserRegistry, InitializingBean, Activat
     }
 
     /**
-     * Indicates whether this bean is active. I.e. should this part of the subsystem be used?
+     * Controls whether this bean is active. I.e. should this part of the subsystem be used?
      * 
      * @param active
      *            <code>true</code> if this bean is active
@@ -165,6 +178,18 @@ public class LDAPUserRegistry implements UserRegistry, InitializingBean, Activat
         this.active = active;
     }
 
+    /**
+     * Controls whether progress estimation is enabled. When enabled, the user query has to be run twice in order to
+     * count entries.
+     * 
+     * @param enableProgressEstimation
+     *            <code>true</code> if progress estimation is enabled
+     */
+    public void setEnableProgressEstimation(boolean enableProgressEstimation)
+    {
+        this.enableProgressEstimation = enableProgressEstimation;
+    }
+
     /**
      * Sets the group id attribute name.
      * 
@@ -309,7 +334,7 @@ public class LDAPUserRegistry implements UserRegistry, InitializingBean, Activat
      */
     public void setTimestampFormat(String timestampFormat)
     {
-        this.timestampFormat = new SimpleDateFormat(timestampFormat);
+        this.timestampFormat = new SimpleDateFormat(timestampFormat, Locale.UK);
         this.timestampFormat.setTimeZone(TimeZone.getTimeZone("GMT"));
     }
 
@@ -450,59 +475,148 @@ public class LDAPUserRegistry implements UserRegistry, InitializingBean, Activat
      * (non-Javadoc)
      * @see org.alfresco.repo.security.sync.UserRegistry#getPersons(java.util.Date)
      */
-    public Iterator<NodeDescription> getPersons(Date modifiedSince)
+    public Collection<NodeDescription> getPersons(Date modifiedSince)
     {
-        return new PersonIterator(modifiedSince);
+        return new PersonCollection(modifiedSince);
+    }
+
+    /**
+     * Retrieves the complete set of known users and groups from the LDAP directory and removes them from the set of
+     * candidate local authorities to be deleted.
+     * 
+     * @param candidateAuthoritiesForDeletion
+     *            the candidate authorities for deletion
+     */
+    private void processDeletions(final Set<String> candidateAuthoritiesForDeletion)
+    {
+        processQuery(new SearchCallback()
+        {
+            public void process(SearchResult result) throws NamingException, ParseException
+            {
+                Attribute nameAttribute = result.getAttributes().get(LDAPUserRegistry.this.userIdAttributeName);
+                if (nameAttribute == null)
+                {
+                    if (LDAPUserRegistry.this.errorOnMissingUID)
+                    {
+                        throw new AlfrescoRuntimeException("User missing user id attribute DN ="
+                                + result.getNameInNamespace() + "  att = " + LDAPUserRegistry.this.userIdAttributeName);
+                    }
+                    else
+                    {
+                        LDAPUserRegistry.logger.warn("User missing user id attribute DN ="
+                                + result.getNameInNamespace() + "  att = " + LDAPUserRegistry.this.userIdAttributeName);
+                    }
+                }
+                else
+                {
+                    String authority = (String) nameAttribute.get();
+                    candidateAuthoritiesForDeletion.remove(authority);
+                }
+            }
+
+            public void close() throws NamingException
+            {
+            }
+
+        }, this.userSearchBase, this.personQuery, new String[]
+        {
+            this.userIdAttributeName
+        });
+        processQuery(new SearchCallback()
+        {
+
+            public void process(SearchResult result) throws NamingException, ParseException
+            {
+                Attribute nameAttribute = result.getAttributes().get(LDAPUserRegistry.this.groupIdAttributeName);
+                if (nameAttribute == null)
+                {
+                    if (LDAPUserRegistry.this.errorOnMissingGID)
+                    {
+                        throw new AlfrescoRuntimeException(
+                                "NodeDescription returned by group search does not have mandatory group id attribute "
+                                        + result.getNameInNamespace());
+                    }
+                    else
+                    {
+                        LDAPUserRegistry.logger.warn("Missing GID on " + result.getNameInNamespace());
+                    }
+                }
+                else
+                {
+                    String authority = "GROUP_" + (String) nameAttribute.get();
+                    candidateAuthoritiesForDeletion.remove(authority);
+                }
+            }
+
+            public void close() throws NamingException
+            {
+            }
+
+        }, this.groupSearchBase, this.groupQuery, new String[]
+        {
+            this.groupIdAttributeName
+        });
     }
 
     /*
      * (non-Javadoc)
      * @see org.alfresco.repo.security.sync.UserRegistry#getGroups(java.util.Date)
      */
-    public Iterator<NodeDescription> getGroups(Date modifiedSince)
+    public Collection<NodeDescription> getGroups(Date modifiedSince, final Set<String> candidateAuthoritiesForDeletion,
+            boolean prune)
     {
-        Map<String, NodeDescription> lookup = new TreeMap<String, NodeDescription>();
-        SearchControls userSearchCtls = new SearchControls();
-        userSearchCtls.setSearchScope(SearchControls.SUBTREE_SCOPE);
-        userSearchCtls.setReturningAttributes(this.groupAttributeNames);
+        // Take the given set of authorities as a starting point for the set of all authorities
+        final Set<String> allAuthorities = new TreeSet<String>(candidateAuthoritiesForDeletion);
 
-        InitialDirContext ctx = null;
-        try
+        // If required, work out what authority deletions are required by pruning down the deletion set and the set of
+        // all authorities
+        if (prune)
         {
-            ctx = this.ldapInitialContextFactory.getDefaultIntialDirContext(this.queryBatchSize);
-
-            LdapName groupDistinguishedNamePrefix = new LdapName(this.groupSearchBase);
-            LdapName userDistinguishedNamePrefix = new LdapName(this.userSearchBase);
+            processDeletions(candidateAuthoritiesForDeletion);
+            allAuthorities.removeAll(candidateAuthoritiesForDeletion);
+        }
 
         // Work out whether the user and group trees are disjoint. This may allow us to optimize reverse DN
         // resolution.
-            boolean disjoint = !groupDistinguishedNamePrefix.startsWith(userDistinguishedNamePrefix)
+        final LdapName groupDistinguishedNamePrefix;
+        final LdapName userDistinguishedNamePrefix;
+        try
+        {
+            groupDistinguishedNamePrefix = new LdapName(this.groupSearchBase.toLowerCase());
+            userDistinguishedNamePrefix = new LdapName(this.userSearchBase.toLowerCase());
+        }
+        catch (InvalidNameException e)
+        {
+            throw new AlfrescoRuntimeException("User and group import failed", e);
+        }
+        final boolean disjoint = !groupDistinguishedNamePrefix.startsWith(userDistinguishedNamePrefix)
                 && !userDistinguishedNamePrefix.startsWith(groupDistinguishedNamePrefix);
 
-            do
-            {
-                NamingEnumeration<SearchResult> searchResults;
-
+        // Choose / generate the query
+        String query;
         if (modifiedSince == null)
         {
-                    searchResults = ctx.search(this.groupSearchBase, this.groupQuery, userSearchCtls);
+            query = this.groupQuery;
         }
         else
         {
-                    searchResults = ctx.search(this.groupSearchBase, this.groupDifferentialQuery, new Object[]
-                    {
-                        this.timestampFormat.format(modifiedSince)
-                    }, userSearchCtls);
+            query = MessageFormat.format(this.groupDifferentialQuery, this.timestampFormat.format(modifiedSince));
         }
 
-                while (searchResults.hasMoreElements())
+        // Run the query and process the results
+        final Map<String, NodeDescription> lookup = new TreeMap<String, NodeDescription>();
+        processQuery(new SearchCallback()
+        {
+            // We get a whole new context to avoid interference with cookies from paged results
+            private DirContext ctx = LDAPUserRegistry.this.ldapInitialContextFactory.getDefaultIntialDirContext();
+
+            public void process(SearchResult result) throws NamingException, ParseException
             {
-                    SearchResult result = searchResults.next();
                 Attributes attributes = result.getAttributes();
-                    Attribute gidAttribute = attributes.get(this.groupIdAttributeName);
+                Attribute gidAttribute = attributes.get(LDAPUserRegistry.this.groupIdAttributeName);
                 if (gidAttribute == null)
                 {
-                        if (this.errorOnMissingGID)
+                    if (LDAPUserRegistry.this.errorOnMissingGID)
                     {
                         throw new AlfrescoRuntimeException(
                                 "NodeDescription returned by group search does not have mandatory group id attribute "
@@ -511,7 +625,7 @@ public class LDAPUserRegistry implements UserRegistry, InitializingBean, Activat
                     else
                     {
                         LDAPUserRegistry.logger.warn("Missing GID on " + attributes);
-                            continue;
+                        return;
                     }
                 }
                 String gid = "GROUP_" + gidAttribute.get(0);
@@ -519,11 +633,12 @@ public class LDAPUserRegistry implements UserRegistry, InitializingBean, Activat
                 NodeDescription group = lookup.get(gid);
                 if (group == null)
                 {
-                        group = new NodeDescription();
+                    group = new NodeDescription(result.getNameInNamespace());
                     group.getProperties().put(ContentModel.PROP_AUTHORITY_NAME, gid);
                     lookup.put(gid, group);
+                    allAuthorities.add(gid);
                 }
-                    else if (this.errorOnDuplicateGID)
+                else if (LDAPUserRegistry.this.errorOnDuplicateGID)
                 {
                     throw new AlfrescoRuntimeException("Duplicate group id found for " + gid);
                 }
@@ -532,14 +647,16 @@ public class LDAPUserRegistry implements UserRegistry, InitializingBean, Activat
                     LDAPUserRegistry.logger.warn("Duplicate gid found for " + gid + " -> merging definitions");
                 }
 
-                    Attribute modifyTimestamp = attributes.get(this.modifyTimestampAttributeName);
+                Attribute modifyTimestamp = attributes.get(LDAPUserRegistry.this.modifyTimestampAttributeName);
                 if (modifyTimestamp != null)
                 {
-                        group.setLastModified(this.timestampFormat.parse(modifyTimestamp.get().toString()));
+                    group
+                            .setLastModified(LDAPUserRegistry.this.timestampFormat.parse(modifyTimestamp.get()
+                                    .toString()));
                 }
                 Set<String> childAssocs = group.getChildAssociations();
 
-                    Attribute memAttribute = attributes.get(this.memberAttributeName);
+                Attribute memAttribute = attributes.get(LDAPUserRegistry.this.memberAttributeName);
                 // check for null
                 if (memAttribute != null)
                 {
@@ -552,19 +669,20 @@ public class LDAPUserRegistry implements UserRegistry, InitializingBean, Activat
                             {
                                 // Attempt to parse the member attribute as a DN. If this fails we have a fallback
                                 // in the catch block
-                                    LdapName distinguishedName = new LdapName(attribute);
+                                LdapName distinguishedName = new LdapName(attribute.toLowerCase());
                                 Attribute nameAttribute;
 
                                 // If the user and group search bases are different we may be able to recognize user
                                 // and group DNs without a secondary lookup
                                 if (disjoint)
                                 {
-                                        Attributes nameAttributes = distinguishedName.getRdn(
-                                                distinguishedName.size() - 1).toAttributes();
+                                    Attributes nameAttributes = distinguishedName.getRdn(distinguishedName.size() - 1)
+                                            .toAttributes();
 
                                     // Recognize user DNs
                                     if (distinguishedName.startsWith(userDistinguishedNamePrefix)
-                                                && (nameAttribute = nameAttributes.get(this.userIdAttributeName)) != null)
+                                            && (nameAttribute = nameAttributes
+                                                    .get(LDAPUserRegistry.this.userIdAttributeName)) != null)
                                     {
                                         childAssocs.add((String) nameAttribute.get());
                                         continue;
@@ -572,7 +690,8 @@ public class LDAPUserRegistry implements UserRegistry, InitializingBean, Activat
 
                                     // Recognize group DNs
                                     if (distinguishedName.startsWith(groupDistinguishedNamePrefix)
-                                                && (nameAttribute = nameAttributes.get(this.groupIdAttributeName)) != null)
+                                            && (nameAttribute = nameAttributes
+                                                    .get(LDAPUserRegistry.this.groupIdAttributeName)) != null)
                                     {
                                         childAssocs.add("GROUP_" + nameAttribute.get());
                                         continue;
@@ -585,27 +704,30 @@ public class LDAPUserRegistry implements UserRegistry, InitializingBean, Activat
                                 {
                                     try
                                     {
-                                            Attributes childAttributes = ctx.getAttributes(attribute, new String[]
+                                        Attributes childAttributes = this.ctx.getAttributes(attribute, new String[]
                                         {
-                                                "objectclass", this.groupIdAttributeName, this.userIdAttributeName
+                                            "objectclass", LDAPUserRegistry.this.groupIdAttributeName,
+                                            LDAPUserRegistry.this.userIdAttributeName
                                         });
                                         Attribute objectClass = childAttributes.get("objectclass");
-                                            if (hasAttributeValue(objectClass, this.personType))
+                                        if (hasAttributeValue(objectClass, LDAPUserRegistry.this.personType))
                                         {
-                                                nameAttribute = childAttributes.get(this.userIdAttributeName);
+                                            nameAttribute = childAttributes
+                                                    .get(LDAPUserRegistry.this.userIdAttributeName);
                                             if (nameAttribute == null)
                                             {
-                                                    if (this.errorOnMissingUID)
+                                                if (LDAPUserRegistry.this.errorOnMissingUID)
                                                 {
                                                     throw new AlfrescoRuntimeException(
                                                             "User missing user id attribute DN =" + attribute
-                                                                        + "  att = " + this.userIdAttributeName);
+                                                                    + "  att = "
+                                                                    + LDAPUserRegistry.this.userIdAttributeName);
                                                 }
                                                 else
                                                 {
-                                                        LDAPUserRegistry.logger
-                                                                .warn("User missing user id attribute DN =" + attribute
-                                                                        + "  att = " + this.userIdAttributeName);
+                                                    LDAPUserRegistry.logger.warn("User missing user id attribute DN ="
+                                                            + attribute + "  att = "
+                                                            + LDAPUserRegistry.this.userIdAttributeName);
                                                     continue;
                                                 }
                                             }
@@ -613,12 +735,13 @@ public class LDAPUserRegistry implements UserRegistry, InitializingBean, Activat
                                             childAssocs.add((String) nameAttribute.get());
                                             continue;
                                         }
-                                            else if (hasAttributeValue(objectClass, this.groupType))
+                                        else if (hasAttributeValue(objectClass, LDAPUserRegistry.this.groupType))
                                         {
-                                                nameAttribute = childAttributes.get(this.groupIdAttributeName);
+                                            nameAttribute = childAttributes
+                                                    .get(LDAPUserRegistry.this.groupIdAttributeName);
                                             if (nameAttribute == null)
                                             {
-                                                    if (this.errorOnMissingGID)
+                                                if (LDAPUserRegistry.this.errorOnMissingGID)
                                                 {
                                                     throw new AlfrescoRuntimeException(
                                                             "Group returned by group search does not have mandatory group id attribute "
@@ -626,8 +749,7 @@ public class LDAPUserRegistry implements UserRegistry, InitializingBean, Activat
                                                 }
                                                 else
                                                 {
-                                                        LDAPUserRegistry.logger.warn("Missing GID on "
-                                                                + childAttributes);
+                                                    LDAPUserRegistry.logger.warn("Missing GID on " + childAttributes);
                                                     continue;
                                                 }
                                             }
@@ -638,9 +760,17 @@ public class LDAPUserRegistry implements UserRegistry, InitializingBean, Activat
                                     catch (NamingException e)
                                     {
                                         // Unresolvable name
+                                        if (LDAPUserRegistry.this.errorOnMissingMembers)
+                                        {
+                                            throw new AlfrescoRuntimeException("Failed to resolve distinguished name: "
+                                                    + attribute, e);
+                                        }
+                                        LDAPUserRegistry.logger.warn("Failed to resolve distinguished name: "
+                                                + attribute, e);
+                                        continue;
                                     }
                                 }
-                                    if (this.errorOnMissingMembers)
+                                if (LDAPUserRegistry.this.errorOnMissingMembers)
                                 {
                                     throw new AlfrescoRuntimeException("Failed to resolve distinguished name: "
                                             + attribute);
@@ -649,22 +779,69 @@ public class LDAPUserRegistry implements UserRegistry, InitializingBean, Activat
                             }
                             catch (InvalidNameException e)
                             {
-                                    // The member attribute didn't parse as a DN. So assume we have a group class like posixGroup (FDS) that directly lists user names
+                                // The member attribute didn't parse as a DN. So assume we have a group class like
+                                // posixGroup (FDS) that directly lists user names
                                 childAssocs.add(attribute);
                             }
                         }
                     }
                 }
             }
+
+            public void close() throws NamingException
+            {
+                this.ctx.close();
             }
-            while (this.ldapInitialContextFactory.hasNextPage(ctx, this.queryBatchSize));
+        }, this.groupSearchBase, query, this.groupAttributeNames);
 
         if (LDAPUserRegistry.logger.isDebugEnabled())
         {
             LDAPUserRegistry.logger.debug("Found " + lookup.size());
         }
 
-            return lookup.values().iterator();
+        // Post-process the group associations to filter out those that point to excluded users or groups (now that we
+        // know the full set of groups)
+        for (NodeDescription group : lookup.values())
+        {
+            group.getChildAssociations().retainAll(allAuthorities);
+        }
+        return lookup.values();
+    }
+
+    /**
+     * Invokes the given callback on each entry returned by the given query.
+     * 
+     * @param callback
+     *            the callback
+     * @param searchBase
+     *            the base DN for the search
+     * @param query
+     *            the query
+     * @param returningAttributes
+     *            the attributes to include in search results
+     */
+    private void processQuery(SearchCallback callback, String searchBase, String query, String[] returningAttributes)
+    {
+        SearchControls searchControls = new SearchControls();
+        searchControls.setSearchScope(SearchControls.SUBTREE_SCOPE);
+        searchControls.setReturningAttributes(returningAttributes);
+
+        InitialDirContext ctx = null;
+        try
+        {
+            ctx = this.ldapInitialContextFactory.getDefaultIntialDirContext(this.queryBatchSize);
+            do
+            {
+                NamingEnumeration<SearchResult> searchResults;
+                searchResults = ctx.search(searchBase, query, searchControls);
+
+                while (searchResults.hasMoreElements())
+                {
+                    SearchResult result = searchResults.next();
+                    callback.process(result);
+                }
+            }
+            while (this.ldapInitialContextFactory.hasNextPage(ctx, this.queryBatchSize));
         }
         catch (NamingException e)
         {
@@ -675,6 +852,57 @@ public class LDAPUserRegistry implements UserRegistry, InitializingBean, Activat
             throw new AlfrescoRuntimeException("User and group import failed", e);
         }
         finally
+        {
+            if (ctx != null)
+            {
+                try
+                {
+                    ctx.close();
+                }
+                catch (NamingException e)
+                {
+                }
+            }
+            try
+            {
+                callback.close();
+            }
+            catch (NamingException e)
+            {
+            }
+        }
+    }
+
+    /*
+     * (non-Javadoc)
+     * @see org.alfresco.repo.security.sync.ldap.LDAPNameResolver#resolveDistinguishedName(java.lang.String)
+     */
+    public String resolveDistinguishedName(String userId) throws AuthenticationException
+    {
+        SearchControls userSearchCtls = new SearchControls();
+        userSearchCtls.setSearchScope(SearchControls.SUBTREE_SCOPE);
+        userSearchCtls.setReturningAttributes(new String[] {});
+        InitialDirContext ctx = null;
+        try
+        {
+            ctx = this.ldapInitialContextFactory.getDefaultIntialDirContext();
+
+            // Execute the user query with an additional condition that ensures only the user with the required ID is
+            // returned
+            NamingEnumeration<SearchResult> searchResults = ctx.search(this.userSearchBase, "(&" + this.personQuery
+                    + "(" + this.userIdAttributeName + "=" + userId + "))", userSearchCtls);
+
+            if (searchResults.hasMoreElements())
+            {
+                return searchResults.next().getNameInNamespace();
+            }
+            throw new AuthenticationException("Failed to resolve user: " + userId);
+        }
+        catch (NamingException e)
+        {
+            throw new AlfrescoRuntimeException("Failed to resolve user ID: " + userId, e);
+        }
+        finally
         {
             if (ctx != null)
             {
@@ -690,7 +918,7 @@ public class LDAPUserRegistry implements UserRegistry, InitializingBean, Activat
     }
 
     /**
-     * Does a case-insensitive search for the given value in an attribute
+     * Does a case-insensitive search for the given value in an attribute.
      * 
      * @param attribute
      *            the attribute
@@ -721,18 +949,107 @@ public class LDAPUserRegistry implements UserRegistry, InitializingBean, Activat
     }
 
     /**
-     * Wraps the LDAP user query as an {@link Iterator}.
+     * Wraps the LDAP user query as a virtual {@link Collection}.
      */
-    public class PersonIterator implements Iterator<NodeDescription>
+    public class PersonCollection extends AbstractCollection<NodeDescription>
+    {
+
+        /** The query. */
+        private String query;
+
+        /** The total estimated size. */
+        private int totalEstimatedSize;
+
+        /**
+         * Instantiates a new person collection.
+         * 
+         * @param modifiedSince
+         *            if non-null, then only descriptions of users modified since this date should be returned; if
+         *            <code>null</code> then descriptions of all users should be returned.
+         */
+        public PersonCollection(Date modifiedSince)
+        {
+            // Choose / generate the appropriate query
+            if (modifiedSince == null)
+            {
+                this.query = LDAPUserRegistry.this.personQuery;
+            }
+            else
+            {
+                this.query = MessageFormat.format(LDAPUserRegistry.this.personDifferentialQuery,
+                        LDAPUserRegistry.this.timestampFormat.format(modifiedSince));
+            }
+
+            // Estimate the size of this collection by running the entire query once, if progress
+            // estimation is enabled
+            if (LDAPUserRegistry.this.enableProgressEstimation)
+            {
+                class CountingCallback implements SearchCallback
+                {
+                    int count;
+
+                    /*
+                     * (non-Javadoc)
+                     * @see
+                     * org.alfresco.repo.security.sync.ldap.LDAPUserRegistry.SearchCallback#process(javax.naming.directory
+                     * .SearchResult)
+                     */
+                    public void process(SearchResult result) throws NamingException, ParseException
+                    {
+                        this.count++;
+                    }
+
+                    /*
+                     * (non-Javadoc)
+                     * @see org.alfresco.repo.security.sync.ldap.LDAPUserRegistry.SearchCallback#close()
+                     */
+                    public void close() throws NamingException
+                    {
+                    }
+
+                }
+                CountingCallback countingCallback = new CountingCallback();
+                processQuery(countingCallback, LDAPUserRegistry.this.userSearchBase, this.query, new String[] {});
+                this.totalEstimatedSize = countingCallback.count;
+            }
+            else
+            {
+                this.totalEstimatedSize = -1;
+            }
+        }
+
+        /*
+         * (non-Javadoc)
+         * @see java.util.AbstractCollection#iterator()
+         */
+        @Override
+        public Iterator<NodeDescription> iterator()
+        {
+            return new PersonIterator();
+        }
+
+        /*
+         * (non-Javadoc)
+         * @see java.util.AbstractCollection#size()
+         */
+        @Override
+        public int size()
+        {
+            return this.totalEstimatedSize;
+        }
+
+        /**
+         * An iterator over the person collection. Wraps the LDAP query in 'real time'.
+         */
+        private class PersonIterator implements Iterator<NodeDescription>
         {
 
             /** The directory context. */
             private InitialDirContext ctx;
 
+            /** The user search controls. */
             private SearchControls userSearchCtls;
 
-        private Date modifiedSince;
-
             /** The search results. */
             private NamingEnumeration<SearchResult> searchResults;
 
@@ -744,27 +1061,19 @@ public class LDAPUserRegistry implements UserRegistry, InitializingBean, Activat
 
             /**
              * Instantiates a new person iterator.
-         * 
-         * @param modifiedSince
-         *            if non-null, then only descriptions of users modified since this date should be returned; if
-         *            <code>null</code> then descriptions of all users should be returned.
              */
-        public PersonIterator(Date modifiedSince)
+            public PersonIterator()
             {
                 try
                 {
+
                     this.ctx = LDAPUserRegistry.this.ldapInitialContextFactory
                             .getDefaultIntialDirContext(LDAPUserRegistry.this.queryBatchSize);
 
-                // Authentication has been successful.
-                // Set the current user, they are now authenticated.
-
                     this.userSearchCtls = new SearchControls();
                     this.userSearchCtls.setSearchScope(SearchControls.SUBTREE_SCOPE);
                     this.userSearchCtls.setReturningAttributes(LDAPUserRegistry.this.userAttributeNames);
 
-                this.modifiedSince = modifiedSince;
-
                     this.next = fetchNext();
                 }
                 catch (NamingException e)
@@ -868,15 +1177,15 @@ public class LDAPUserRegistry implements UserRegistry, InitializingBean, Activat
                             LDAPUserRegistry.logger.debug("Adding user for " + uid);
                         }
 
-                    NodeDescription person = new NodeDescription();
+                        NodeDescription person = new NodeDescription(result.getNameInNamespace());
 
                         Attribute modifyTimestamp = attributes.get(LDAPUserRegistry.this.modifyTimestampAttributeName);
                         if (modifyTimestamp != null)
                         {
                             try
                             {
-                            person.setLastModified(LDAPUserRegistry.this.timestampFormat.parse(modifyTimestamp.get()
-                                    .toString()));
+                                person.setLastModified(LDAPUserRegistry.this.timestampFormat.parse(modifyTimestamp
+                                        .get().toString()));
                             }
                             catch (ParseException e)
                             {
@@ -932,20 +1241,9 @@ public class LDAPUserRegistry implements UserRegistry, InitializingBean, Activat
 
                     // Fetch the next page if there is one
                     if (readyForNextPage)
-                {
-                    if (this.modifiedSince == null)
                     {
                         this.searchResults = this.ctx.search(LDAPUserRegistry.this.userSearchBase,
-                                LDAPUserRegistry.this.personQuery, this.userSearchCtls);
-                    }
-                    else
-                    {
-                        this.searchResults = this.ctx.search(LDAPUserRegistry.this.userSearchBase,
-                                LDAPUserRegistry.this.personDifferentialQuery, new Object[]
-                                {
-                                    LDAPUserRegistry.this.timestampFormat.format(this.modifiedSince)
-                                }, this.userSearchCtls);
-                    }
+                                PersonCollection.this.query, this.userSearchCtls);
                     }
                 }
                 while (readyForNextPage);
@@ -964,5 +1262,35 @@ public class LDAPUserRegistry implements UserRegistry, InitializingBean, Activat
             {
                 throw new UnsupportedOperationException();
             }
+        };
     }
+
+    /**
+     * An interface for callbacks passed to the
+     * {@link LDAPUserRegistry#processQuery(SearchCallback, String, String, String[])} method.
+     */
+    protected static interface SearchCallback
+    {
+
+        /**
+         * Processes the given search result.
+         * 
+         * @param result
+         *            the result
+         * @throws NamingException
+         *             on naming exceptions
+         * @throws ParseException
+         *             on parse exceptions
+         */
+        public void process(SearchResult result) throws NamingException, ParseException;
+
+        /**
+         * Release any resources held by the callback.
+         * 
+         * @throws NamingException
+         *             the naming exception
+         */
+        public void close() throws NamingException;
+    }
+
 }

source/java/org/alfresco/repo/template/People.java

@@ -227,6 +227,19 @@ public class People extends BaseTemplateProcessorExtension
         return this.authorityService.isAdminAuthority((String)person.getProperties().get(ContentModel.PROP_USERNAME));
     }
     
+    /**
+     * Return true if the specified user is an Administrator authority.
+     * 
+     * @param person to test
+     * 
+     * @return true if an admin, false otherwise
+     */
+    public boolean isGuest(TemplateNode person)
+    {
+        ParameterCheck.mandatory("Person", person);
+        return this.authorityService.isGuestAuthority((String)person.getProperties().get(ContentModel.PROP_USERNAME));
+    }
+
     /**
      * Return true if the specified user account is enabled.
      *  

source/test-resources/sync-test-context.xml

@@ -24,6 +24,9 @@
         <property name="sourceBeanName">
             <value>userRegistry</value>
         </property>
+        <property name="loggingInterval">
+            <value>10</value>
+        </property>
     </bean>
 
     <!--  A fake applicaton context manager into which we can inject test-specific beans -->