Dave Ward 2e62d4fb29 Merged DEV/ALAN/SITE_PERF to HEAD
30342: Dev branch for Site performance issues (including rework of AuthorityService.getAuthorities() to use a 'lazy' set and DM indexing rework)
   ALF-9899 Huge share site migration, add group to site and user access site related performance issue.
   ALF-9208 Performance issue, during load tests /share/page/user/user-sites is showing to be the most expensive.
   ALF-9692 Performance: General performance of Alfresco degrades when there are 1000s of sites present
   - ancestor-preloading
   - hasAuthority
   - huge site test
   30370: - Save changed to do with adding childAuthorityCache to AuthorityDAOImpl
   - Increase aspectsTransactionalCache size as it blows up
   30387: Experimental solution to 'cascading reindex' performance problem
   - Now only Lucene container documents for a single subtree are reprocessed on addition / removal of a secondary child association
   - No need to delete and re-evaluate ALL the paths to all the nodes in the subtree - just the paths within the subtree
   - Lucene deltas now store the IDs of ANCESTORs to mask out as well as documents to reindex
   - Merge handles deletion of these efficiently
   - Node service cycle checks changed from getPaths to recursive cycleCheck method
   - Adding a group to 60,000 sites might not require all paths to all sites to be re-evaluated on every change!
   30389: Missed files from last checkin
   30390: Optimizations / fixes to Alan's test!
   30393: Bug fix - wasn't adding new documents into the index!
   30397: Fixed a problem with bulk loading trying to bulk load zero parent associations
   Also tweaked reindex calls
   30399: Correction - don't cascade below containers during path cascading
   30400: Another optimization - no need to trigger node bulk loading during path cascading - pass false for the preload flag
   30404: Further optimizations
   - On creation of a secondary child association, make a decision on whether it is cheaper to cascade reindex the parent or the child, based on the number of parent associations to the child
     - Assumes that if there are more than 5 parent associations, it's cheaper to cascade reindex the parent
     - Add a new authority to a zone (containing 60,000 authorities) - cascade reindex the authority, not the zone
     - Add a group (in 60,000 sites) to a site - cascade reindex the site, not the group
   - Caching of child associations already traversed during cascade reindexing
   - Site creation time much reduced!
   30407: Logic fix: Use 'delete only nodes' behaviour on DM index filtering and merging, now we are managing container deletions separately
   30408: Small correction related to last change.
   30409: Correction to deletion reindex behaviour (no need to regenerate masked out containers)
   - Site CRUD operations now all sub-second with 60,000 sites!
   30410: Stop the heartbeat from trying to load and count all site groups
   - Too expensive, as we might have 60,000 sites, each with 4 groups
   - Now just counts the groups in the default zone (the UI visible ones)
   30411: Increased lucene parameters to allow for 'path explosion'
   - 9 million lucene documents in my index after creating 60,000 Share sites (most of them probably paths) resulting in sluggish index write performance
   - Set lucene.indexer.mergerTargetIndexCount=8 (142 documents in smallest index)
   - Increased lucene.indexer.maxDocsForInMemoryMerge, lucene.indexer.maxDocsForInMemoryIndex
   30412: Test fixes
   30413: Revert 'parent association batch loading' changes (as it was a bad idea and is no longer necessary!)
   - Retain a few caching bug fixes however
   30416: Moved UserAuthoritySet (lazy load authority set) from PermissionServiceImpl to AuthorityServiceImpl
   30418: - Remove 'new' hasAuthority from authorityService so it is back to where we started.
   - SiteServiceHugeTest minor changes
   30421: Prevent creation of a duplicate root node on updating the root
   - Use the ANCESTOR field rather than ISCONTAINER to detect a node document, as the root node is both a container and a node!
   30447: Pulled new indexing behaviour into ADMLuceneIndexerImpl and restored old behaviour to AVMLuceneIndexerImpl to restore normal AVM behaviour
   30448: - Cache in PermissionServiceImpl cleared if an authority container has an association added or removed
     Supports the generateKey method which includes the username
     Supports changes in group structures
   - Moved logic to do with ROLE_GUEST from PermissionServiceImpl to AuthorityServiceImpl 
   30465: - Tidy up tests in SiteServiceTestHuge 
   30532: - Added getContainingAuthoritiesInZone to AuthorityService
     - Dave Changed PeopleService.getContainerGroups to only return groups in the DEFAULT zone
   - Fixed RM code to use getAuthoritiesForUser method with just the username again.
   30558: Build fixes
   - Fixed cycleCheck to throw a CyclicChildRelationshipException
   - More tidy up of AVM / ADM indexer split
   - Properly control when path generation is cascaded (not required on a full reindex or a tracker transaction)
   - Support indexing of a 'fake root' parent. Ouch my head hurts!
   30588: Build fixes
   - StringIndexOutOfBoundsException in NodeMonitor
   - Corrections to 'node only' delete behaviour
   - Use the PATH field to detect non-leaf nodes (it's the only stored field with which we can recognize the root)
   - Moved DOD5015Test.testVitalRecords() to the end - the only way I could work out how to get the full TestCase to run
   30600: More build fixes
   - Broadcast ALL node deletions to indexer (even those from cascade deletion of primary associations)
     - Allows indexer to wipe out all affected documents from the delta even if some have already been flushed under different parents by an intricate DOD unit test!
   - Pause FTS in DOD5015Test to prevent intermittent test failures (FTS can temporarily leave deleted documents in the index until it catches up)
   - More tidy up of ADMLuceneIndexerImpl
     - flushPending optimized and some unnecessary member variables removed
     - correction to cascade deletion behaviour (leave behind containers of unaffected secondary references)
     - unused MOVE action removed
     - further legacy logic moved into AVMLuceneIndexerImpl
   30620: More build fixes
   - Cope with a node morphing from a 'leaf' to a container during its lifetime
   - Container documents now created lazily in index as and when necessary
   - Blank out 'nth sibling' field of synthesized paths
   - ADMLuceneTest now passes!
   - TaggingServiceImplTest also passes - more special treatment for categories
   30627: Multi tenancy fixes
   30629: Possible build fix - retrying transaction in ReplicationServiceIntegrationTest.tearDown()
   30632: Build fix - lazy container generation after a move
   30636: Build fix: authority comparisons are case sensitive, even when that authority corresponds to a user (PermissionServiceTest.testPermissionCase())
   30638: Run SiteServiceTestHuge form a cmd line
      set SITE_CPATH=%TOMCAT_HOME%/lib/*;%TOMCAT_HOME%/endorsed/*;%TOMCAT_HOME%/webapps/alfresco/WEB-INF/lib/*;\
                     %TOMCAT_HOME%/webapps/alfresco/WEB-INF/classes;%TOMCAT_HOME%/shared/classes;
      java -Xmx2048m -XX:MaxPermSize=512M -classpath %SITE_CPATH% org.alfresco.repo.site.SiteServiceTestHuge ...
   
      Usage: -Daction=usersOnly
             -Dfrom=<fromSiteId> -Dto=<toSiteId>
             -Dfrom=<fromSiteId> -Dto=<toSiteId> -Daction=sites  -Drestart=<restartAtSiteId>
             -Dfrom=<fromSiteId> -Dto=<toSiteId> -Daction=groups -Drestart=<restartAtSiteId>
   30639: Minor changes to commented out command line code for SiteServiceTestHuge
   30643: Round of improvements to MySites dashlet relating to huge DB testing:
    - 10,000 site database, user is a member of ~2000 sites
    - Improvements to site.lib.ftl and related SiteService methods
    - To return MySites dashlet for the user, order of magnitude improvement from 7562ms to 618ms in the profiler (now ~350ms in the browser)
   30644: Fixed performance regression - too much opening and closing of the delta reader and writer
   30661: More reader opening / closing
   30668: Performance improvements to Site Finder and My Sites in user profile page.
    - faster to bring back lists and site memberships (used by the Site Finder)
    - related further improvements to APIs used by this and My Sites on dashboard
   30713: Configuration for MySites dashlet maximum list size
   30725: Merged V3.4-BUG-FIX to DEV/ALAN/SITE_PERF
      30708: ALF-10040: Added missing ReferenceCountingReadOnlyIndexReaderFactory wrapper to IndexInfo.getMainIndexReferenceCountingReadOnlyIndexReader() to make it consistent with IndexInfo.getMainIndexReferenceCountingReadOnlyIndexReader(String, Set<String>, boolean) and allow SingleFieldSelectors to make it through from LeafScorer to the path caches! Affects ALL Lucene queries that run OUTSIDE of a transaction.
   30729: Use getAuthoritiesForUser rather than getContainingAuthorities if possible.
   SiteServiceTestHuge: command line version
   30733: Performance improves to user dashboard relating to User Calendar 
    - converted web-tier calendar dashlet to Ajax client-side rendering - faster user experience and also less load on the web-tier
    - improvements to query from Andy
    - maximum sites/list size to query now configurable (default 100 instead of previously 1000)
   30743: Restore site CRUD performance from cold caches
   - Introduced NodeService.getAllRootNodes(), returning all nodes in a store with the root aspect, backed by a transactional cache and invalidated at key points
   - Means indexing doesn't have to load all parent nodes just to check for 'fake roots'
   - Site CRUD performance now back to sub-second with 60,000 nodes
   30747: Improvement to previous checkin - prevent cross cluster invalidation of every store root when a single store drops out of the cache
   30748: User dashboard finally loading within seconds with 60,000 sites, 60 groups, 100 users (thanks mostly to Kev's UI changes)
   - post-process IBatis mapped statements with MySQL dialect to apply fetchSize=Integer.MIN_VALUE to all _Limited statements
      - Means we can stream first 10,000 site groups without the MySQL JDBC driver reading all 240,000 into memory
   - New NodeService getChildAssocs method with a maxResults argument (makes use of the above)
   - Perfected getContainingAuthoritiesInZone implementation, adding a cutoff parameter, allowing only the first 1000 site memberships to be returned quickly and caches to be warmed for ACL evaluations
   - New cache of first 10,000 groups in APP.SHARE zone
   - Cache sizes tuned for 60,000 site scenario
   - Site service warms caches on bootstrap
   - PreferencesService applies ASPECT_IGNORE_INHERITED_RULES to person node to prevent the rule service trying to crawl the group hierarchy on a preference save
   - WorkflowServiceImpl.getPooledTasks only looks in APP.DEFAULT zone (thus avoiding site group noise)
   30749: Fix compilation errors
   30761: Minor change to SiteServiceTestHuge
   30762: Derek code review: Reworked fetchSize specification for select_ChildAssocsOfParent_Limited statement for MySQL
   - Now fetchSize stated explicitly in a MySQL specific config file resolved by the HierarchicalResourceLoader
   - No need for any Java-based post processing
   30763: Build fix: don't add a user into its own authorities (until specifically asked to)
   30767: Build fix
   - IBatis / MySQL needs a streaming result statement to be run in an isolation transaction (because it doesn't release PreparedStatements until the end)
   30771: Backed out previous change which was fundamentally flawed
   - Resolved underlying problem which was that the select_ChildAssocsOfParent_Limited SQL string needs to be unique in order to not cause confusion in the prepared statement cache
   30772: Backed out previous change which was fundamentally flawed
   - Resolved underlying problem which was that the select_ChildAssocsOfParent_Limited SQL string needs to be unique in order to not cause confusion in the prepared statement cache


git-svn-id: https://svn.alfresco.com/repos/alfresco-enterprise/alfresco/HEAD/root@30797 c4b6b30b-aa2e-2d43-bbcb-ca4b014f7261
2011-09-27 12:24:57 +00:00

366 lines
14 KiB
Java

/*
* Copyright (C) 2005-2010 Alfresco Software Limited.
*
* This file is part of Alfresco
*
* Alfresco is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Alfresco is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with Alfresco. If not, see <http://www.gnu.org/licenses/>.
*/
package org.alfresco.repo.site.script;
import java.util.ArrayList;
import java.util.List;
import org.alfresco.repo.jscript.BaseScopableProcessorExtension;
import org.alfresco.repo.jscript.ScriptNode;
import org.alfresco.repo.security.authentication.AuthenticationUtil;
import org.alfresco.repo.site.SiteServiceImpl;
import org.alfresco.service.ServiceRegistry;
import org.alfresco.service.cmr.repository.NodeRef;
import org.alfresco.service.cmr.site.SiteInfo;
import org.alfresco.service.cmr.site.SiteService;
import org.alfresco.service.cmr.site.SiteVisibility;
import org.alfresco.service.namespace.QName;
import org.springframework.extensions.surf.util.ParameterCheck;
/**
* Script object representing the site service.
*
* @author Roy Wetherall
*/
public class ScriptSiteService extends BaseScopableProcessorExtension
{
/** Visibility helper constants */
public static final String PUBLIC_SITE = "PUBLIC";
public static final String MODERATED_SITE = "MODERATED";
public static final String PRIVATE_SITE = "PRIVATE";
/** Service Registry */
private ServiceRegistry serviceRegistry;
/** The site service */
private SiteService siteService;
/**
* Sets the Service Registry
*
* @param serviceRegistry
*/
public void setServiceRegistry(ServiceRegistry serviceRegistry)
{
this.serviceRegistry = serviceRegistry;
}
/**
* Set the site service
*
* @param siteService the site service
*/
public void setSiteService(SiteService siteService)
{
this.siteService = siteService;
}
/**
* @see {@link #createSite(String, String, String, String, String)}
*
* @param sitePreset site preset
* @param shortName site short name
* @param title site title
* @param description site description
* @param isPublic whether the site is public or not
* @return Site the created site
* @deprecated as of version 3.2, replaced by {@link #createSite(String, String, String, String, String)}
*/
public Site createSite(String sitePreset, String shortName, String title, String description, boolean isPublic)
{
SiteInfo siteInfo = this.siteService.createSite(sitePreset, shortName, title, description, isPublic);
return new Site(siteInfo, this.serviceRegistry, this.siteService, getScope());
}
/**
* Create a new site.
* <p>
* The site short name will be used to uniquely identify the site so it must be unique.
*
* @param sitePreset site preset
* @param shortName site short name
* @param title site title
* @param description site description
* @param visibility visibility of the site (public|moderated|private)
* @return Site the created site
*/
public Site createSite(String sitePreset, String shortName, String title, String description, String visibility)
{
ParameterCheck.mandatoryString("visibility", visibility);
SiteVisibility siteVisibility = SiteVisibility.valueOf(visibility);
SiteInfo siteInfo = this.siteService.createSite(sitePreset, shortName, title, description, siteVisibility);
return new Site(siteInfo, this.serviceRegistry, this.siteService, getScope());
}
/**
* Create a new site.
* <p>
* The site short name will be used to uniquely identify the site so it must be unique.
*
* @param sitePreset site preset
* @param shortName site short name
* @param title site title
* @param description site description
* @param visibility visibility of the site (public|moderated|private)
* @param siteType qname of site type to create
* @return Site the created site
*/
public Site createSite(String sitePreset, String shortName, String title, String description, String visibility, String siteType)
{
ParameterCheck.mandatoryString("visibility", visibility);
SiteVisibility siteVisibility = SiteVisibility.valueOf(visibility);
QName siteTypeQName = QName.createQName(siteType);
SiteInfo siteInfo = this.siteService.createSite(sitePreset, shortName, title, description, siteVisibility, siteTypeQName);
return new Site(siteInfo, this.serviceRegistry, this.siteService, getScope());
}
/**
* This method checks if the currently authenticated user has permission to create sites.
*
* @return <code>true</code> if the current user is a Contributor to "Sites" node, else <code>false</code>.
* @since 3.4
*/
public boolean hasCreateSitePermissions()
{
return siteService.hasCreateSitePermissions();
}
/**
* Determines if the current user is a manager of the given site.
*
* @param siteId The short name of the site to check
* @return true if the current user is a manager of the given site
*/
public boolean isSiteManager(String siteId)
{
String role = siteService.getMembersRole(siteId, AuthenticationUtil.getFullyAuthenticatedUser());
return (SiteServiceImpl.SITE_MANAGER.equals(role));
}
/**
* Retrieves the sites available in the repository. The returned list can optionally be filtered by name and site
* preset. If no filters are specified then all the available sites are returned.
*
* NOTE: If the filter starts with a * a Lucene based search will be performed, this may discover a wider range
* of results i.e. those sites that contain the search term as opposed to those that start with the search term,
* but newly created sites may not be found until the underlying search indexes are updated.
*
* @param filter inclusion filter for returned sites. Only sites whose cm:name OR cm:title
* OR cm:description start with the filter string will be returned.
* @param sitePresetFilter site preset filter
* @param size max results size crop if >0
* @return Site[] a list of the site filtered as appropriate
*/
public Site[] getSites(String filter, String sitePresetFilter, int size)
{
// reset filter if necessary
if (filter != null && (filter.length() == 0 || filter.equals("*")))
{
filter = null;
}
if (filter != null && (filter.startsWith("*")))
{
// findSites will add the wildcard so remove here
filter = filter.substring(1, filter.length());
// use findSites to do a "contains" search
return findSites(filter, sitePresetFilter, size);
}
else
{
// use listSites to do a canned query (will provide consistent results)
return listSites(filter, sitePresetFilter, size);
}
}
/**
* List the sites available in the repository. The returned list can optionally be filtered by name and site
* preset.
* <p>
* If no filters are specified then all the available sites are returned.
*
* @param filter inclusion filter for returned sites. Only sites whose cm:name OR cm:title
* OR cm:description start with the filter string will be returned.
* @param sitePresetFilter site preset filter
* @return Site[] a list of the site filtered as appropriate
*
* @see SiteService#listSites(String, String, int) for a description of the limitations of this method.
*/
public Site[] listSites(String filter, String sitePresetFilter)
{
return listSites(filter, sitePresetFilter, -1);
}
/**
* List the sites available in the repository. The returned list can optionally be filtered by name and site
* preset.
* <p/>
* If no filters are specified then all the available sites are returned.
*
* @param filter inclusion filter for returned sites. Only sites whose cm:name OR cm:title
* OR cm:description start with the filter string will be returned.
* @param sitePresetFilter site preset filter
* @param size max results size crop if >0
* @return Site[] a list of the site filtered as appropriate
*
* @see SiteService#listSites(String, String, int) for a description of the limitations of this method.
*/
public Site[] listSites(String filter, String sitePresetFilter, int size)
{
List<SiteInfo> siteInfos = this.siteService.listSites(filter, sitePresetFilter, size);
return makeSitesArray(siteInfos);
}
/**
* Find (search) the sites available in the repository. The returned list can optionally be filtered by name and site
* preset.
* <p/>
* If no filters are specified then all the available sites are returned.
*
* @param filter inclusion filter for returned sites. Only sites whose cm:name OR cm:title
* OR cm:description CONTAIN the filter string will be returned.
* @param sitePresetFilter site preset filter
* @param size max results size crop if >0
*
* @return Site[] a list of the site filtered as appropriate
*
* @see SiteService#findSites(String, String, int) for a description of the limitations of this method.
* @since 4.0
*/
public Site[] findSites(String filter, String sitePresetFilter, int size)
{
List<SiteInfo> siteInfos = this.siteService.findSites(filter, sitePresetFilter, size);
return makeSitesArray(siteInfos);
}
/**
* Converts the given List of SiteInfo objects to a JavaScript friendly array
* of Site objects.
*
* @param siteInfos
* @return Array of Site objects
*/
protected Site[] makeSitesArray(List<SiteInfo> siteInfos)
{
List<Site> sites = new ArrayList<Site>(siteInfos.size());
for (SiteInfo siteInfo : siteInfos)
{
sites.add(new Site(siteInfo, this.serviceRegistry, this.siteService, getScope()));
}
return sites.toArray(new Site[sites.size()]);
}
/**
* List all the sites that the specified user has an explicit membership to.
*
* @param userName user name
* @param size maximum list size
* @return Site[] a list of sites the user has an explicit membership to
*/
public Site[] listUserSites(String userName, int size)
{
List<SiteInfo> siteInfos = this.siteService.listSites(userName, size);
List<Site> sites = new ArrayList<Site>(siteInfos.size());
for (SiteInfo siteInfo : siteInfos)
{
sites.add(new Site(siteInfo, this.serviceRegistry, this.siteService, getScope()));
}
return sites.toArray(new Site[sites.size()]);
}
/**
* List all the sites that the specified user has an explicit membership to.
*
* @param userName user name
* @return Site[] a list of sites the user has an explicit membership to
*/
public Site[] listUserSites(String userName)
{
return listUserSites(userName, 0);
}
/**
* Get a site for a provided site short name.
* <p>
* Returns null if the site does not exist.
*
* @param shortName short name of the site
* @return Site the site, null if does not exist
*/
public Site getSite(String shortName)
{
Site site = null;
SiteInfo siteInfo = this.siteService.getSite(shortName);
if (siteInfo != null)
{
site = new Site(siteInfo, this.serviceRegistry, this.siteService, getScope());
}
return site;
}
/**
* Returns an array of all the roles that can be assigned to a member of a site.
*
* @return String[] roles available to assign to a member of a site
*/
public String[] listSiteRoles()
{
List<String> roles = this.siteService.getSiteRoles();
return roles.toArray(new String[roles.size()]);
}
/**
* It removes permissions which pertain to sites other than the node's current site.
*
* @param targetNode the root node which is to have its permissions cleaned.
* @see SiteService#cleanSitePermissions(NodeRef, SiteInfo)
*/
public void cleanSitePermissions(NodeRef targetNode)
{
this.siteService.cleanSitePermissions(targetNode, null);
}
/**
* This method cleans up the permissions on the specified node and all its primary children.
* It removes permissions which pertain to sites other than the node's current site.
*
* @since 3.4.2
* @see SiteService#cleanSitePermissions(NodeRef, SiteInfo)
*/
public void cleanSitePermissions(ScriptNode targetNode)
{
this.cleanSitePermissions(targetNode.getNodeRef());
}
/**
* Returns an array of all the roles that can be assigned to a member of a
* specific site.
*
* @return String[] roles available to assign to a member of a site
*/
public String[] listSiteRoles(String shortName)
{
List<String> roles = this.siteService.getSiteRoles(shortName);
return (String[])roles.toArray(new String[roles.size()]);
}
}