inteligr8/SearchServices
1
0
Fork 0
mirror of https://github.com/Alfresco/SearchServices.git synced 2025-09-24 14:31:29 +00:00
Code Issues Packages Projects Releases Wiki Activity

SEARCH-2345: Toolset of Docker Compose templates to help Release testing stage.

Browse Source
This commit is contained in:
Angel Borroy
2020-08-13 14:23:38 +02:00
parent 072241a252
commit 3be6c9ed03
348 changed files with 13771 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

108
e2e-test/release-testing/README.md Normal file
View File

@@ -0,0 +1,108 @@
# release-testing
> Tools and Docker Compose templates for Release Testing
This project includes a collection of Docker Compose Templates to test Repository and Search Services/Insight Engine with different configurations.
Test execution information is available in [Test Execution internal wiki page](https://w3.alfresco.com/confluence/display/ENG/Test+Execution).
## Project structure
Following folders, including Docker Compose templates, are provided.
```
% tree -L 1
.
├── insight-engine
├── insight-engine-sharding
├── insight-engine-sharding-replica
├── insight-engine-zeppelin
├── mtls
├── search-services
├── search-services-backup
├── search-services-disable-tracking
├── search-services-replication
└── search-services-upgrade
```
Configurations have been generated mainly with [generator-alfresco-docker-compose](../generator-alfresco-docker-compose) project.
## Docker Compose templates
Every Docker Compose folder includes a `.env` file with versions settings.
```
ALFRESCO_TAG=6.2.2
ALFRESCO_CE_TAG=6.2.0-ga
SHARE_TAG=6.2.1
POSTGRES_TAG=11.4
TRANSFORM_ROUTER_TAG=1.2.0
TRANSFORM_CORE_AIO_TAG=2.2.1
TRANSFORMER_TAG=2.1.0
SHARED_FILE_STORE_TAG=0.7.0
ACTIVE_MQ_TAG=5.15.8
DIGITAL_WORKSPACE_TAG=1.5.0
ACS_NGINX_TAG=3.0.1
ACS_COMMUNITY_NGINX_TAG=1.0.0
SEARCH_TAG=1.4.3.0-RC1
ZEPPELIN_TAG=1.4.3.0-RC1
ACA_TAG=master-latest
```
Version number must be changed when execution the tests on a new release. Additionally, Docker Compose templates may need some modifications in order to be adapted to new configurations.
Once started, services are available in following URLs:
* http://localhost:8080/alfresco
* http://localhost:8080/share
* http://localhost:8083/solr
* http://localhost:8084/solr (when using Sharding or Replica)
* http://localhost:9090/zeppelin
When using mTLS for Repository and SOLR communication, the endpoint for SOLR servers should be changed to:
* https://localhost:8083
* https://localhost:8084 (when using Sharding or Replica)
Tests requiring additional instructions include an additional `README.md` file.
Tests with persistent storage create a local folder named `data` with the *repository*, *db* and *solr* data.
## Test Catalog
Search via Share: Basic, Advanced, Faceted, Live Search, Highlighting, Language
* [search-services](search-services)
* [insight-engine](insight-engine)
Search configurations: sharding diff types
* [insight-engine-sharding](insight-engine-sharding)
Search configurations: master slave including master-slave contentStore replication + docker image
* [search-services-replication](search-services-replication)
Search configurations: sharding with replication + A test for enable.alfresco.tracking=false
* [insight-engine-sharding-replica](insight-engine-sharding-replica)
* [search-services-disable-tracking](search-services-disable-tracking)
Upgrades
* [search-services-upgrade](search-services-upgrade)
Backup
* [search-services-backup](search-services-backup)
JDBC driver with DBVisualizer
* [insight-engine](insight-engine)
Zeppelin
* [insight-engine-zeppelin](insight-engine-zeppelin)
SSL between ACS and Solr
* [mtls](mtls)

15
e2e-test/release-testing/insight-engine-sharding-replica/.env Executable file
View File

@@ -0,0 +1,15 @@
ALFRESCO_TAG=6.2.2
ALFRESCO_CE_TAG=6.2.0-ga
SHARE_TAG=6.2.1
POSTGRES_TAG=11.4
TRANSFORM_ROUTER_TAG=1.2.0
TRANSFORM_CORE_AIO_TAG=2.2.1
TRANSFORMER_TAG=2.1.0
SHARED_FILE_STORE_TAG=0.7.0
ACTIVE_MQ_TAG=5.15.8
DIGITAL_WORKSPACE_TAG=1.5.0
ACS_NGINX_TAG=3.0.1
ACS_COMMUNITY_NGINX_TAG=1.0.0
SEARCH_TAG=1.4.3.0-RC1
ZEPPELIN_TAG=latest
ACA_TAG=master-latest

9
e2e-test/release-testing/insight-engine-sharding-replica/README.md Normal file
View File

@@ -0,0 +1,9 @@
# sharding-replica
Once Docker Compose template is started, type following URLs in the browser:
http://127.0.0.1:8083/solr/admin/cores?action=newCore&storeRef=workspace://SpacesStore&numShards=3&numNodes=3&nodeInstance=1&template=rerank&property.data.dir.root=<>&shardIds=0,1&property.alfresco.port=8080
http://127.0.0.1:8084/solr/admin/cores?action=newCore&storeRef=workspace://SpacesStore&numShards=3&numNodes=3&nodeInstance=2&template=rerank&property.data.dir.root=<>&shardIds=1,2&property.alfresco.port=8080
http://127.0.0.1:8085/solr/admin/cores?action=newCore&storeRef=workspace://SpacesStore&numShards=3&numNodes=3&nodeInstance=3&template=rerank&property.data.dir.root=<>&shardIds=0,2&property.alfresco.port=8080
This will create 3 Shards with 2 core replicas on each one.

73
e2e-test/release-testing/insight-engine-sharding-replica/alfresco/Dockerfile Executable file
View File

@@ -0,0 +1,73 @@
ARG ALFRESCO_TAG
FROM alfresco/alfresco-content-repository:${ALFRESCO_TAG}
ARG TOMCAT_DIR=/usr/local/tomcat
ARG ALF_DATA_DIR=${TOMCAT_DIR}/alf_data
# COMMS
ARG SOLR_COMMS
ENV SOLR_COMMS $SOLR_COMMS
# SSL
ARG TRUSTSTORE_TYPE
ARG TRUSTSTORE_PASS
ARG KEYSTORE_TYPE
ARG KEYSTORE_PASS
ENV TRUSTSTORE_TYPE=$TRUSTSTORE_TYPE \
TRUSTSTORE_PASS=$TRUSTSTORE_PASS \
KEYSTORE_TYPE=$KEYSTORE_TYPE \
KEYSTORE_PASS=$KEYSTORE_PASS
# Expose keystore folder
# Useless for 'none'/'http' communications with SOLR
VOLUME ["${ALF_DATA_DIR}/keystore"]
USER root
# Install modules and addons
RUN mkdir -p $TOMCAT_DIR/amps
COPY modules/amps/* $TOMCAT_DIR/amps/
COPY modules/jars/* $TOMCAT_DIR/webapps/alfresco/WEB-INF/lib/
RUN java -jar $TOMCAT_DIR/alfresco-mmt/alfresco-mmt*.jar install \
$TOMCAT_DIR/amps $TOMCAT_DIR/webapps/alfresco -directory -nobackup -force;
# Default value in "repository.properties" is "dir.keystore=classpath:alfresco/keystore"
RUN if [ "$SOLR_COMMS" == "https" ] ; then \
echo -e "\n\
dir.keystore=${ALF_DATA_DIR}/keystore\n\
alfresco.encryption.ssl.keystore.type=${TRUSTSTORE_TYPE}\n\
alfresco.encryption.ssl.truststore.type=${KEYSTORE_TYPE}\n\
" >> ${TOMCAT_DIR}/shared/classes/alfresco-global.properties; \
fi
# Enable SSL by adding the proper Connector to server.xml
RUN if [ "$SOLR_COMMS" == "https" ] ; then \
sed -i "s/\
[[:space:]]\+<\/Engine>/\n\
<\/Engine>\n\
<Connector port=\"8443\" protocol=\"org.apache.coyote.http11.Http11Protocol\"\n\
connectionTimeout=\"20000\"\n\
SSLEnabled=\"true\" maxThreads=\"150\" scheme=\"https\"\n\
keystoreFile=\"\/usr\/local\/tomcat\/alf_data\/keystore\/ssl.keystore\"\n\
keystorePass=\"${KEYSTORE_PASS}\" keystoreType=\"${KEYSTORE_TYPE}\" secure=\"true\"\n\
truststoreFile=\"\/usr\/local\/tomcat\/alf_data\/keystore\/ssl.truststore\"\n\
truststorePass=\"${TRUSTSTORE_PASS}\" truststoreType=\"${TRUSTSTORE_TYPE}\" clientAuth=\"want\" sslProtocol=\"TLS\">\n\
<\/Connector>/g" ${TOMCAT_DIR}/conf/server.xml; \
fi
# GZIP COMPRESSION
ARG COMPRESS_CONTENT
ENV COMPRESS_CONTENT $COMPRESS_CONTENT
RUN if [ "$COMPRESS_CONTENT" == "true" ] ; then \
sed -i "s/\
[[:space:]]\+connectionTimeout=\"20000\"/\n\
connectionTimeout=\"20000\"\n\
compression=\"on\"\n\
compressionMinSize=\"1\"\n\
/g" ${TOMCAT_DIR}/conf/server.xml; \
fi
# Copy custom content model to deployment folder
COPY model/* $TOMCAT_DIR/shared/classes/alfresco/extension/

15
e2e-test/release-testing/insight-engine-sharding-replica/alfresco/model/sharding-content-model-context.xml Normal file
View File

@@ -0,0 +1,15 @@
<?xml version='1.0' encoding='UTF-8'?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.0.xsd">
<bean id="org.alfresco.sharding-content-model.dictionaryBootstrap" parent="dictionaryModelBootstrap" depends-on="dictionaryBootstrap">
<property name="models">
<list>
<value>alfresco/extension/sharding-content-model.xml</value>
</list>
</property>
</bean>
</beans>

27
e2e-test/release-testing/insight-engine-sharding-replica/alfresco/model/sharding-content-model.xml Normal file
View File

@@ -0,0 +1,27 @@
<?xml version="1.0" encoding="UTF-8"?>
<model name="shard:contentModel" xmlns="http://www.alfresco.org/model/dictionary/1.0">
<description>Explicit Routing for Sharding Sample Model</description>
<imports>
<import uri="http://www.alfresco.org/model/dictionary/1.0" prefix="d"/>
</imports>
<namespaces>
<namespace uri="http://www.alfresco.org/model/sharding/1.0" prefix="shard"/>
</namespaces>
<aspects>
<aspect name="shard:sharding">
<title>Sharding</title>
<properties>
<!-- Shard number to index this content -->
<property name="shard:shardId">
<title>Shard Id</title>
<type>d:text</type>
</property>
</properties>
</aspect>
</aspects>
</model>

0
e2e-test/release-testing/insight-engine-sharding-replica/alfresco/modules/amps/empty Normal file
View File

0
e2e-test/release-testing/insight-engine-sharding-replica/alfresco/modules/jars/empty Normal file
View File

67
e2e-test/release-testing/insight-engine-sharding-replica/config/nginx.conf Normal file
View File

@@ -0,0 +1,67 @@
worker_processes 1;
events {
worker_connections 1024;
}
http {
server {
listen *:8080;
client_max_body_size 0;
set $allowOriginSite *;
proxy_pass_request_headers on;
proxy_pass_header Set-Cookie;
# External settings, do not remove
#ENV_ACCESS_LOG
proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $host:$server_port;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass_header Set-Cookie;
# Alfresco Repository
location /alfresco/ {
proxy_pass http://alfresco:8080;
}
# REST API (Swagger)
location /api-explorer/ {
proxy_pass http://alfresco:8080;
}
# SOLR Web Console (Master)
location /solr/ {
proxy_pass http://solr6:8983;
# Basic authentication
auth_basic "Solr web console";
auth_basic_user_file /etc/nginx/conf.d/nginx.htpasswd;
}
# Alfresco Share Web App
location /share/ {
proxy_pass http://share:8080;
}
# Alfresco Content App
location / {
proxy_pass http://content-app:8080;
}
}
}

1
e2e-test/release-testing/insight-engine-sharding-replica/config/nginx.htpasswd Normal file
View File

@@ -0,0 +1 @@
admin:YWG41BPzVAkN6

214
e2e-test/release-testing/insight-engine-sharding-replica/docker-compose.yml Executable file
View File

@@ -0,0 +1,214 @@
# Using version 2 as 3 does not support resource constraint options (cpu_*, mem_* limits) for non swarm mode in Compose
version: "2"
services:
alfresco:
build:
context: ./alfresco
args:
ALFRESCO_TAG: ${ALFRESCO_TAG}
SOLR_COMMS: none
COMPRESS_CONTENT: "false"
mem_limit: 1800m
environment:
JAVA_OPTS: "
-Ddb.driver=org.postgresql.Driver
-Ddb.username=alfresco
-Ddb.password=alfresco
-Ddb.url=jdbc:postgresql://postgres:5432/alfresco
-Dsolr.host=solr6
-Dsolr.port.ssl=8983
-Dsolr.secureComms=none
-Dsolr.baseUrl=/solr
-Dsolr.useDynamicShardRegistration=true
-Dindex.subsystem.name=solr6
-Daos.baseUrlOverwrite=http://localhost:8080/alfresco/aos
-Dmessaging.broker.url=\"failover:(nio://activemq:61616)?timeout=3000&jms.useCompression=true\"
-Ddeployment.method=DOCKER_COMPOSE
-Dtransform.service.enabled=true
-Dtransform.service.url=http://transform-router:8095
-Dsfs.url=http://shared-file-store:8099/
-DlocalTransform.core-aio.url=http://transform-core-aio:8090/
-Dalfresco-pdf-renderer.url=http://transform-core-aio:8090/
-Djodconverter.url=http://transform-core-aio:8090/
-Dimg.url=http://transform-core-aio:8090/
-Dtika.url=http://transform-core-aio:8090/
-Dtransform.misc.url=http://transform-core-aio:8090/
-Dcsrf.filter.enabled=false
-Dalfresco.restApi.basicAuthScheme=true
-Xms1500m -Xmx1500m
"
solr6:
build:
context: ./search
args:
SEARCH_TAG: ${SEARCH_TAG}
SOLR_HOSTNAME: solr6
ALFRESCO_HOSTNAME: alfresco
ALFRESCO_COMMS: none
COMPRESS_CONTENT: "false"
SEARCH_LOG_LEVEL: WARN
mem_limit: 1200m
environment:
#Solr needs to know how to register itself with Alfresco
SOLR_ALFRESCO_HOST: "alfresco"
SOLR_ALFRESCO_PORT: "8080"
ALFRESCO_SECURE_COMMS: none
#Alfresco needs to know how to call solr
SOLR_SOLR_HOST: "solr6"
SOLR_SOLR_PORT: "8983"
SOLR_JAVA_MEM: "-Xms1g -Xmx1g"
ports:
- 8083:8983
solr6secondary:
build:
context: ./search
args:
SEARCH_TAG: ${SEARCH_TAG}
SOLR_HOSTNAME: solr6secondary
ALFRESCO_HOSTNAME: alfresco
ALFRESCO_COMMS: none
mem_limit: 1200m
environment:
#Solr needs to know how to register itself with Alfresco
SOLR_ALFRESCO_HOST: "alfresco"
SOLR_ALFRESCO_PORT: "8080"
ALFRESCO_SECURE_COMMS: none
#Alfresco needs to know how to call solr
SOLR_SOLR_HOST: "solr6secondary"
SOLR_SOLR_PORT: "8983"
SOLR_JAVA_MEM: "-Xms1g -Xmx1g"
ports:
- 8084:8983
solr6third:
build:
context: ./search
args:
SEARCH_TAG: ${SEARCH_TAG}
SOLR_HOSTNAME: solr6third
ALFRESCO_HOSTNAME: alfresco
ALFRESCO_COMMS: none
mem_limit: 1200m
environment:
#Solr needs to know how to register itself with Alfresco
SOLR_ALFRESCO_HOST: "alfresco"
SOLR_ALFRESCO_PORT: "8080"
ALFRESCO_SECURE_COMMS: none
#Alfresco needs to know how to call solr
SOLR_SOLR_HOST: "solr6third"
SOLR_SOLR_PORT: "8983"
SOLR_JAVA_MEM: "-Xms1g -Xmx1g"
ports:
- 8085:8983
transform-router:
mem_limit: 512m
image: quay.io/alfresco/alfresco-transform-router:${TRANSFORM_ROUTER_TAG}
environment:
JAVA_OPTS: " -Xms256m -Xmx512m"
ACTIVEMQ_URL: "nio://activemq:61616"
CORE_AIO_URL : "http://transform-core-aio:8090"
FILE_STORE_URL: "http://shared-file-store:8099/alfresco/api/-default-/private/sfs/versions/1/file"
ports:
- 8095:8095
links:
- activemq
transform-core-aio:
image: alfresco/alfresco-transform-core-aio:${TRANSFORM_CORE_AIO_TAG}
mem_limit: 1536m
environment:
JAVA_OPTS: " -Xms256m -Xmx1536m"
ACTIVEMQ_URL: "nio://activemq:61616"
FILE_STORE_URL: "http://shared-file-store:8099/alfresco/api/-default-/private/sfs/versions/1/file"
ports:
- 8090:8090
links:
- activemq
shared-file-store:
image: alfresco/alfresco-shared-file-store:${SHARED_FILE_STORE_TAG}
mem_limit: 512m
environment:
JAVA_OPTS: " -Xms256m -Xmx512m"
scheduler.content.age.millis: 86400000
scheduler.cleanup.interval: 86400000
ports:
- 8099:8099
volumes:
- shared-file-store-volume:/tmp/Alfresco/sfs
share:
build:
context: ./share
args:
SHARE_TAG: ${SHARE_TAG}
mem_limit: 1g
environment:
REPO_HOST: "alfresco"
REPO_PORT: "8080"
JAVA_OPTS: "
-Xms500m
-Xmx500m
-Dalfresco.host=localhost
-Dalfresco.port=8080
-Dalfresco.context=alfresco
-Dalfresco.protocol=http
"
postgres:
image: postgres:${POSTGRES_TAG}
mem_limit: 512m
environment:
- POSTGRES_PASSWORD=alfresco
- POSTGRES_USER=alfresco
- POSTGRES_DB=alfresco
command: postgres -c max_connections=300 -c log_min_messages=LOG
ports:
- 5432:5432
activemq:
image: alfresco/alfresco-activemq:${ACTIVE_MQ_TAG}
mem_limit: 1g
ports:
- 8161:8161 # Web Console
- 5672:5672 # AMQP
- 61616:61616 # OpenWire
- 61613:61613 # STOMP
content-app:
image: quay.io/alfresco/alfresco-digital-workspace:${DIGITAL_WORKSPACE_TAG}
mem_limit: 128m
environment:
BASEPATH: ./
# HTTP proxy to provide HTTP Default port access to services
# SOLR API and SOLR Web Console are protected to avoid unauthenticated access
proxy:
image: nginx:stable-alpine
mem_limit: 128m
depends_on:
- alfresco
volumes:
- ./config/nginx.conf:/etc/nginx/nginx.conf
- ./config/nginx.htpasswd:/etc/nginx/conf.d/nginx.htpasswd
ports:
- 8080:8080
links:
- alfresco
- share
- solr6
- content-app
volumes:
shared-file-store-volume:
driver_opts:
type: tmpfs
device: tmpfs

163
e2e-test/release-testing/insight-engine-sharding-replica/search/Dockerfile Executable file
View File

@@ -0,0 +1,163 @@
ARG SEARCH_TAG
FROM quay.io/alfresco/insight-engine:${SEARCH_TAG}
# COMMON
ARG ALFRESCO_HOSTNAME
ARG SOLR_HOSTNAME
ENV ALFRESCO_HOSTNAME $ALFRESCO_HOSTNAME
ENV SOLR_HOSTNAME $SOLR_HOSTNAME
# Configure Alfresco Service Name
RUN sed -i '/^bash.*/i sed -i "'"s/alfresco.host=localhost/alfresco.host=${ALFRESCO_HOSTNAME}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh && \
sed -i '/^bash.*/i sed -i "'"s/solr.host=localhost/solr.host=${SOLR_HOSTNAME}/g"'" ${DIST_DIR}/solrhome/conf/shared.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh
# COMMS
ARG ALFRESCO_COMMS
ENV ALFRESCO_COMMS $ALFRESCO_COMMS
# Configure SOLR cores to run in HTTPs mode from template
RUN if [ "$ALFRESCO_COMMS" == "https" ] ; then \
sed -i '/^bash.*/i sed -i "'"s/alfresco.secureComms=none/alfresco.secureComms=https/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
else \
sed -i '/^bash.*/i sed -i "'"s/alfresco.secureComms=https/alfresco.secureComms=none/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
# SSL
ARG TRUSTSTORE_TYPE
ENV TRUSTSTORE_TYPE $TRUSTSTORE_TYPE
ARG KEYSTORE_TYPE
ENV KEYSTORE_TYPE $KEYSTORE_TYPE
# Passwords using Env Vars
ARG PASSWORDS_AS_ENV_VARS
ENV PASSWORDS_AS_ENV_VARS $PASSWORDS_AS_ENV_VARS
# Set SSL properties
RUN if [ "$ALFRESCO_COMMS" == "https" ] ; then \
if [ "$PASSWORDS_AS_ENV_VARS" == "true" ]; then \
sed -i '/^bash.*/i \
sed -i "'"s/alfresco.encryption.ssl.keystore.location=.*/alfresco.encryption.ssl.keystore.location=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl-repo-client.keystore/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.keystore.passwordFileLocation=.*/alfresco.encryption.ssl.keystore.passwordFileLocation=/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.keystore.type=.*/alfresco.encryption.ssl.keystore.type=${KEYSTORE_TYPE}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.location=.*/alfresco.encryption.ssl.truststore.location=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl-repo-client.truststore/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.passwordFileLocation=.*/alfresco.encryption.ssl.truststore.passwordFileLocation=/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.type=.*/alfresco.encryption.ssl.truststore.type=${TRUSTSTORE_TYPE}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
else \
sed -i '/^bash.*/i \
sed -i "'"s/alfresco.encryption.ssl.keystore.location=.*/alfresco.encryption.ssl.keystore.location=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl.repo.client.keystore/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.keystore.passwordFileLocation=.*/alfresco.encryption.ssl.keystore.passwordFileLocation=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl-keystore-passwords.properties/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.keystore.type=.*/alfresco.encryption.ssl.keystore.type=${KEYSTORE_TYPE}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.location=.*/alfresco.encryption.ssl.truststore.location=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl.repo.client.truststore/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.passwordFileLocation=.*/alfresco.encryption.ssl.truststore.passwordFileLocation=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl-truststore-passwords.properties/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.type=.*/alfresco.encryption.ssl.truststore.type=${TRUSTSTORE_TYPE}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi \
fi
# REPLICATION
ARG ENABLE_MASTER
ARG ENABLE_SLAVE
ARG MASTER_HOST
ENV ENABLE_MASTER $ENABLE_MASTER
ENV ENABLE_SLAVE $ENABLE_SLAVE
ENV MASTER_HOST $MASTER_HOST
# Set Master / Slave configuration for this Node
RUN if [ "$ENABLE_MASTER" == "true" ] ; then \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
sed -i "/^bash.*/i sed -i '/^\\\\\s*<requestHandler name=\"\\\\/replication\".*/a \
<lst name=\"master\">\
<str name=\"replicateAfter\">commit</str>\
<str name=\"replicateAfter\">startup</str>\
<str name=\"confFiles\">schema.xml,stopwords.txt</str>\
</lst>' ${DIST_DIR}/solrhome/templates/rerank/conf/solrconfig.xml\n" ${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
RUN if [ "$ENABLE_SLAVE" == "true" ] ; then \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
sed -i "/^bash.*/i sed -i '/^\\\\\s*<requestHandler name=\"\\\\/replication\".*/a \
<lst name=\"slave\">\
<str name=\"masterUrl\">http://${MASTER_HOST}:8983/solr/alfresco</str>\
<str name=\"pollInterval\">00:00:60</str>\
</lst>' ${DIST_DIR}/solrhome/templates/rerank/conf/solrconfig.xml\n" ${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
# SHARDING
ARG ENABLE_SHARDING
ARG NUM_SHARDS
ARG SHARD_ID
ARG SHARDING_METHOD
ENV ENABLE_SHARDING $ENABLE_SHARDING
ENV NUM_SHARDS $NUM_SHARDS
ENV SHARD_ID $SHARD_ID
ENV SHARDING_METHOD $SHARDING_METHOD
# Set Port Number and Sharding ID for this Shard Service
RUN if [ "$ENABLE_SHARDING" == "true" ] ; then \
sed -i '/^bash.*/i echo "\nsolr.port.ssl=8983\nshard.instance=${SHARD_ID}" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh && \
if [[ ("$SHARDING_METHOD" != "DB_ID_RANGE") && ("$SHARDING_METHOD" != "EXPLICIT_ID_FALLBACK_LRIS") ]]; then \
sed -i '/^bash.*/i echo "\nshard.count=${NUM_SHARDS}" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
sed -i '/^bash.*/i echo "\nalfresco.port=8080\nalfresco.port.ssl=8443\nalfresco.baseUrl=/alfresco" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
# SHARDING METHODS
ARG SHARD_RANGE
ENV SHARD_RANGE $SHARD_RANGE
ARG SHARD_KEY
ENV SHARD_KEY $SHARD_KEY
ARG SHARD_DATE_GROUPING
ENV SHARD_DATE_GROUPING $SHARD_DATE_GROUPING
# Set Shard Method and Shard Key property name
RUN if [ "$ENABLE_SHARDING" == "true" ] ; then \
sed -i '/^bash.*/i sed -i "'"s/shard.method=DB_ID/shard.method=${SHARDING_METHOD}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
if [ "$SHARDING_METHOD" == "DB_ID_RANGE" ]; then \
sed -i '/^bash.*/i echo "\nshard.range=${SHARD_RANGE}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
if [ "$SHARDING_METHOD" == "DATE" ]; then \
sed -i '/^bash.*/i echo "\nshard.key=${SHARD_KEY}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh && \
sed -i '/^bash.*/i echo "\nshard.date.grouping=${SHARD_DATE_GROUPING}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
if [ "$SHARDING_METHOD" == "PROPERTY" ]; then \
sed -i '/^bash.*/i echo "\nshard.key=${SHARD_KEY}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
if [[ ("$SHARDING_METHOD" == "EXPLICIT_ID") || ("$SHARDING_METHOD" == "EXPLICIT_ID_FALLBACK_LRIS") ]]; then \
sed -i '/^bash.*/i echo "\nshard.key=${SHARD_KEY}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
fi
# GZIP COMPRESSION
ARG COMPRESS_CONTENT
ENV COMPRESS_CONTENT $COMPRESS_CONTENT
RUN if [ "$COMPRESS_CONTENT" == "true" ] ; then \
sed -i '/^bash.*/i sed -i "'"s/solr.request.content.compress=false/solr.request.content.compress=true/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
# Useless for 'none'/'http' communications with Alfresco
RUN mkdir ${DIST_DIR}/keystore \
&& chown -R solr:solr ${DIST_DIR}/keystore
# Set the search log level if requested.
ARG SEARCH_LOG_LEVEL
ENV SEARCH_LOG_LEVEL $SEARCH_LOG_LEVEL
RUN if [ "$SEARCH_LOG_LEVEL" ] ; then \
sed -i '/^bash.*/i sed -i "'"s/log4j.rootLogger=WARN, file, CONSOLE/log4j.rootLogger=${SEARCH_LOG_LEVEL}, file, CONSOLE/g"'" ${DIST_DIR}/logs/log4j.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
VOLUME ["${DIST_DIR}/keystore"]

16
e2e-test/release-testing/insight-engine-sharding-replica/share/Dockerfile Normal file
View File

@@ -0,0 +1,16 @@
ARG SHARE_TAG
FROM quay.io/alfresco/alfresco-share:${SHARE_TAG}
ARG TOMCAT_DIR=/usr/local/tomcat
# Install modules
RUN mkdir -p $TOMCAT_DIR/amps
COPY modules/amps/* $TOMCAT_DIR/amps/
COPY modules/jars/* $TOMCAT_DIR/webapps/share/WEB-INF/lib/
RUN java -jar $TOMCAT_DIR/alfresco-mmt/alfresco-mmt*.jar install \
$TOMCAT_DIR/amps $TOMCAT_DIR/webapps/share -directory -nobackup -force;
# Copy custom content forms to deployment folder
COPY model/* $TOMCAT_DIR/shared/classes/alfresco/web-extension/

27
e2e-test/release-testing/insight-engine-sharding-replica/share/model/share-config-custom-dev.xml Normal file
View File

@@ -0,0 +1,27 @@
<alfresco-config>
<!-- Add custom configuration -->
<config evaluator="string-compare" condition="DocumentLibrary">
<aspects>
<visible>
<aspect name="shard:sharding"/>
</visible>
<addable>
</addable>
<removeable>
</removeable>
</aspects>
</config>
<config evaluator="aspect" condition="shard:sharding">
<forms>
<form>
<field-visibility>
<show id="shard:shardId" />
</field-visibility>
</form>
</forms>
</config>
</alfresco-config>

0
e2e-test/release-testing/insight-engine-sharding-replica/share/modules/amps/empty Normal file
View File

0
e2e-test/release-testing/insight-engine-sharding-replica/share/modules/jars/empty Normal file
View File

15
e2e-test/release-testing/insight-engine-sharding/acl_id/.env Executable file
View File

@@ -0,0 +1,15 @@
ALFRESCO_TAG=6.2.2
ALFRESCO_CE_TAG=6.2.0-ga
SHARE_TAG=6.2.1
POSTGRES_TAG=11.4
TRANSFORM_ROUTER_TAG=1.2.0
TRANSFORM_CORE_AIO_TAG=2.2.1
TRANSFORMER_TAG=2.1.0
SHARED_FILE_STORE_TAG=0.7.0
ACTIVE_MQ_TAG=5.15.8
DIGITAL_WORKSPACE_TAG=1.5.0
ACS_NGINX_TAG=3.0.1
ACS_COMMUNITY_NGINX_TAG=1.0.0
SEARCH_TAG=1.4.3.0-RC1
ZEPPELIN_TAG=latest
ACA_TAG=master-latest

73
e2e-test/release-testing/insight-engine-sharding/acl_id/alfresco/Dockerfile Executable file
View File

@@ -0,0 +1,73 @@
ARG ALFRESCO_TAG
FROM alfresco/alfresco-content-repository:${ALFRESCO_TAG}
ARG TOMCAT_DIR=/usr/local/tomcat
ARG ALF_DATA_DIR=${TOMCAT_DIR}/alf_data
# COMMS
ARG SOLR_COMMS
ENV SOLR_COMMS $SOLR_COMMS
# SSL
ARG TRUSTSTORE_TYPE
ARG TRUSTSTORE_PASS
ARG KEYSTORE_TYPE
ARG KEYSTORE_PASS
ENV TRUSTSTORE_TYPE=$TRUSTSTORE_TYPE \
TRUSTSTORE_PASS=$TRUSTSTORE_PASS \
KEYSTORE_TYPE=$KEYSTORE_TYPE \
KEYSTORE_PASS=$KEYSTORE_PASS
# Expose keystore folder
# Useless for 'none'/'http' communications with SOLR
VOLUME ["${ALF_DATA_DIR}/keystore"]
USER root
# Install modules and addons
RUN mkdir -p $TOMCAT_DIR/amps
COPY modules/amps/* $TOMCAT_DIR/amps/
COPY modules/jars/* $TOMCAT_DIR/webapps/alfresco/WEB-INF/lib/
RUN java -jar $TOMCAT_DIR/alfresco-mmt/alfresco-mmt*.jar install \
$TOMCAT_DIR/amps $TOMCAT_DIR/webapps/alfresco -directory -nobackup -force;
# Default value in "repository.properties" is "dir.keystore=classpath:alfresco/keystore"
RUN if [ "$SOLR_COMMS" == "https" ] ; then \
echo -e "\n\
dir.keystore=${ALF_DATA_DIR}/keystore\n\
alfresco.encryption.ssl.keystore.type=${TRUSTSTORE_TYPE}\n\
alfresco.encryption.ssl.truststore.type=${KEYSTORE_TYPE}\n\
" >> ${TOMCAT_DIR}/shared/classes/alfresco-global.properties; \
fi
# Enable SSL by adding the proper Connector to server.xml
RUN if [ "$SOLR_COMMS" == "https" ] ; then \
sed -i "s/\
[[:space:]]\+<\/Engine>/\n\
<\/Engine>\n\
<Connector port=\"8443\" protocol=\"org.apache.coyote.http11.Http11Protocol\"\n\
connectionTimeout=\"20000\"\n\
SSLEnabled=\"true\" maxThreads=\"150\" scheme=\"https\"\n\
keystoreFile=\"\/usr\/local\/tomcat\/alf_data\/keystore\/ssl.keystore\"\n\
keystorePass=\"${KEYSTORE_PASS}\" keystoreType=\"${KEYSTORE_TYPE}\" secure=\"true\"\n\
truststoreFile=\"\/usr\/local\/tomcat\/alf_data\/keystore\/ssl.truststore\"\n\
truststorePass=\"${TRUSTSTORE_PASS}\" truststoreType=\"${TRUSTSTORE_TYPE}\" clientAuth=\"want\" sslProtocol=\"TLS\">\n\
<\/Connector>/g" ${TOMCAT_DIR}/conf/server.xml; \
fi
# GZIP COMPRESSION
ARG COMPRESS_CONTENT
ENV COMPRESS_CONTENT $COMPRESS_CONTENT
RUN if [ "$COMPRESS_CONTENT" == "true" ] ; then \
sed -i "s/\
[[:space:]]\+connectionTimeout=\"20000\"/\n\
connectionTimeout=\"20000\"\n\
compression=\"on\"\n\
compressionMinSize=\"1\"\n\
/g" ${TOMCAT_DIR}/conf/server.xml; \
fi
# Copy custom content model to deployment folder
COPY model/* $TOMCAT_DIR/shared/classes/alfresco/extension/

15
e2e-test/release-testing/insight-engine-sharding/acl_id/alfresco/model/sharding-content-model-context.xml Normal file
View File

@@ -0,0 +1,15 @@
<?xml version='1.0' encoding='UTF-8'?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.0.xsd">
<bean id="org.alfresco.sharding-content-model.dictionaryBootstrap" parent="dictionaryModelBootstrap" depends-on="dictionaryBootstrap">
<property name="models">
<list>
<value>alfresco/extension/sharding-content-model.xml</value>
</list>
</property>
</bean>
</beans>

27
e2e-test/release-testing/insight-engine-sharding/acl_id/alfresco/model/sharding-content-model.xml Normal file
View File

@@ -0,0 +1,27 @@
<?xml version="1.0" encoding="UTF-8"?>
<model name="shard:contentModel" xmlns="http://www.alfresco.org/model/dictionary/1.0">
<description>Explicit Routing for Sharding Sample Model</description>
<imports>
<import uri="http://www.alfresco.org/model/dictionary/1.0" prefix="d"/>
</imports>
<namespaces>
<namespace uri="http://www.alfresco.org/model/sharding/1.0" prefix="shard"/>
</namespaces>
<aspects>
<aspect name="shard:sharding">
<title>Sharding</title>
<properties>
<!-- Shard number to index this content -->
<property name="shard:shardId">
<title>Shard Id</title>
<type>d:text</type>
</property>
</properties>
</aspect>
</aspects>
</model>

0
e2e-test/release-testing/insight-engine-sharding/acl_id/alfresco/modules/amps/empty Normal file
View File

0
e2e-test/release-testing/insight-engine-sharding/acl_id/alfresco/modules/jars/empty Normal file
View File

67
e2e-test/release-testing/insight-engine-sharding/acl_id/config/nginx.conf Normal file
View File

@@ -0,0 +1,67 @@
worker_processes 1;
events {
worker_connections 1024;
}
http {
server {
listen *:8080;
client_max_body_size 0;
set $allowOriginSite *;
proxy_pass_request_headers on;
proxy_pass_header Set-Cookie;
# External settings, do not remove
#ENV_ACCESS_LOG
proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $host:$server_port;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass_header Set-Cookie;
# Alfresco Repository
location /alfresco/ {
proxy_pass http://alfresco:8080;
}
# REST API (Swagger)
location /api-explorer/ {
proxy_pass http://alfresco:8080;
}
# SOLR Web Console (Master)
location /solr/ {
proxy_pass http://solr6:8983;
# Basic authentication
auth_basic "Solr web console";
auth_basic_user_file /etc/nginx/conf.d/nginx.htpasswd;
}
# Alfresco Share Web App
location /share/ {
proxy_pass http://share:8080;
}
# Alfresco Content App
location / {
proxy_pass http://content-app:8080;
}
}
}

1
e2e-test/release-testing/insight-engine-sharding/acl_id/config/nginx.htpasswd Normal file
View File

@@ -0,0 +1 @@
admin:YWG41BPzVAkN6

209
e2e-test/release-testing/insight-engine-sharding/acl_id/docker-compose.yml Executable file
View File

@@ -0,0 +1,209 @@
# Using version 2 as 3 does not support resource constraint options (cpu_*, mem_* limits) for non swarm mode in Compose
version: "2"
services:
alfresco:
build:
context: ./alfresco
args:
ALFRESCO_TAG: ${ALFRESCO_TAG}
SOLR_COMMS: none
COMPRESS_CONTENT: "false"
mem_limit: 1800m
environment:
JAVA_OPTS: "
-Ddb.driver=org.postgresql.Driver
-Ddb.username=alfresco
-Ddb.password=alfresco
-Ddb.url=jdbc:postgresql://postgres:5432/alfresco
-Dsolr.host=solr6
-Dsolr.port.ssl=8983
-Dsolr.secureComms=none
-Dsolr.baseUrl=/solr
-Dsolr.useDynamicShardRegistration=true
-Dindex.subsystem.name=solr6
-Daos.baseUrlOverwrite=http://localhost:8080/alfresco/aos
-Dmessaging.broker.url=\"failover:(nio://activemq:61616)?timeout=3000&jms.useCompression=true\"
-Ddeployment.method=DOCKER_COMPOSE
-Dtransform.service.enabled=true
-Dtransform.service.url=http://transform-router:8095
-Dsfs.url=http://shared-file-store:8099/
-DlocalTransform.core-aio.url=http://transform-core-aio:8090/
-Dalfresco-pdf-renderer.url=http://transform-core-aio:8090/
-Djodconverter.url=http://transform-core-aio:8090/
-Dimg.url=http://transform-core-aio:8090/
-Dtika.url=http://transform-core-aio:8090/
-Dtransform.misc.url=http://transform-core-aio:8090/
-Dcsrf.filter.enabled=false
-Dalfresco.restApi.basicAuthScheme=true
-Xms1500m -Xmx1500m
"
solr6:
build:
context: ./search
args:
SEARCH_TAG: ${SEARCH_TAG}
SOLR_HOSTNAME: solr6
ALFRESCO_HOSTNAME: alfresco
ALFRESCO_COMMS: none
ENABLE_SHARDING: "true"
NUM_SHARDS: "2"
SHARD_ID: "0"
SHARDING_METHOD: ACL_ID
COMPRESS_CONTENT: "false"
SEARCH_LOG_LEVEL: WARN
mem_limit: 1200m
environment:
#Solr needs to know how to register itself with Alfresco
SOLR_ALFRESCO_HOST: "alfresco"
SOLR_ALFRESCO_PORT: "8080"
ALFRESCO_SECURE_COMMS: none
#Alfresco needs to know how to call solr
SOLR_SOLR_HOST: "solr6"
SOLR_SOLR_PORT: "8983"
#Create the default alfresco and archive cores
SOLR_CREATE_ALFRESCO_DEFAULTS: "alfresco,archive"
SOLR_JAVA_MEM: "-Xms1g -Xmx1g"
ports:
- 8083:8983
solr6secondary:
build:
context: ./search
args:
SEARCH_TAG: ${SEARCH_TAG}
SOLR_HOSTNAME: solr6secondary
ALFRESCO_HOSTNAME: alfresco
ALFRESCO_COMMS: none
ENABLE_SHARDING: "true"
NUM_SHARDS: "2"
SHARD_ID: "1"
SHARDING_METHOD: ACL_ID
COMPRESS_CONTENT: "false"
mem_limit: 1200m
environment:
#Solr needs to know how to register itself with Alfresco
SOLR_ALFRESCO_HOST: "alfresco"
SOLR_ALFRESCO_PORT: "8080"
ALFRESCO_SECURE_COMMS: none
#Alfresco needs to know how to call solr
SOLR_SOLR_HOST: "solr6secondary"
SOLR_SOLR_PORT: "8983"
#Create the default alfresco and archive cores
SOLR_CREATE_ALFRESCO_DEFAULTS: "alfresco,archive"
SOLR_JAVA_MEM: "-Xms1g -Xmx1g"
ports:
- 8084:8983
transform-router:
mem_limit: 512m
image: quay.io/alfresco/alfresco-transform-router:${TRANSFORM_ROUTER_TAG}
environment:
JAVA_OPTS: " -Xms256m -Xmx512m"
ACTIVEMQ_URL: "nio://activemq:61616"
CORE_AIO_URL : "http://transform-core-aio:8090"
FILE_STORE_URL: "http://shared-file-store:8099/alfresco/api/-default-/private/sfs/versions/1/file"
ports:
- 8095:8095
links:
- activemq
transform-core-aio:
image: alfresco/alfresco-transform-core-aio:${TRANSFORM_CORE_AIO_TAG}
mem_limit: 1536m
environment:
JAVA_OPTS: " -Xms256m -Xmx1536m"
ACTIVEMQ_URL: "nio://activemq:61616"
FILE_STORE_URL: "http://shared-file-store:8099/alfresco/api/-default-/private/sfs/versions/1/file"
ports:
- 8090:8090
links:
- activemq
shared-file-store:
image: alfresco/alfresco-shared-file-store:${SHARED_FILE_STORE_TAG}
mem_limit: 512m
environment:
JAVA_OPTS: " -Xms256m -Xmx512m"
scheduler.content.age.millis: 86400000
scheduler.cleanup.interval: 86400000
ports:
- 8099:8099
volumes:
- shared-file-store-volume:/tmp/Alfresco/sfs
share:
build:
context: ./share
args:
SHARE_TAG: ${SHARE_TAG}
mem_limit: 1g
environment:
REPO_HOST: "alfresco"
REPO_PORT: "8080"
JAVA_OPTS: "
-Xms500m
-Xmx500m
-Dalfresco.host=localhost
-Dalfresco.port=8080
-Dalfresco.context=alfresco
-Dalfresco.protocol=http
"
postgres:
image: postgres:${POSTGRES_TAG}
mem_limit: 512m
environment:
- POSTGRES_PASSWORD=alfresco
- POSTGRES_USER=alfresco
- POSTGRES_DB=alfresco
command: postgres -c max_connections=300 -c log_min_messages=LOG
ports:
- 5432:5432
activemq:
image: alfresco/alfresco-activemq:${ACTIVE_MQ_TAG}
mem_limit: 1g
ports:
- 8161:8161 # Web Console
- 5672:5672 # AMQP
- 61616:61616 # OpenWire
- 61613:61613 # STOMP
content-app:
image: quay.io/alfresco/alfresco-digital-workspace:${DIGITAL_WORKSPACE_TAG}
mem_limit: 128m
environment:
BASEPATH: ./
# HTTP proxy to provide HTTP Default port access to services
# SOLR API and SOLR Web Console are protected to avoid unauthenticated access
proxy:
image: nginx:stable-alpine
mem_limit: 128m
depends_on:
- alfresco
volumes:
- ./config/nginx.conf:/etc/nginx/nginx.conf
- ./config/nginx.htpasswd:/etc/nginx/conf.d/nginx.htpasswd
ports:
- 8080:8080
links:
- alfresco
- share
- solr6
- content-app
volumes:
shared-file-store-volume:
driver_opts:
type: tmpfs
device: tmpfs

163
e2e-test/release-testing/insight-engine-sharding/acl_id/search/Dockerfile Executable file
View File

@@ -0,0 +1,163 @@
ARG SEARCH_TAG
FROM quay.io/alfresco/insight-engine:${SEARCH_TAG}
# COMMON
ARG ALFRESCO_HOSTNAME
ARG SOLR_HOSTNAME
ENV ALFRESCO_HOSTNAME $ALFRESCO_HOSTNAME
ENV SOLR_HOSTNAME $SOLR_HOSTNAME
# Configure Alfresco Service Name
RUN sed -i '/^bash.*/i sed -i "'"s/alfresco.host=localhost/alfresco.host=${ALFRESCO_HOSTNAME}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh && \
sed -i '/^bash.*/i sed -i "'"s/solr.host=localhost/solr.host=${SOLR_HOSTNAME}/g"'" ${DIST_DIR}/solrhome/conf/shared.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh
# COMMS
ARG ALFRESCO_COMMS
ENV ALFRESCO_COMMS $ALFRESCO_COMMS
# Configure SOLR cores to run in HTTPs mode from template
RUN if [ "$ALFRESCO_COMMS" == "https" ] ; then \
sed -i '/^bash.*/i sed -i "'"s/alfresco.secureComms=none/alfresco.secureComms=https/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
else \
sed -i '/^bash.*/i sed -i "'"s/alfresco.secureComms=https/alfresco.secureComms=none/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
# SSL
ARG TRUSTSTORE_TYPE
ENV TRUSTSTORE_TYPE $TRUSTSTORE_TYPE
ARG KEYSTORE_TYPE
ENV KEYSTORE_TYPE $KEYSTORE_TYPE
# Passwords using Env Vars
ARG PASSWORDS_AS_ENV_VARS
ENV PASSWORDS_AS_ENV_VARS $PASSWORDS_AS_ENV_VARS
# Set SSL properties
RUN if [ "$ALFRESCO_COMMS" == "https" ] ; then \
if [ "$PASSWORDS_AS_ENV_VARS" == "true" ]; then \
sed -i '/^bash.*/i \
sed -i "'"s/alfresco.encryption.ssl.keystore.location=.*/alfresco.encryption.ssl.keystore.location=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl-repo-client.keystore/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.keystore.passwordFileLocation=.*/alfresco.encryption.ssl.keystore.passwordFileLocation=/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.keystore.type=.*/alfresco.encryption.ssl.keystore.type=${KEYSTORE_TYPE}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.location=.*/alfresco.encryption.ssl.truststore.location=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl-repo-client.truststore/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.passwordFileLocation=.*/alfresco.encryption.ssl.truststore.passwordFileLocation=/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.type=.*/alfresco.encryption.ssl.truststore.type=${TRUSTSTORE_TYPE}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
else \
sed -i '/^bash.*/i \
sed -i "'"s/alfresco.encryption.ssl.keystore.location=.*/alfresco.encryption.ssl.keystore.location=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl.repo.client.keystore/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.keystore.passwordFileLocation=.*/alfresco.encryption.ssl.keystore.passwordFileLocation=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl-keystore-passwords.properties/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.keystore.type=.*/alfresco.encryption.ssl.keystore.type=${KEYSTORE_TYPE}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.location=.*/alfresco.encryption.ssl.truststore.location=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl.repo.client.truststore/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.passwordFileLocation=.*/alfresco.encryption.ssl.truststore.passwordFileLocation=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl-truststore-passwords.properties/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.type=.*/alfresco.encryption.ssl.truststore.type=${TRUSTSTORE_TYPE}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi \
fi
# REPLICATION
ARG ENABLE_MASTER
ARG ENABLE_SLAVE
ARG MASTER_HOST
ENV ENABLE_MASTER $ENABLE_MASTER
ENV ENABLE_SLAVE $ENABLE_SLAVE
ENV MASTER_HOST $MASTER_HOST
# Set Master / Slave configuration for this Node
RUN if [ "$ENABLE_MASTER" == "true" ] ; then \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
sed -i "/^bash.*/i sed -i '/^\\\\\s*<requestHandler name=\"\\\\/replication\".*/a \
<lst name=\"master\">\
<str name=\"replicateAfter\">commit</str>\
<str name=\"replicateAfter\">startup</str>\
<str name=\"confFiles\">schema.xml,stopwords.txt</str>\
</lst>' ${DIST_DIR}/solrhome/templates/rerank/conf/solrconfig.xml\n" ${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
RUN if [ "$ENABLE_SLAVE" == "true" ] ; then \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
sed -i "/^bash.*/i sed -i '/^\\\\\s*<requestHandler name=\"\\\\/replication\".*/a \
<lst name=\"slave\">\
<str name=\"masterUrl\">http://${MASTER_HOST}:8983/solr/alfresco</str>\
<str name=\"pollInterval\">00:00:60</str>\
</lst>' ${DIST_DIR}/solrhome/templates/rerank/conf/solrconfig.xml\n" ${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
# SHARDING
ARG ENABLE_SHARDING
ARG NUM_SHARDS
ARG SHARD_ID
ARG SHARDING_METHOD
ENV ENABLE_SHARDING $ENABLE_SHARDING
ENV NUM_SHARDS $NUM_SHARDS
ENV SHARD_ID $SHARD_ID
ENV SHARDING_METHOD $SHARDING_METHOD
# Set Port Number and Sharding ID for this Shard Service
RUN if [ "$ENABLE_SHARDING" == "true" ] ; then \
sed -i '/^bash.*/i echo "\nsolr.port.ssl=8983\nshard.instance=${SHARD_ID}" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh && \
if [[ ("$SHARDING_METHOD" != "DB_ID_RANGE") && ("$SHARDING_METHOD" != "EXPLICIT_ID_FALLBACK_LRIS") ]]; then \
sed -i '/^bash.*/i echo "\nshard.count=${NUM_SHARDS}" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
sed -i '/^bash.*/i echo "\nalfresco.port=8080\nalfresco.port.ssl=8443\nalfresco.baseUrl=/alfresco" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
# SHARDING METHODS
ARG SHARD_RANGE
ENV SHARD_RANGE $SHARD_RANGE
ARG SHARD_KEY
ENV SHARD_KEY $SHARD_KEY
ARG SHARD_DATE_GROUPING
ENV SHARD_DATE_GROUPING $SHARD_DATE_GROUPING
# Set Shard Method and Shard Key property name
RUN if [ "$ENABLE_SHARDING" == "true" ] ; then \
sed -i '/^bash.*/i sed -i "'"s/shard.method=DB_ID/shard.method=${SHARDING_METHOD}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
if [ "$SHARDING_METHOD" == "DB_ID_RANGE" ]; then \
sed -i '/^bash.*/i echo "\nshard.range=${SHARD_RANGE}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
if [ "$SHARDING_METHOD" == "DATE" ]; then \
sed -i '/^bash.*/i echo "\nshard.key=${SHARD_KEY}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh && \
sed -i '/^bash.*/i echo "\nshard.date.grouping=${SHARD_DATE_GROUPING}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
if [ "$SHARDING_METHOD" == "PROPERTY" ]; then \
sed -i '/^bash.*/i echo "\nshard.key=${SHARD_KEY}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
if [[ ("$SHARDING_METHOD" == "EXPLICIT_ID") || ("$SHARDING_METHOD" == "EXPLICIT_ID_FALLBACK_LRIS") ]]; then \
sed -i '/^bash.*/i echo "\nshard.key=${SHARD_KEY}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
fi
# GZIP COMPRESSION
ARG COMPRESS_CONTENT
ENV COMPRESS_CONTENT $COMPRESS_CONTENT
RUN if [ "$COMPRESS_CONTENT" == "true" ] ; then \
sed -i '/^bash.*/i sed -i "'"s/solr.request.content.compress=false/solr.request.content.compress=true/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
# Useless for 'none'/'http' communications with Alfresco
RUN mkdir ${DIST_DIR}/keystore \
&& chown -R solr:solr ${DIST_DIR}/keystore
# Set the search log level if requested.
ARG SEARCH_LOG_LEVEL
ENV SEARCH_LOG_LEVEL $SEARCH_LOG_LEVEL
RUN if [ "$SEARCH_LOG_LEVEL" ] ; then \
sed -i '/^bash.*/i sed -i "'"s/log4j.rootLogger=WARN, file, CONSOLE/log4j.rootLogger=${SEARCH_LOG_LEVEL}, file, CONSOLE/g"'" ${DIST_DIR}/logs/log4j.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
VOLUME ["${DIST_DIR}/keystore"]

16
e2e-test/release-testing/insight-engine-sharding/acl_id/share/Dockerfile Normal file
View File

@@ -0,0 +1,16 @@
ARG SHARE_TAG
FROM quay.io/alfresco/alfresco-share:${SHARE_TAG}
ARG TOMCAT_DIR=/usr/local/tomcat
# Install modules
RUN mkdir -p $TOMCAT_DIR/amps
COPY modules/amps/* $TOMCAT_DIR/amps/
COPY modules/jars/* $TOMCAT_DIR/webapps/share/WEB-INF/lib/
RUN java -jar $TOMCAT_DIR/alfresco-mmt/alfresco-mmt*.jar install \
$TOMCAT_DIR/amps $TOMCAT_DIR/webapps/share -directory -nobackup -force;
# Copy custom content forms to deployment folder
COPY model/* $TOMCAT_DIR/shared/classes/alfresco/web-extension/

27
e2e-test/release-testing/insight-engine-sharding/acl_id/share/model/share-config-custom-dev.xml Normal file
View File

@@ -0,0 +1,27 @@
<alfresco-config>
<!-- Add custom configuration -->
<config evaluator="string-compare" condition="DocumentLibrary">
<aspects>
<visible>
<aspect name="shard:sharding"/>
</visible>
<addable>
</addable>
<removeable>
</removeable>
</aspects>
</config>
<config evaluator="aspect" condition="shard:sharding">
<forms>
<form>
<field-visibility>
<show id="shard:shardId" />
</field-visibility>
</form>
</forms>
</config>
</alfresco-config>

0
e2e-test/release-testing/insight-engine-sharding/acl_id/share/modules/amps/empty Normal file
View File

0
e2e-test/release-testing/insight-engine-sharding/acl_id/share/modules/jars/empty Normal file
View File

15
e2e-test/release-testing/insight-engine-sharding/date/.env Executable file
View File

@@ -0,0 +1,15 @@
ALFRESCO_TAG=6.2.2
ALFRESCO_CE_TAG=6.2.0-ga
SHARE_TAG=6.2.1
POSTGRES_TAG=11.4
TRANSFORM_ROUTER_TAG=1.2.0
TRANSFORM_CORE_AIO_TAG=2.2.1
TRANSFORMER_TAG=2.1.0
SHARED_FILE_STORE_TAG=0.7.0
ACTIVE_MQ_TAG=5.15.8
DIGITAL_WORKSPACE_TAG=1.5.0
ACS_NGINX_TAG=3.0.1
ACS_COMMUNITY_NGINX_TAG=1.0.0
SEARCH_TAG=1.4.3.0-RC1
ZEPPELIN_TAG=latest
ACA_TAG=master-latest

73
e2e-test/release-testing/insight-engine-sharding/date/alfresco/Dockerfile Executable file
View File

@@ -0,0 +1,73 @@
ARG ALFRESCO_TAG
FROM alfresco/alfresco-content-repository:${ALFRESCO_TAG}
ARG TOMCAT_DIR=/usr/local/tomcat
ARG ALF_DATA_DIR=${TOMCAT_DIR}/alf_data
# COMMS
ARG SOLR_COMMS
ENV SOLR_COMMS $SOLR_COMMS
# SSL
ARG TRUSTSTORE_TYPE
ARG TRUSTSTORE_PASS
ARG KEYSTORE_TYPE
ARG KEYSTORE_PASS
ENV TRUSTSTORE_TYPE=$TRUSTSTORE_TYPE \
TRUSTSTORE_PASS=$TRUSTSTORE_PASS \
KEYSTORE_TYPE=$KEYSTORE_TYPE \
KEYSTORE_PASS=$KEYSTORE_PASS
# Expose keystore folder
# Useless for 'none'/'http' communications with SOLR
VOLUME ["${ALF_DATA_DIR}/keystore"]
USER root
# Install modules and addons
RUN mkdir -p $TOMCAT_DIR/amps
COPY modules/amps/* $TOMCAT_DIR/amps/
COPY modules/jars/* $TOMCAT_DIR/webapps/alfresco/WEB-INF/lib/
RUN java -jar $TOMCAT_DIR/alfresco-mmt/alfresco-mmt*.jar install \
$TOMCAT_DIR/amps $TOMCAT_DIR/webapps/alfresco -directory -nobackup -force;
# Default value in "repository.properties" is "dir.keystore=classpath:alfresco/keystore"
RUN if [ "$SOLR_COMMS" == "https" ] ; then \
echo -e "\n\
dir.keystore=${ALF_DATA_DIR}/keystore\n\
alfresco.encryption.ssl.keystore.type=${TRUSTSTORE_TYPE}\n\
alfresco.encryption.ssl.truststore.type=${KEYSTORE_TYPE}\n\
" >> ${TOMCAT_DIR}/shared/classes/alfresco-global.properties; \
fi
# Enable SSL by adding the proper Connector to server.xml
RUN if [ "$SOLR_COMMS" == "https" ] ; then \
sed -i "s/\
[[:space:]]\+<\/Engine>/\n\
<\/Engine>\n\
<Connector port=\"8443\" protocol=\"org.apache.coyote.http11.Http11Protocol\"\n\
connectionTimeout=\"20000\"\n\
SSLEnabled=\"true\" maxThreads=\"150\" scheme=\"https\"\n\
keystoreFile=\"\/usr\/local\/tomcat\/alf_data\/keystore\/ssl.keystore\"\n\
keystorePass=\"${KEYSTORE_PASS}\" keystoreType=\"${KEYSTORE_TYPE}\" secure=\"true\"\n\
truststoreFile=\"\/usr\/local\/tomcat\/alf_data\/keystore\/ssl.truststore\"\n\
truststorePass=\"${TRUSTSTORE_PASS}\" truststoreType=\"${TRUSTSTORE_TYPE}\" clientAuth=\"want\" sslProtocol=\"TLS\">\n\
<\/Connector>/g" ${TOMCAT_DIR}/conf/server.xml; \
fi
# GZIP COMPRESSION
ARG COMPRESS_CONTENT
ENV COMPRESS_CONTENT $COMPRESS_CONTENT
RUN if [ "$COMPRESS_CONTENT" == "true" ] ; then \
sed -i "s/\
[[:space:]]\+connectionTimeout=\"20000\"/\n\
connectionTimeout=\"20000\"\n\
compression=\"on\"\n\
compressionMinSize=\"1\"\n\
/g" ${TOMCAT_DIR}/conf/server.xml; \
fi
# Copy custom content model to deployment folder
COPY model/* $TOMCAT_DIR/shared/classes/alfresco/extension/

15
e2e-test/release-testing/insight-engine-sharding/date/alfresco/model/sharding-content-model-context.xml Normal file
View File

@@ -0,0 +1,15 @@
<?xml version='1.0' encoding='UTF-8'?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.0.xsd">
<bean id="org.alfresco.sharding-content-model.dictionaryBootstrap" parent="dictionaryModelBootstrap" depends-on="dictionaryBootstrap">
<property name="models">
<list>
<value>alfresco/extension/sharding-content-model.xml</value>
</list>
</property>
</bean>
</beans>

27
e2e-test/release-testing/insight-engine-sharding/date/alfresco/model/sharding-content-model.xml Normal file
View File

@@ -0,0 +1,27 @@
<?xml version="1.0" encoding="UTF-8"?>
<model name="shard:contentModel" xmlns="http://www.alfresco.org/model/dictionary/1.0">
<description>Explicit Routing for Sharding Sample Model</description>
<imports>
<import uri="http://www.alfresco.org/model/dictionary/1.0" prefix="d"/>
</imports>
<namespaces>
<namespace uri="http://www.alfresco.org/model/sharding/1.0" prefix="shard"/>
</namespaces>
<aspects>
<aspect name="shard:sharding">
<title>Sharding</title>
<properties>
<!-- Shard number to index this content -->
<property name="shard:shardId">
<title>Shard Id</title>
<type>d:text</type>
</property>
</properties>
</aspect>
</aspects>
</model>

0
e2e-test/release-testing/insight-engine-sharding/date/alfresco/modules/amps/empty Normal file
View File

0
e2e-test/release-testing/insight-engine-sharding/date/alfresco/modules/jars/empty Normal file
View File

67
e2e-test/release-testing/insight-engine-sharding/date/config/nginx.conf Normal file
View File

@@ -0,0 +1,67 @@
worker_processes 1;
events {
worker_connections 1024;
}
http {
server {
listen *:8080;
client_max_body_size 0;
set $allowOriginSite *;
proxy_pass_request_headers on;
proxy_pass_header Set-Cookie;
# External settings, do not remove
#ENV_ACCESS_LOG
proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $host:$server_port;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass_header Set-Cookie;
# Alfresco Repository
location /alfresco/ {
proxy_pass http://alfresco:8080;
}
# REST API (Swagger)
location /api-explorer/ {
proxy_pass http://alfresco:8080;
}
# SOLR Web Console (Master)
location /solr/ {
proxy_pass http://solr6:8983;
# Basic authentication
auth_basic "Solr web console";
auth_basic_user_file /etc/nginx/conf.d/nginx.htpasswd;
}
# Alfresco Share Web App
location /share/ {
proxy_pass http://share:8080;
}
# Alfresco Content App
location / {
proxy_pass http://content-app:8080;
}
}
}

1
e2e-test/release-testing/insight-engine-sharding/date/config/nginx.htpasswd Normal file
View File

@@ -0,0 +1 @@
admin:YWG41BPzVAkN6

213
e2e-test/release-testing/insight-engine-sharding/date/docker-compose.yml Executable file
View File

@@ -0,0 +1,213 @@
# Using version 2 as 3 does not support resource constraint options (cpu_*, mem_* limits) for non swarm mode in Compose
version: "2"
services:
alfresco:
build:
context: ./alfresco
args:
ALFRESCO_TAG: ${ALFRESCO_TAG}
SOLR_COMMS: none
COMPRESS_CONTENT: "false"
mem_limit: 1800m
environment:
JAVA_OPTS: "
-Ddb.driver=org.postgresql.Driver
-Ddb.username=alfresco
-Ddb.password=alfresco
-Ddb.url=jdbc:postgresql://postgres:5432/alfresco
-Dsolr.host=solr6
-Dsolr.port.ssl=8983
-Dsolr.secureComms=none
-Dsolr.baseUrl=/solr
-Dsolr.useDynamicShardRegistration=true
-Dindex.subsystem.name=solr6
-Daos.baseUrlOverwrite=http://localhost:8080/alfresco/aos
-Dmessaging.broker.url=\"failover:(nio://activemq:61616)?timeout=3000&jms.useCompression=true\"
-Ddeployment.method=DOCKER_COMPOSE
-Dtransform.service.enabled=true
-Dtransform.service.url=http://transform-router:8095
-Dsfs.url=http://shared-file-store:8099/
-DlocalTransform.core-aio.url=http://transform-core-aio:8090/
-Dalfresco-pdf-renderer.url=http://transform-core-aio:8090/
-Djodconverter.url=http://transform-core-aio:8090/
-Dimg.url=http://transform-core-aio:8090/
-Dtika.url=http://transform-core-aio:8090/
-Dtransform.misc.url=http://transform-core-aio:8090/
-Dcsrf.filter.enabled=false
-Dalfresco.restApi.basicAuthScheme=true
-Xms1500m -Xmx1500m
"
solr6:
build:
context: ./search
args:
SEARCH_TAG: ${SEARCH_TAG}
SOLR_HOSTNAME: solr6
ALFRESCO_HOSTNAME: alfresco
ALFRESCO_COMMS: none
ENABLE_SHARDING: "true"
NUM_SHARDS: "2"
SHARD_ID: "0"
SHARD_KEY: "cm:created"
SHARD_DATE_GROUPING: "2"
SHARDING_METHOD: DATE
COMPRESS_CONTENT: "false"
SEARCH_LOG_LEVEL: WARN
mem_limit: 1200m
environment:
#Solr needs to know how to register itself with Alfresco
SOLR_ALFRESCO_HOST: "alfresco"
SOLR_ALFRESCO_PORT: "8080"
ALFRESCO_SECURE_COMMS: none
#Alfresco needs to know how to call solr
SOLR_SOLR_HOST: "solr6"
SOLR_SOLR_PORT: "8983"
#Create the default alfresco and archive cores
SOLR_CREATE_ALFRESCO_DEFAULTS: "alfresco,archive"
SOLR_JAVA_MEM: "-Xms1g -Xmx1g"
ports:
- 8083:8983
solr6secondary:
build:
context: ./search
args:
SEARCH_TAG: ${SEARCH_TAG}
SOLR_HOSTNAME: solr6secondary
ALFRESCO_HOSTNAME: alfresco
ALFRESCO_COMMS: none
ENABLE_SHARDING: "true"
NUM_SHARDS: "2"
SHARD_ID: "1"
SHARD_KEY: "cm:created"
SHARD_DATE_GROUPING: "2"
SHARDING_METHOD: DATE
COMPRESS_CONTENT: "false"
mem_limit: 1200m
environment:
#Solr needs to know how to register itself with Alfresco
SOLR_ALFRESCO_HOST: "alfresco"
SOLR_ALFRESCO_PORT: "8080"
ALFRESCO_SECURE_COMMS: none
#Alfresco needs to know how to call solr
SOLR_SOLR_HOST: "solr6secondary"
SOLR_SOLR_PORT: "8983"
#Create the default alfresco and archive cores
SOLR_CREATE_ALFRESCO_DEFAULTS: "alfresco,archive"
SOLR_JAVA_MEM: "-Xms1g -Xmx1g"
ports:
- 8084:8983
transform-router:
mem_limit: 512m
image: quay.io/alfresco/alfresco-transform-router:${TRANSFORM_ROUTER_TAG}
environment:
JAVA_OPTS: " -Xms256m -Xmx512m"
ACTIVEMQ_URL: "nio://activemq:61616"
CORE_AIO_URL : "http://transform-core-aio:8090"
FILE_STORE_URL: "http://shared-file-store:8099/alfresco/api/-default-/private/sfs/versions/1/file"
ports:
- 8095:8095
links:
- activemq
transform-core-aio:
image: alfresco/alfresco-transform-core-aio:${TRANSFORM_CORE_AIO_TAG}
mem_limit: 1536m
environment:
JAVA_OPTS: " -Xms256m -Xmx1536m"
ACTIVEMQ_URL: "nio://activemq:61616"
FILE_STORE_URL: "http://shared-file-store:8099/alfresco/api/-default-/private/sfs/versions/1/file"
ports:
- 8090:8090
links:
- activemq
shared-file-store:
image: alfresco/alfresco-shared-file-store:${SHARED_FILE_STORE_TAG}
mem_limit: 512m
environment:
JAVA_OPTS: " -Xms256m -Xmx512m"
scheduler.content.age.millis: 86400000
scheduler.cleanup.interval: 86400000
ports:
- 8099:8099
volumes:
- shared-file-store-volume:/tmp/Alfresco/sfs
share:
build:
context: ./share
args:
SHARE_TAG: ${SHARE_TAG}
mem_limit: 1g
environment:
REPO_HOST: "alfresco"
REPO_PORT: "8080"
JAVA_OPTS: "
-Xms500m
-Xmx500m
-Dalfresco.host=localhost
-Dalfresco.port=8080
-Dalfresco.context=alfresco
-Dalfresco.protocol=http
"
postgres:
image: postgres:${POSTGRES_TAG}
mem_limit: 512m
environment:
- POSTGRES_PASSWORD=alfresco
- POSTGRES_USER=alfresco
- POSTGRES_DB=alfresco
command: postgres -c max_connections=300 -c log_min_messages=LOG
ports:
- 5432:5432
activemq:
image: alfresco/alfresco-activemq:${ACTIVE_MQ_TAG}
mem_limit: 1g
ports:
- 8161:8161 # Web Console
- 5672:5672 # AMQP
- 61616:61616 # OpenWire
- 61613:61613 # STOMP
content-app:
image: quay.io/alfresco/alfresco-digital-workspace:${DIGITAL_WORKSPACE_TAG}
mem_limit: 128m
environment:
BASEPATH: ./
# HTTP proxy to provide HTTP Default port access to services
# SOLR API and SOLR Web Console are protected to avoid unauthenticated access
proxy:
image: nginx:stable-alpine
mem_limit: 128m
depends_on:
- alfresco
volumes:
- ./config/nginx.conf:/etc/nginx/nginx.conf
- ./config/nginx.htpasswd:/etc/nginx/conf.d/nginx.htpasswd
ports:
- 8080:8080
links:
- alfresco
- share
- solr6
- content-app
volumes:
shared-file-store-volume:
driver_opts:
type: tmpfs
device: tmpfs

163
e2e-test/release-testing/insight-engine-sharding/date/search/Dockerfile Executable file
View File

@@ -0,0 +1,163 @@
ARG SEARCH_TAG
FROM quay.io/alfresco/insight-engine:${SEARCH_TAG}
# COMMON
ARG ALFRESCO_HOSTNAME
ARG SOLR_HOSTNAME
ENV ALFRESCO_HOSTNAME $ALFRESCO_HOSTNAME
ENV SOLR_HOSTNAME $SOLR_HOSTNAME
# Configure Alfresco Service Name
RUN sed -i '/^bash.*/i sed -i "'"s/alfresco.host=localhost/alfresco.host=${ALFRESCO_HOSTNAME}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh && \
sed -i '/^bash.*/i sed -i "'"s/solr.host=localhost/solr.host=${SOLR_HOSTNAME}/g"'" ${DIST_DIR}/solrhome/conf/shared.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh
# COMMS
ARG ALFRESCO_COMMS
ENV ALFRESCO_COMMS $ALFRESCO_COMMS
# Configure SOLR cores to run in HTTPs mode from template
RUN if [ "$ALFRESCO_COMMS" == "https" ] ; then \
sed -i '/^bash.*/i sed -i "'"s/alfresco.secureComms=none/alfresco.secureComms=https/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
else \
sed -i '/^bash.*/i sed -i "'"s/alfresco.secureComms=https/alfresco.secureComms=none/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
# SSL
ARG TRUSTSTORE_TYPE
ENV TRUSTSTORE_TYPE $TRUSTSTORE_TYPE
ARG KEYSTORE_TYPE
ENV KEYSTORE_TYPE $KEYSTORE_TYPE
# Passwords using Env Vars
ARG PASSWORDS_AS_ENV_VARS
ENV PASSWORDS_AS_ENV_VARS $PASSWORDS_AS_ENV_VARS
# Set SSL properties
RUN if [ "$ALFRESCO_COMMS" == "https" ] ; then \
if [ "$PASSWORDS_AS_ENV_VARS" == "true" ]; then \
sed -i '/^bash.*/i \
sed -i "'"s/alfresco.encryption.ssl.keystore.location=.*/alfresco.encryption.ssl.keystore.location=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl-repo-client.keystore/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.keystore.passwordFileLocation=.*/alfresco.encryption.ssl.keystore.passwordFileLocation=/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.keystore.type=.*/alfresco.encryption.ssl.keystore.type=${KEYSTORE_TYPE}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.location=.*/alfresco.encryption.ssl.truststore.location=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl-repo-client.truststore/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.passwordFileLocation=.*/alfresco.encryption.ssl.truststore.passwordFileLocation=/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.type=.*/alfresco.encryption.ssl.truststore.type=${TRUSTSTORE_TYPE}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
else \
sed -i '/^bash.*/i \
sed -i "'"s/alfresco.encryption.ssl.keystore.location=.*/alfresco.encryption.ssl.keystore.location=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl.repo.client.keystore/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.keystore.passwordFileLocation=.*/alfresco.encryption.ssl.keystore.passwordFileLocation=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl-keystore-passwords.properties/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.keystore.type=.*/alfresco.encryption.ssl.keystore.type=${KEYSTORE_TYPE}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.location=.*/alfresco.encryption.ssl.truststore.location=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl.repo.client.truststore/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.passwordFileLocation=.*/alfresco.encryption.ssl.truststore.passwordFileLocation=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl-truststore-passwords.properties/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.type=.*/alfresco.encryption.ssl.truststore.type=${TRUSTSTORE_TYPE}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi \
fi
# REPLICATION
ARG ENABLE_MASTER
ARG ENABLE_SLAVE
ARG MASTER_HOST
ENV ENABLE_MASTER $ENABLE_MASTER
ENV ENABLE_SLAVE $ENABLE_SLAVE
ENV MASTER_HOST $MASTER_HOST
# Set Master / Slave configuration for this Node
RUN if [ "$ENABLE_MASTER" == "true" ] ; then \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
sed -i "/^bash.*/i sed -i '/^\\\\\s*<requestHandler name=\"\\\\/replication\".*/a \
<lst name=\"master\">\
<str name=\"replicateAfter\">commit</str>\
<str name=\"replicateAfter\">startup</str>\
<str name=\"confFiles\">schema.xml,stopwords.txt</str>\
</lst>' ${DIST_DIR}/solrhome/templates/rerank/conf/solrconfig.xml\n" ${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
RUN if [ "$ENABLE_SLAVE" == "true" ] ; then \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
sed -i "/^bash.*/i sed -i '/^\\\\\s*<requestHandler name=\"\\\\/replication\".*/a \
<lst name=\"slave\">\
<str name=\"masterUrl\">http://${MASTER_HOST}:8983/solr/alfresco</str>\
<str name=\"pollInterval\">00:00:60</str>\
</lst>' ${DIST_DIR}/solrhome/templates/rerank/conf/solrconfig.xml\n" ${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
# SHARDING
ARG ENABLE_SHARDING
ARG NUM_SHARDS
ARG SHARD_ID
ARG SHARDING_METHOD
ENV ENABLE_SHARDING $ENABLE_SHARDING
ENV NUM_SHARDS $NUM_SHARDS
ENV SHARD_ID $SHARD_ID
ENV SHARDING_METHOD $SHARDING_METHOD
# Set Port Number and Sharding ID for this Shard Service
RUN if [ "$ENABLE_SHARDING" == "true" ] ; then \
sed -i '/^bash.*/i echo "\nsolr.port.ssl=8983\nshard.instance=${SHARD_ID}" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh && \
if [[ ("$SHARDING_METHOD" != "DB_ID_RANGE") && ("$SHARDING_METHOD" != "EXPLICIT_ID_FALLBACK_LRIS") ]]; then \
sed -i '/^bash.*/i echo "\nshard.count=${NUM_SHARDS}" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
sed -i '/^bash.*/i echo "\nalfresco.port=8080\nalfresco.port.ssl=8443\nalfresco.baseUrl=/alfresco" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
# SHARDING METHODS
ARG SHARD_RANGE
ENV SHARD_RANGE $SHARD_RANGE
ARG SHARD_KEY
ENV SHARD_KEY $SHARD_KEY
ARG SHARD_DATE_GROUPING
ENV SHARD_DATE_GROUPING $SHARD_DATE_GROUPING
# Set Shard Method and Shard Key property name
RUN if [ "$ENABLE_SHARDING" == "true" ] ; then \
sed -i '/^bash.*/i sed -i "'"s/shard.method=DB_ID/shard.method=${SHARDING_METHOD}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
if [ "$SHARDING_METHOD" == "DB_ID_RANGE" ]; then \
sed -i '/^bash.*/i echo "\nshard.range=${SHARD_RANGE}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
if [ "$SHARDING_METHOD" == "DATE" ]; then \
sed -i '/^bash.*/i echo "\nshard.key=${SHARD_KEY}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh && \
sed -i '/^bash.*/i echo "\nshard.date.grouping=${SHARD_DATE_GROUPING}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
if [ "$SHARDING_METHOD" == "PROPERTY" ]; then \
sed -i '/^bash.*/i echo "\nshard.key=${SHARD_KEY}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
if [[ ("$SHARDING_METHOD" == "EXPLICIT_ID") || ("$SHARDING_METHOD" == "EXPLICIT_ID_FALLBACK_LRIS") ]]; then \
sed -i '/^bash.*/i echo "\nshard.key=${SHARD_KEY}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
fi
# GZIP COMPRESSION
ARG COMPRESS_CONTENT
ENV COMPRESS_CONTENT $COMPRESS_CONTENT
RUN if [ "$COMPRESS_CONTENT" == "true" ] ; then \
sed -i '/^bash.*/i sed -i "'"s/solr.request.content.compress=false/solr.request.content.compress=true/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
# Useless for 'none'/'http' communications with Alfresco
RUN mkdir ${DIST_DIR}/keystore \
&& chown -R solr:solr ${DIST_DIR}/keystore
# Set the search log level if requested.
ARG SEARCH_LOG_LEVEL
ENV SEARCH_LOG_LEVEL $SEARCH_LOG_LEVEL
RUN if [ "$SEARCH_LOG_LEVEL" ] ; then \
sed -i '/^bash.*/i sed -i "'"s/log4j.rootLogger=WARN, file, CONSOLE/log4j.rootLogger=${SEARCH_LOG_LEVEL}, file, CONSOLE/g"'" ${DIST_DIR}/logs/log4j.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
VOLUME ["${DIST_DIR}/keystore"]

16
e2e-test/release-testing/insight-engine-sharding/date/share/Dockerfile Normal file
View File

@@ -0,0 +1,16 @@
ARG SHARE_TAG
FROM quay.io/alfresco/alfresco-share:${SHARE_TAG}
ARG TOMCAT_DIR=/usr/local/tomcat
# Install modules
RUN mkdir -p $TOMCAT_DIR/amps
COPY modules/amps/* $TOMCAT_DIR/amps/
COPY modules/jars/* $TOMCAT_DIR/webapps/share/WEB-INF/lib/
RUN java -jar $TOMCAT_DIR/alfresco-mmt/alfresco-mmt*.jar install \
$TOMCAT_DIR/amps $TOMCAT_DIR/webapps/share -directory -nobackup -force;
# Copy custom content forms to deployment folder
COPY model/* $TOMCAT_DIR/shared/classes/alfresco/web-extension/

27
e2e-test/release-testing/insight-engine-sharding/date/share/model/share-config-custom-dev.xml Normal file
View File

@@ -0,0 +1,27 @@
<alfresco-config>
<!-- Add custom configuration -->
<config evaluator="string-compare" condition="DocumentLibrary">
<aspects>
<visible>
<aspect name="shard:sharding"/>
</visible>
<addable>
</addable>
<removeable>
</removeable>
</aspects>
</config>
<config evaluator="aspect" condition="shard:sharding">
<forms>
<form>
<field-visibility>
<show id="shard:shardId" />
</field-visibility>
</form>
</forms>
</config>
</alfresco-config>

0
e2e-test/release-testing/insight-engine-sharding/date/share/modules/amps/empty Normal file
View File

0
e2e-test/release-testing/insight-engine-sharding/date/share/modules/jars/empty Normal file
View File

15
e2e-test/release-testing/insight-engine-sharding/db_id/.env Executable file
View File

@@ -0,0 +1,15 @@
ALFRESCO_TAG=6.2.2
ALFRESCO_CE_TAG=6.2.0-ga
SHARE_TAG=6.2.1
POSTGRES_TAG=11.4
TRANSFORM_ROUTER_TAG=1.2.0
TRANSFORM_CORE_AIO_TAG=2.2.1
TRANSFORMER_TAG=2.1.0
SHARED_FILE_STORE_TAG=0.7.0
ACTIVE_MQ_TAG=5.15.8
DIGITAL_WORKSPACE_TAG=1.5.0
ACS_NGINX_TAG=3.0.1
ACS_COMMUNITY_NGINX_TAG=1.0.0
SEARCH_TAG=1.4.3.0-RC1
ZEPPELIN_TAG=latest
ACA_TAG=master-latest

73
e2e-test/release-testing/insight-engine-sharding/db_id/alfresco/Dockerfile Executable file
View File

@@ -0,0 +1,73 @@
ARG ALFRESCO_TAG
FROM alfresco/alfresco-content-repository:${ALFRESCO_TAG}
ARG TOMCAT_DIR=/usr/local/tomcat
ARG ALF_DATA_DIR=${TOMCAT_DIR}/alf_data
# COMMS
ARG SOLR_COMMS
ENV SOLR_COMMS $SOLR_COMMS
# SSL
ARG TRUSTSTORE_TYPE
ARG TRUSTSTORE_PASS
ARG KEYSTORE_TYPE
ARG KEYSTORE_PASS
ENV TRUSTSTORE_TYPE=$TRUSTSTORE_TYPE \
TRUSTSTORE_PASS=$TRUSTSTORE_PASS \
KEYSTORE_TYPE=$KEYSTORE_TYPE \
KEYSTORE_PASS=$KEYSTORE_PASS
# Expose keystore folder
# Useless for 'none'/'http' communications with SOLR
VOLUME ["${ALF_DATA_DIR}/keystore"]
USER root
# Install modules and addons
RUN mkdir -p $TOMCAT_DIR/amps
COPY modules/amps/* $TOMCAT_DIR/amps/
COPY modules/jars/* $TOMCAT_DIR/webapps/alfresco/WEB-INF/lib/
RUN java -jar $TOMCAT_DIR/alfresco-mmt/alfresco-mmt*.jar install \
$TOMCAT_DIR/amps $TOMCAT_DIR/webapps/alfresco -directory -nobackup -force;
# Default value in "repository.properties" is "dir.keystore=classpath:alfresco/keystore"
RUN if [ "$SOLR_COMMS" == "https" ] ; then \
echo -e "\n\
dir.keystore=${ALF_DATA_DIR}/keystore\n\
alfresco.encryption.ssl.keystore.type=${TRUSTSTORE_TYPE}\n\
alfresco.encryption.ssl.truststore.type=${KEYSTORE_TYPE}\n\
" >> ${TOMCAT_DIR}/shared/classes/alfresco-global.properties; \
fi
# Enable SSL by adding the proper Connector to server.xml
RUN if [ "$SOLR_COMMS" == "https" ] ; then \
sed -i "s/\
[[:space:]]\+<\/Engine>/\n\
<\/Engine>\n\
<Connector port=\"8443\" protocol=\"org.apache.coyote.http11.Http11Protocol\"\n\
connectionTimeout=\"20000\"\n\
SSLEnabled=\"true\" maxThreads=\"150\" scheme=\"https\"\n\
keystoreFile=\"\/usr\/local\/tomcat\/alf_data\/keystore\/ssl.keystore\"\n\
keystorePass=\"${KEYSTORE_PASS}\" keystoreType=\"${KEYSTORE_TYPE}\" secure=\"true\"\n\
truststoreFile=\"\/usr\/local\/tomcat\/alf_data\/keystore\/ssl.truststore\"\n\
truststorePass=\"${TRUSTSTORE_PASS}\" truststoreType=\"${TRUSTSTORE_TYPE}\" clientAuth=\"want\" sslProtocol=\"TLS\">\n\
<\/Connector>/g" ${TOMCAT_DIR}/conf/server.xml; \
fi
# GZIP COMPRESSION
ARG COMPRESS_CONTENT
ENV COMPRESS_CONTENT $COMPRESS_CONTENT
RUN if [ "$COMPRESS_CONTENT" == "true" ] ; then \
sed -i "s/\
[[:space:]]\+connectionTimeout=\"20000\"/\n\
connectionTimeout=\"20000\"\n\
compression=\"on\"\n\
compressionMinSize=\"1\"\n\
/g" ${TOMCAT_DIR}/conf/server.xml; \
fi
# Copy custom content model to deployment folder
COPY model/* $TOMCAT_DIR/shared/classes/alfresco/extension/

15
e2e-test/release-testing/insight-engine-sharding/db_id/alfresco/model/sharding-content-model-context.xml Normal file
View File

@@ -0,0 +1,15 @@
<?xml version='1.0' encoding='UTF-8'?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.0.xsd">
<bean id="org.alfresco.sharding-content-model.dictionaryBootstrap" parent="dictionaryModelBootstrap" depends-on="dictionaryBootstrap">
<property name="models">
<list>
<value>alfresco/extension/sharding-content-model.xml</value>
</list>
</property>
</bean>
</beans>

27
e2e-test/release-testing/insight-engine-sharding/db_id/alfresco/model/sharding-content-model.xml Normal file
View File

@@ -0,0 +1,27 @@
<?xml version="1.0" encoding="UTF-8"?>
<model name="shard:contentModel" xmlns="http://www.alfresco.org/model/dictionary/1.0">
<description>Explicit Routing for Sharding Sample Model</description>
<imports>
<import uri="http://www.alfresco.org/model/dictionary/1.0" prefix="d"/>
</imports>
<namespaces>
<namespace uri="http://www.alfresco.org/model/sharding/1.0" prefix="shard"/>
</namespaces>
<aspects>
<aspect name="shard:sharding">
<title>Sharding</title>
<properties>
<!-- Shard number to index this content -->
<property name="shard:shardId">
<title>Shard Id</title>
<type>d:text</type>
</property>
</properties>
</aspect>
</aspects>
</model>

0
e2e-test/release-testing/insight-engine-sharding/db_id/alfresco/modules/amps/empty Normal file
View File

0
e2e-test/release-testing/insight-engine-sharding/db_id/alfresco/modules/jars/empty Normal file
View File

67
e2e-test/release-testing/insight-engine-sharding/db_id/config/nginx.conf Normal file
View File

@@ -0,0 +1,67 @@
worker_processes 1;
events {
worker_connections 1024;
}
http {
server {
listen *:8080;
client_max_body_size 0;
set $allowOriginSite *;
proxy_pass_request_headers on;
proxy_pass_header Set-Cookie;
# External settings, do not remove
#ENV_ACCESS_LOG
proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $host:$server_port;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass_header Set-Cookie;
# Alfresco Repository
location /alfresco/ {
proxy_pass http://alfresco:8080;
}
# REST API (Swagger)
location /api-explorer/ {
proxy_pass http://alfresco:8080;
}
# SOLR Web Console (Master)
location /solr/ {
proxy_pass http://solr6:8983;
# Basic authentication
auth_basic "Solr web console";
auth_basic_user_file /etc/nginx/conf.d/nginx.htpasswd;
}
# Alfresco Share Web App
location /share/ {
proxy_pass http://share:8080;
}
# Alfresco Content App
location / {
proxy_pass http://content-app:8080;
}
}
}

1
e2e-test/release-testing/insight-engine-sharding/db_id/config/nginx.htpasswd Normal file
View File

@@ -0,0 +1 @@
admin:YWG41BPzVAkN6

209
e2e-test/release-testing/insight-engine-sharding/db_id/docker-compose.yml Executable file
View File

@@ -0,0 +1,209 @@
# Using version 2 as 3 does not support resource constraint options (cpu_*, mem_* limits) for non swarm mode in Compose
version: "2"
services:
alfresco:
build:
context: ./alfresco
args:
ALFRESCO_TAG: ${ALFRESCO_TAG}
SOLR_COMMS: none
COMPRESS_CONTENT: "false"
mem_limit: 1800m
environment:
JAVA_OPTS: "
-Ddb.driver=org.postgresql.Driver
-Ddb.username=alfresco
-Ddb.password=alfresco
-Ddb.url=jdbc:postgresql://postgres:5432/alfresco
-Dsolr.host=solr6
-Dsolr.port.ssl=8983
-Dsolr.secureComms=none
-Dsolr.baseUrl=/solr
-Dsolr.useDynamicShardRegistration=true
-Dindex.subsystem.name=solr6
-Daos.baseUrlOverwrite=http://localhost:8080/alfresco/aos
-Dmessaging.broker.url=\"failover:(nio://activemq:61616)?timeout=3000&jms.useCompression=true\"
-Ddeployment.method=DOCKER_COMPOSE
-Dtransform.service.enabled=true
-Dtransform.service.url=http://transform-router:8095
-Dsfs.url=http://shared-file-store:8099/
-DlocalTransform.core-aio.url=http://transform-core-aio:8090/
-Dalfresco-pdf-renderer.url=http://transform-core-aio:8090/
-Djodconverter.url=http://transform-core-aio:8090/
-Dimg.url=http://transform-core-aio:8090/
-Dtika.url=http://transform-core-aio:8090/
-Dtransform.misc.url=http://transform-core-aio:8090/
-Dcsrf.filter.enabled=false
-Dalfresco.restApi.basicAuthScheme=true
-Xms1500m -Xmx1500m
"
solr6:
build:
context: ./search
args:
SEARCH_TAG: ${SEARCH_TAG}
SOLR_HOSTNAME: solr6
ALFRESCO_HOSTNAME: alfresco
ALFRESCO_COMMS: none
ENABLE_SHARDING: "true"
NUM_SHARDS: "2"
SHARD_ID: "0"
SHARDING_METHOD: DB_ID
COMPRESS_CONTENT: "false"
SEARCH_LOG_LEVEL: WARN
mem_limit: 1200m
environment:
#Solr needs to know how to register itself with Alfresco
SOLR_ALFRESCO_HOST: "alfresco"
SOLR_ALFRESCO_PORT: "8080"
ALFRESCO_SECURE_COMMS: none
#Alfresco needs to know how to call solr
SOLR_SOLR_HOST: "solr6"
SOLR_SOLR_PORT: "8983"
#Create the default alfresco and archive cores
SOLR_CREATE_ALFRESCO_DEFAULTS: "alfresco,archive"
SOLR_JAVA_MEM: "-Xms1g -Xmx1g"
ports:
- 8083:8983
solr6secondary:
build:
context: ./search
args:
SEARCH_TAG: ${SEARCH_TAG}
SOLR_HOSTNAME: solr6secondary
ALFRESCO_HOSTNAME: alfresco
ALFRESCO_COMMS: none
ENABLE_SHARDING: "true"
NUM_SHARDS: "2"
SHARD_ID: "1"
SHARDING_METHOD: DB_ID
COMPRESS_CONTENT: "false"
mem_limit: 1200m
environment:
#Solr needs to know how to register itself with Alfresco
SOLR_ALFRESCO_HOST: "alfresco"
SOLR_ALFRESCO_PORT: "8080"
ALFRESCO_SECURE_COMMS: none
#Alfresco needs to know how to call solr
SOLR_SOLR_HOST: "solr6secondary"
SOLR_SOLR_PORT: "8983"
#Create the default alfresco and archive cores
SOLR_CREATE_ALFRESCO_DEFAULTS: "alfresco,archive"
SOLR_JAVA_MEM: "-Xms1g -Xmx1g"
ports:
- 8084:8983
transform-router:
mem_limit: 512m
image: quay.io/alfresco/alfresco-transform-router:${TRANSFORM_ROUTER_TAG}
environment:
JAVA_OPTS: " -Xms256m -Xmx512m"
ACTIVEMQ_URL: "nio://activemq:61616"
CORE_AIO_URL : "http://transform-core-aio:8090"
FILE_STORE_URL: "http://shared-file-store:8099/alfresco/api/-default-/private/sfs/versions/1/file"
ports:
- 8095:8095
links:
- activemq
transform-core-aio:
image: alfresco/alfresco-transform-core-aio:${TRANSFORM_CORE_AIO_TAG}
mem_limit: 1536m
environment:
JAVA_OPTS: " -Xms256m -Xmx1536m"
ACTIVEMQ_URL: "nio://activemq:61616"
FILE_STORE_URL: "http://shared-file-store:8099/alfresco/api/-default-/private/sfs/versions/1/file"
ports:
- 8090:8090
links:
- activemq
shared-file-store:
image: alfresco/alfresco-shared-file-store:${SHARED_FILE_STORE_TAG}
mem_limit: 512m
environment:
JAVA_OPTS: " -Xms256m -Xmx512m"
scheduler.content.age.millis: 86400000
scheduler.cleanup.interval: 86400000
ports:
- 8099:8099
volumes:
- shared-file-store-volume:/tmp/Alfresco/sfs
share:
build:
context: ./share
args:
SHARE_TAG: ${SHARE_TAG}
mem_limit: 1g
environment:
REPO_HOST: "alfresco"
REPO_PORT: "8080"
JAVA_OPTS: "
-Xms500m
-Xmx500m
-Dalfresco.host=localhost
-Dalfresco.port=8080
-Dalfresco.context=alfresco
-Dalfresco.protocol=http
"
postgres:
image: postgres:${POSTGRES_TAG}
mem_limit: 512m
environment:
- POSTGRES_PASSWORD=alfresco
- POSTGRES_USER=alfresco
- POSTGRES_DB=alfresco
command: postgres -c max_connections=300 -c log_min_messages=LOG
ports:
- 5432:5432
activemq:
image: alfresco/alfresco-activemq:${ACTIVE_MQ_TAG}
mem_limit: 1g
ports:
- 8161:8161 # Web Console
- 5672:5672 # AMQP
- 61616:61616 # OpenWire
- 61613:61613 # STOMP
content-app:
image: quay.io/alfresco/alfresco-digital-workspace:${DIGITAL_WORKSPACE_TAG}
mem_limit: 128m
environment:
BASEPATH: ./
# HTTP proxy to provide HTTP Default port access to services
# SOLR API and SOLR Web Console are protected to avoid unauthenticated access
proxy:
image: nginx:stable-alpine
mem_limit: 128m
depends_on:
- alfresco
volumes:
- ./config/nginx.conf:/etc/nginx/nginx.conf
- ./config/nginx.htpasswd:/etc/nginx/conf.d/nginx.htpasswd
ports:
- 8080:8080
links:
- alfresco
- share
- solr6
- content-app
volumes:
shared-file-store-volume:
driver_opts:
type: tmpfs
device: tmpfs

163
e2e-test/release-testing/insight-engine-sharding/db_id/search/Dockerfile Executable file
View File

@@ -0,0 +1,163 @@
ARG SEARCH_TAG
FROM quay.io/alfresco/insight-engine:${SEARCH_TAG}
# COMMON
ARG ALFRESCO_HOSTNAME
ARG SOLR_HOSTNAME
ENV ALFRESCO_HOSTNAME $ALFRESCO_HOSTNAME
ENV SOLR_HOSTNAME $SOLR_HOSTNAME
# Configure Alfresco Service Name
RUN sed -i '/^bash.*/i sed -i "'"s/alfresco.host=localhost/alfresco.host=${ALFRESCO_HOSTNAME}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh && \
sed -i '/^bash.*/i sed -i "'"s/solr.host=localhost/solr.host=${SOLR_HOSTNAME}/g"'" ${DIST_DIR}/solrhome/conf/shared.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh
# COMMS
ARG ALFRESCO_COMMS
ENV ALFRESCO_COMMS $ALFRESCO_COMMS
# Configure SOLR cores to run in HTTPs mode from template
RUN if [ "$ALFRESCO_COMMS" == "https" ] ; then \
sed -i '/^bash.*/i sed -i "'"s/alfresco.secureComms=none/alfresco.secureComms=https/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
else \
sed -i '/^bash.*/i sed -i "'"s/alfresco.secureComms=https/alfresco.secureComms=none/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
# SSL
ARG TRUSTSTORE_TYPE
ENV TRUSTSTORE_TYPE $TRUSTSTORE_TYPE
ARG KEYSTORE_TYPE
ENV KEYSTORE_TYPE $KEYSTORE_TYPE
# Passwords using Env Vars
ARG PASSWORDS_AS_ENV_VARS
ENV PASSWORDS_AS_ENV_VARS $PASSWORDS_AS_ENV_VARS
# Set SSL properties
RUN if [ "$ALFRESCO_COMMS" == "https" ] ; then \
if [ "$PASSWORDS_AS_ENV_VARS" == "true" ]; then \
sed -i '/^bash.*/i \
sed -i "'"s/alfresco.encryption.ssl.keystore.location=.*/alfresco.encryption.ssl.keystore.location=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl-repo-client.keystore/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.keystore.passwordFileLocation=.*/alfresco.encryption.ssl.keystore.passwordFileLocation=/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.keystore.type=.*/alfresco.encryption.ssl.keystore.type=${KEYSTORE_TYPE}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.location=.*/alfresco.encryption.ssl.truststore.location=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl-repo-client.truststore/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.passwordFileLocation=.*/alfresco.encryption.ssl.truststore.passwordFileLocation=/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.type=.*/alfresco.encryption.ssl.truststore.type=${TRUSTSTORE_TYPE}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
else \
sed -i '/^bash.*/i \
sed -i "'"s/alfresco.encryption.ssl.keystore.location=.*/alfresco.encryption.ssl.keystore.location=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl.repo.client.keystore/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.keystore.passwordFileLocation=.*/alfresco.encryption.ssl.keystore.passwordFileLocation=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl-keystore-passwords.properties/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.keystore.type=.*/alfresco.encryption.ssl.keystore.type=${KEYSTORE_TYPE}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.location=.*/alfresco.encryption.ssl.truststore.location=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl.repo.client.truststore/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.passwordFileLocation=.*/alfresco.encryption.ssl.truststore.passwordFileLocation=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl-truststore-passwords.properties/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.type=.*/alfresco.encryption.ssl.truststore.type=${TRUSTSTORE_TYPE}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi \
fi
# REPLICATION
ARG ENABLE_MASTER
ARG ENABLE_SLAVE
ARG MASTER_HOST
ENV ENABLE_MASTER $ENABLE_MASTER
ENV ENABLE_SLAVE $ENABLE_SLAVE
ENV MASTER_HOST $MASTER_HOST
# Set Master / Slave configuration for this Node
RUN if [ "$ENABLE_MASTER" == "true" ] ; then \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
sed -i "/^bash.*/i sed -i '/^\\\\\s*<requestHandler name=\"\\\\/replication\".*/a \
<lst name=\"master\">\
<str name=\"replicateAfter\">commit</str>\
<str name=\"replicateAfter\">startup</str>\
<str name=\"confFiles\">schema.xml,stopwords.txt</str>\
</lst>' ${DIST_DIR}/solrhome/templates/rerank/conf/solrconfig.xml\n" ${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
RUN if [ "$ENABLE_SLAVE" == "true" ] ; then \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
sed -i "/^bash.*/i sed -i '/^\\\\\s*<requestHandler name=\"\\\\/replication\".*/a \
<lst name=\"slave\">\
<str name=\"masterUrl\">http://${MASTER_HOST}:8983/solr/alfresco</str>\
<str name=\"pollInterval\">00:00:60</str>\
</lst>' ${DIST_DIR}/solrhome/templates/rerank/conf/solrconfig.xml\n" ${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
# SHARDING
ARG ENABLE_SHARDING
ARG NUM_SHARDS
ARG SHARD_ID
ARG SHARDING_METHOD
ENV ENABLE_SHARDING $ENABLE_SHARDING
ENV NUM_SHARDS $NUM_SHARDS
ENV SHARD_ID $SHARD_ID
ENV SHARDING_METHOD $SHARDING_METHOD
# Set Port Number and Sharding ID for this Shard Service
RUN if [ "$ENABLE_SHARDING" == "true" ] ; then \
sed -i '/^bash.*/i echo "\nsolr.port.ssl=8983\nshard.instance=${SHARD_ID}" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh && \
if [[ ("$SHARDING_METHOD" != "DB_ID_RANGE") && ("$SHARDING_METHOD" != "EXPLICIT_ID_FALLBACK_LRIS") ]]; then \
sed -i '/^bash.*/i echo "\nshard.count=${NUM_SHARDS}" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
sed -i '/^bash.*/i echo "\nalfresco.port=8080\nalfresco.port.ssl=8443\nalfresco.baseUrl=/alfresco" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
# SHARDING METHODS
ARG SHARD_RANGE
ENV SHARD_RANGE $SHARD_RANGE
ARG SHARD_KEY
ENV SHARD_KEY $SHARD_KEY
ARG SHARD_DATE_GROUPING
ENV SHARD_DATE_GROUPING $SHARD_DATE_GROUPING
# Set Shard Method and Shard Key property name
RUN if [ "$ENABLE_SHARDING" == "true" ] ; then \
sed -i '/^bash.*/i sed -i "'"s/shard.method=DB_ID/shard.method=${SHARDING_METHOD}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
if [ "$SHARDING_METHOD" == "DB_ID_RANGE" ]; then \
sed -i '/^bash.*/i echo "\nshard.range=${SHARD_RANGE}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
if [ "$SHARDING_METHOD" == "DATE" ]; then \
sed -i '/^bash.*/i echo "\nshard.key=${SHARD_KEY}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh && \
sed -i '/^bash.*/i echo "\nshard.date.grouping=${SHARD_DATE_GROUPING}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
if [ "$SHARDING_METHOD" == "PROPERTY" ]; then \
sed -i '/^bash.*/i echo "\nshard.key=${SHARD_KEY}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
if [[ ("$SHARDING_METHOD" == "EXPLICIT_ID") || ("$SHARDING_METHOD" == "EXPLICIT_ID_FALLBACK_LRIS") ]]; then \
sed -i '/^bash.*/i echo "\nshard.key=${SHARD_KEY}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
fi
# GZIP COMPRESSION
ARG COMPRESS_CONTENT
ENV COMPRESS_CONTENT $COMPRESS_CONTENT
RUN if [ "$COMPRESS_CONTENT" == "true" ] ; then \
sed -i '/^bash.*/i sed -i "'"s/solr.request.content.compress=false/solr.request.content.compress=true/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
# Useless for 'none'/'http' communications with Alfresco
RUN mkdir ${DIST_DIR}/keystore \
&& chown -R solr:solr ${DIST_DIR}/keystore
# Set the search log level if requested.
ARG SEARCH_LOG_LEVEL
ENV SEARCH_LOG_LEVEL $SEARCH_LOG_LEVEL
RUN if [ "$SEARCH_LOG_LEVEL" ] ; then \
sed -i '/^bash.*/i sed -i "'"s/log4j.rootLogger=WARN, file, CONSOLE/log4j.rootLogger=${SEARCH_LOG_LEVEL}, file, CONSOLE/g"'" ${DIST_DIR}/logs/log4j.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
VOLUME ["${DIST_DIR}/keystore"]

16
e2e-test/release-testing/insight-engine-sharding/db_id/share/Dockerfile Normal file
View File

@@ -0,0 +1,16 @@
ARG SHARE_TAG
FROM quay.io/alfresco/alfresco-share:${SHARE_TAG}
ARG TOMCAT_DIR=/usr/local/tomcat
# Install modules
RUN mkdir -p $TOMCAT_DIR/amps
COPY modules/amps/* $TOMCAT_DIR/amps/
COPY modules/jars/* $TOMCAT_DIR/webapps/share/WEB-INF/lib/
RUN java -jar $TOMCAT_DIR/alfresco-mmt/alfresco-mmt*.jar install \
$TOMCAT_DIR/amps $TOMCAT_DIR/webapps/share -directory -nobackup -force;
# Copy custom content forms to deployment folder
COPY model/* $TOMCAT_DIR/shared/classes/alfresco/web-extension/

27
e2e-test/release-testing/insight-engine-sharding/db_id/share/model/share-config-custom-dev.xml Normal file
View File

@@ -0,0 +1,27 @@
<alfresco-config>
<!-- Add custom configuration -->
<config evaluator="string-compare" condition="DocumentLibrary">
<aspects>
<visible>
<aspect name="shard:sharding"/>
</visible>
<addable>
</addable>
<removeable>
</removeable>
</aspects>
</config>
<config evaluator="aspect" condition="shard:sharding">
<forms>
<form>
<field-visibility>
<show id="shard:shardId" />
</field-visibility>
</form>
</forms>
</config>
</alfresco-config>

0
e2e-test/release-testing/insight-engine-sharding/db_id/share/modules/amps/empty Normal file
View File

0
e2e-test/release-testing/insight-engine-sharding/db_id/share/modules/jars/empty Normal file
View File

15
e2e-test/release-testing/insight-engine-sharding/db_id_range/.env Executable file
View File

@@ -0,0 +1,15 @@
ALFRESCO_TAG=6.2.2
ALFRESCO_CE_TAG=6.2.0-ga
SHARE_TAG=6.2.1
POSTGRES_TAG=11.4
TRANSFORM_ROUTER_TAG=1.2.0
TRANSFORM_CORE_AIO_TAG=2.2.1
TRANSFORMER_TAG=2.1.0
SHARED_FILE_STORE_TAG=0.7.0
ACTIVE_MQ_TAG=5.15.8
DIGITAL_WORKSPACE_TAG=1.5.0
ACS_NGINX_TAG=3.0.1
ACS_COMMUNITY_NGINX_TAG=1.0.0
SEARCH_TAG=1.4.3.0-RC1
ZEPPELIN_TAG=latest
ACA_TAG=master-latest

73
e2e-test/release-testing/insight-engine-sharding/db_id_range/alfresco/Dockerfile Executable file
View File

@@ -0,0 +1,73 @@
ARG ALFRESCO_TAG
FROM alfresco/alfresco-content-repository:${ALFRESCO_TAG}
ARG TOMCAT_DIR=/usr/local/tomcat
ARG ALF_DATA_DIR=${TOMCAT_DIR}/alf_data
# COMMS
ARG SOLR_COMMS
ENV SOLR_COMMS $SOLR_COMMS
# SSL
ARG TRUSTSTORE_TYPE
ARG TRUSTSTORE_PASS
ARG KEYSTORE_TYPE
ARG KEYSTORE_PASS
ENV TRUSTSTORE_TYPE=$TRUSTSTORE_TYPE \
TRUSTSTORE_PASS=$TRUSTSTORE_PASS \
KEYSTORE_TYPE=$KEYSTORE_TYPE \
KEYSTORE_PASS=$KEYSTORE_PASS
# Expose keystore folder
# Useless for 'none'/'http' communications with SOLR
VOLUME ["${ALF_DATA_DIR}/keystore"]
USER root
# Install modules and addons
RUN mkdir -p $TOMCAT_DIR/amps
COPY modules/amps/* $TOMCAT_DIR/amps/
COPY modules/jars/* $TOMCAT_DIR/webapps/alfresco/WEB-INF/lib/
RUN java -jar $TOMCAT_DIR/alfresco-mmt/alfresco-mmt*.jar install \
$TOMCAT_DIR/amps $TOMCAT_DIR/webapps/alfresco -directory -nobackup -force;
# Default value in "repository.properties" is "dir.keystore=classpath:alfresco/keystore"
RUN if [ "$SOLR_COMMS" == "https" ] ; then \
echo -e "\n\
dir.keystore=${ALF_DATA_DIR}/keystore\n\
alfresco.encryption.ssl.keystore.type=${TRUSTSTORE_TYPE}\n\
alfresco.encryption.ssl.truststore.type=${KEYSTORE_TYPE}\n\
" >> ${TOMCAT_DIR}/shared/classes/alfresco-global.properties; \
fi
# Enable SSL by adding the proper Connector to server.xml
RUN if [ "$SOLR_COMMS" == "https" ] ; then \
sed -i "s/\
[[:space:]]\+<\/Engine>/\n\
<\/Engine>\n\
<Connector port=\"8443\" protocol=\"org.apache.coyote.http11.Http11Protocol\"\n\
connectionTimeout=\"20000\"\n\
SSLEnabled=\"true\" maxThreads=\"150\" scheme=\"https\"\n\
keystoreFile=\"\/usr\/local\/tomcat\/alf_data\/keystore\/ssl.keystore\"\n\
keystorePass=\"${KEYSTORE_PASS}\" keystoreType=\"${KEYSTORE_TYPE}\" secure=\"true\"\n\
truststoreFile=\"\/usr\/local\/tomcat\/alf_data\/keystore\/ssl.truststore\"\n\
truststorePass=\"${TRUSTSTORE_PASS}\" truststoreType=\"${TRUSTSTORE_TYPE}\" clientAuth=\"want\" sslProtocol=\"TLS\">\n\
<\/Connector>/g" ${TOMCAT_DIR}/conf/server.xml; \
fi
# GZIP COMPRESSION
ARG COMPRESS_CONTENT
ENV COMPRESS_CONTENT $COMPRESS_CONTENT
RUN if [ "$COMPRESS_CONTENT" == "true" ] ; then \
sed -i "s/\
[[:space:]]\+connectionTimeout=\"20000\"/\n\
connectionTimeout=\"20000\"\n\
compression=\"on\"\n\
compressionMinSize=\"1\"\n\
/g" ${TOMCAT_DIR}/conf/server.xml; \
fi
# Copy custom content model to deployment folder
COPY model/* $TOMCAT_DIR/shared/classes/alfresco/extension/

15
e2e-test/release-testing/insight-engine-sharding/db_id_range/alfresco/model/sharding-content-model-context.xml Normal file
View File

@@ -0,0 +1,15 @@
<?xml version='1.0' encoding='UTF-8'?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.0.xsd">
<bean id="org.alfresco.sharding-content-model.dictionaryBootstrap" parent="dictionaryModelBootstrap" depends-on="dictionaryBootstrap">
<property name="models">
<list>
<value>alfresco/extension/sharding-content-model.xml</value>
</list>
</property>
</bean>
</beans>

27
e2e-test/release-testing/insight-engine-sharding/db_id_range/alfresco/model/sharding-content-model.xml Normal file
View File

@@ -0,0 +1,27 @@
<?xml version="1.0" encoding="UTF-8"?>
<model name="shard:contentModel" xmlns="http://www.alfresco.org/model/dictionary/1.0">
<description>Explicit Routing for Sharding Sample Model</description>
<imports>
<import uri="http://www.alfresco.org/model/dictionary/1.0" prefix="d"/>
</imports>
<namespaces>
<namespace uri="http://www.alfresco.org/model/sharding/1.0" prefix="shard"/>
</namespaces>
<aspects>
<aspect name="shard:sharding">
<title>Sharding</title>
<properties>
<!-- Shard number to index this content -->
<property name="shard:shardId">
<title>Shard Id</title>
<type>d:text</type>
</property>
</properties>
</aspect>
</aspects>
</model>

0
e2e-test/release-testing/insight-engine-sharding/db_id_range/alfresco/modules/amps/empty Normal file
View File

0
e2e-test/release-testing/insight-engine-sharding/db_id_range/alfresco/modules/jars/empty Normal file
View File

67
e2e-test/release-testing/insight-engine-sharding/db_id_range/config/nginx.conf Normal file
View File

@@ -0,0 +1,67 @@
worker_processes 1;
events {
worker_connections 1024;
}
http {
server {
listen *:8080;
client_max_body_size 0;
set $allowOriginSite *;
proxy_pass_request_headers on;
proxy_pass_header Set-Cookie;
# External settings, do not remove
#ENV_ACCESS_LOG
proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $host:$server_port;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass_header Set-Cookie;
# Alfresco Repository
location /alfresco/ {
proxy_pass http://alfresco:8080;
}
# REST API (Swagger)
location /api-explorer/ {
proxy_pass http://alfresco:8080;
}
# SOLR Web Console (Master)
location /solr/ {
proxy_pass http://solr6:8983;
# Basic authentication
auth_basic "Solr web console";
auth_basic_user_file /etc/nginx/conf.d/nginx.htpasswd;
}
# Alfresco Share Web App
location /share/ {
proxy_pass http://share:8080;
}
# Alfresco Content App
location / {
proxy_pass http://content-app:8080;
}
}
}

1
e2e-test/release-testing/insight-engine-sharding/db_id_range/config/nginx.htpasswd Normal file
View File

@@ -0,0 +1 @@
admin:YWG41BPzVAkN6

211
e2e-test/release-testing/insight-engine-sharding/db_id_range/docker-compose.yml Executable file
View File

@@ -0,0 +1,211 @@
# Using version 2 as 3 does not support resource constraint options (cpu_*, mem_* limits) for non swarm mode in Compose
version: "2"
services:
alfresco:
build:
context: ./alfresco
args:
ALFRESCO_TAG: ${ALFRESCO_TAG}
SOLR_COMMS: none
COMPRESS_CONTENT: "false"
mem_limit: 1800m
environment:
JAVA_OPTS: "
-Ddb.driver=org.postgresql.Driver
-Ddb.username=alfresco
-Ddb.password=alfresco
-Ddb.url=jdbc:postgresql://postgres:5432/alfresco
-Dsolr.host=solr6
-Dsolr.port.ssl=8983
-Dsolr.secureComms=none
-Dsolr.baseUrl=/solr
-Dsolr.useDynamicShardRegistration=true
-Dindex.subsystem.name=solr6
-Daos.baseUrlOverwrite=http://localhost:8080/alfresco/aos
-Dmessaging.broker.url=\"failover:(nio://activemq:61616)?timeout=3000&jms.useCompression=true\"
-Ddeployment.method=DOCKER_COMPOSE
-Dtransform.service.enabled=true
-Dtransform.service.url=http://transform-router:8095
-Dsfs.url=http://shared-file-store:8099/
-DlocalTransform.core-aio.url=http://transform-core-aio:8090/
-Dalfresco-pdf-renderer.url=http://transform-core-aio:8090/
-Djodconverter.url=http://transform-core-aio:8090/
-Dimg.url=http://transform-core-aio:8090/
-Dtika.url=http://transform-core-aio:8090/
-Dtransform.misc.url=http://transform-core-aio:8090/
-Dcsrf.filter.enabled=false
-Dalfresco.restApi.basicAuthScheme=true
-Xms1500m -Xmx1500m
"
solr6:
build:
context: ./search
args:
SEARCH_TAG: ${SEARCH_TAG}
SOLR_HOSTNAME: solr6
ALFRESCO_HOSTNAME: alfresco
ALFRESCO_COMMS: none
ENABLE_SHARDING: "true"
NUM_SHARDS: "2"
SHARD_ID: "0"
SHARD_RANGE: "0-800"
SHARDING_METHOD: DB_ID_RANGE
COMPRESS_CONTENT: "false"
SEARCH_LOG_LEVEL: WARN
mem_limit: 1200m
environment:
#Solr needs to know how to register itself with Alfresco
SOLR_ALFRESCO_HOST: "alfresco"
SOLR_ALFRESCO_PORT: "8080"
ALFRESCO_SECURE_COMMS: none
#Alfresco needs to know how to call solr
SOLR_SOLR_HOST: "solr6"
SOLR_SOLR_PORT: "8983"
#Create the default alfresco and archive cores
SOLR_CREATE_ALFRESCO_DEFAULTS: "alfresco,archive"
SOLR_JAVA_MEM: "-Xms1g -Xmx1g"
ports:
- 8083:8983
solr6secondary:
build:
context: ./search
args:
SEARCH_TAG: ${SEARCH_TAG}
SOLR_HOSTNAME: solr6secondary
ALFRESCO_HOSTNAME: alfresco
ALFRESCO_COMMS: none
ENABLE_SHARDING: "true"
NUM_SHARDS: "2"
SHARD_ID: "1"
SHARD_RANGE: "801-40000"
SHARDING_METHOD: DB_ID_RANGE
COMPRESS_CONTENT: "false"
mem_limit: 1200m
environment:
#Solr needs to know how to register itself with Alfresco
SOLR_ALFRESCO_HOST: "alfresco"
SOLR_ALFRESCO_PORT: "8080"
ALFRESCO_SECURE_COMMS: none
#Alfresco needs to know how to call solr
SOLR_SOLR_HOST: "solr6secondary"
SOLR_SOLR_PORT: "8983"
#Create the default alfresco and archive cores
SOLR_CREATE_ALFRESCO_DEFAULTS: "alfresco,archive"
SOLR_JAVA_MEM: "-Xms1g -Xmx1g"
ports:
- 8084:8983
transform-router:
mem_limit: 512m
image: quay.io/alfresco/alfresco-transform-router:${TRANSFORM_ROUTER_TAG}
environment:
JAVA_OPTS: " -Xms256m -Xmx512m"
ACTIVEMQ_URL: "nio://activemq:61616"
CORE_AIO_URL : "http://transform-core-aio:8090"
FILE_STORE_URL: "http://shared-file-store:8099/alfresco/api/-default-/private/sfs/versions/1/file"
ports:
- 8095:8095
links:
- activemq
transform-core-aio:
image: alfresco/alfresco-transform-core-aio:${TRANSFORM_CORE_AIO_TAG}
mem_limit: 1536m
environment:
JAVA_OPTS: " -Xms256m -Xmx1536m"
ACTIVEMQ_URL: "nio://activemq:61616"
FILE_STORE_URL: "http://shared-file-store:8099/alfresco/api/-default-/private/sfs/versions/1/file"
ports:
- 8090:8090
links:
- activemq
shared-file-store:
image: alfresco/alfresco-shared-file-store:${SHARED_FILE_STORE_TAG}
mem_limit: 512m
environment:
JAVA_OPTS: " -Xms256m -Xmx512m"
scheduler.content.age.millis: 86400000
scheduler.cleanup.interval: 86400000
ports:
- 8099:8099
volumes:
- shared-file-store-volume:/tmp/Alfresco/sfs
share:
build:
context: ./share
args:
SHARE_TAG: ${SHARE_TAG}
mem_limit: 1g
environment:
REPO_HOST: "alfresco"
REPO_PORT: "8080"
JAVA_OPTS: "
-Xms500m
-Xmx500m
-Dalfresco.host=localhost
-Dalfresco.port=8080
-Dalfresco.context=alfresco
-Dalfresco.protocol=http
"
postgres:
image: postgres:${POSTGRES_TAG}
mem_limit: 512m
environment:
- POSTGRES_PASSWORD=alfresco
- POSTGRES_USER=alfresco
- POSTGRES_DB=alfresco
command: postgres -c max_connections=300 -c log_min_messages=LOG
ports:
- 5432:5432
activemq:
image: alfresco/alfresco-activemq:${ACTIVE_MQ_TAG}
mem_limit: 1g
ports:
- 8161:8161 # Web Console
- 5672:5672 # AMQP
- 61616:61616 # OpenWire
- 61613:61613 # STOMP
content-app:
image: quay.io/alfresco/alfresco-digital-workspace:${DIGITAL_WORKSPACE_TAG}
mem_limit: 128m
environment:
BASEPATH: ./
# HTTP proxy to provide HTTP Default port access to services
# SOLR API and SOLR Web Console are protected to avoid unauthenticated access
proxy:
image: nginx:stable-alpine
mem_limit: 128m
depends_on:
- alfresco
volumes:
- ./config/nginx.conf:/etc/nginx/nginx.conf
- ./config/nginx.htpasswd:/etc/nginx/conf.d/nginx.htpasswd
ports:
- 8080:8080
links:
- alfresco
- share
- solr6
- content-app
volumes:
shared-file-store-volume:
driver_opts:
type: tmpfs
device: tmpfs

163
e2e-test/release-testing/insight-engine-sharding/db_id_range/search/Dockerfile Executable file
View File

@@ -0,0 +1,163 @@
ARG SEARCH_TAG
FROM quay.io/alfresco/insight-engine:${SEARCH_TAG}
# COMMON
ARG ALFRESCO_HOSTNAME
ARG SOLR_HOSTNAME
ENV ALFRESCO_HOSTNAME $ALFRESCO_HOSTNAME
ENV SOLR_HOSTNAME $SOLR_HOSTNAME
# Configure Alfresco Service Name
RUN sed -i '/^bash.*/i sed -i "'"s/alfresco.host=localhost/alfresco.host=${ALFRESCO_HOSTNAME}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh && \
sed -i '/^bash.*/i sed -i "'"s/solr.host=localhost/solr.host=${SOLR_HOSTNAME}/g"'" ${DIST_DIR}/solrhome/conf/shared.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh
# COMMS
ARG ALFRESCO_COMMS
ENV ALFRESCO_COMMS $ALFRESCO_COMMS
# Configure SOLR cores to run in HTTPs mode from template
RUN if [ "$ALFRESCO_COMMS" == "https" ] ; then \
sed -i '/^bash.*/i sed -i "'"s/alfresco.secureComms=none/alfresco.secureComms=https/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
else \
sed -i '/^bash.*/i sed -i "'"s/alfresco.secureComms=https/alfresco.secureComms=none/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
# SSL
ARG TRUSTSTORE_TYPE
ENV TRUSTSTORE_TYPE $TRUSTSTORE_TYPE
ARG KEYSTORE_TYPE
ENV KEYSTORE_TYPE $KEYSTORE_TYPE
# Passwords using Env Vars
ARG PASSWORDS_AS_ENV_VARS
ENV PASSWORDS_AS_ENV_VARS $PASSWORDS_AS_ENV_VARS
# Set SSL properties
RUN if [ "$ALFRESCO_COMMS" == "https" ] ; then \
if [ "$PASSWORDS_AS_ENV_VARS" == "true" ]; then \
sed -i '/^bash.*/i \
sed -i "'"s/alfresco.encryption.ssl.keystore.location=.*/alfresco.encryption.ssl.keystore.location=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl-repo-client.keystore/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.keystore.passwordFileLocation=.*/alfresco.encryption.ssl.keystore.passwordFileLocation=/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.keystore.type=.*/alfresco.encryption.ssl.keystore.type=${KEYSTORE_TYPE}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.location=.*/alfresco.encryption.ssl.truststore.location=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl-repo-client.truststore/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.passwordFileLocation=.*/alfresco.encryption.ssl.truststore.passwordFileLocation=/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.type=.*/alfresco.encryption.ssl.truststore.type=${TRUSTSTORE_TYPE}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
else \
sed -i '/^bash.*/i \
sed -i "'"s/alfresco.encryption.ssl.keystore.location=.*/alfresco.encryption.ssl.keystore.location=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl.repo.client.keystore/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.keystore.passwordFileLocation=.*/alfresco.encryption.ssl.keystore.passwordFileLocation=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl-keystore-passwords.properties/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.keystore.type=.*/alfresco.encryption.ssl.keystore.type=${KEYSTORE_TYPE}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.location=.*/alfresco.encryption.ssl.truststore.location=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl.repo.client.truststore/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.passwordFileLocation=.*/alfresco.encryption.ssl.truststore.passwordFileLocation=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl-truststore-passwords.properties/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.type=.*/alfresco.encryption.ssl.truststore.type=${TRUSTSTORE_TYPE}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi \
fi
# REPLICATION
ARG ENABLE_MASTER
ARG ENABLE_SLAVE
ARG MASTER_HOST
ENV ENABLE_MASTER $ENABLE_MASTER
ENV ENABLE_SLAVE $ENABLE_SLAVE
ENV MASTER_HOST $MASTER_HOST
# Set Master / Slave configuration for this Node
RUN if [ "$ENABLE_MASTER" == "true" ] ; then \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
sed -i "/^bash.*/i sed -i '/^\\\\\s*<requestHandler name=\"\\\\/replication\".*/a \
<lst name=\"master\">\
<str name=\"replicateAfter\">commit</str>\
<str name=\"replicateAfter\">startup</str>\
<str name=\"confFiles\">schema.xml,stopwords.txt</str>\
</lst>' ${DIST_DIR}/solrhome/templates/rerank/conf/solrconfig.xml\n" ${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
RUN if [ "$ENABLE_SLAVE" == "true" ] ; then \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
sed -i "/^bash.*/i sed -i '/^\\\\\s*<requestHandler name=\"\\\\/replication\".*/a \
<lst name=\"slave\">\
<str name=\"masterUrl\">http://${MASTER_HOST}:8983/solr/alfresco</str>\
<str name=\"pollInterval\">00:00:60</str>\
</lst>' ${DIST_DIR}/solrhome/templates/rerank/conf/solrconfig.xml\n" ${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
# SHARDING
ARG ENABLE_SHARDING
ARG NUM_SHARDS
ARG SHARD_ID
ARG SHARDING_METHOD
ENV ENABLE_SHARDING $ENABLE_SHARDING
ENV NUM_SHARDS $NUM_SHARDS
ENV SHARD_ID $SHARD_ID
ENV SHARDING_METHOD $SHARDING_METHOD
# Set Port Number and Sharding ID for this Shard Service
RUN if [ "$ENABLE_SHARDING" == "true" ] ; then \
sed -i '/^bash.*/i echo "\nsolr.port.ssl=8983\nshard.instance=${SHARD_ID}" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh && \
if [[ ("$SHARDING_METHOD" != "DB_ID_RANGE") && ("$SHARDING_METHOD" != "EXPLICIT_ID_FALLBACK_LRIS") ]]; then \
sed -i '/^bash.*/i echo "\nshard.count=${NUM_SHARDS}" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
sed -i '/^bash.*/i echo "\nalfresco.port=8080\nalfresco.port.ssl=8443\nalfresco.baseUrl=/alfresco" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
# SHARDING METHODS
ARG SHARD_RANGE
ENV SHARD_RANGE $SHARD_RANGE
ARG SHARD_KEY
ENV SHARD_KEY $SHARD_KEY
ARG SHARD_DATE_GROUPING
ENV SHARD_DATE_GROUPING $SHARD_DATE_GROUPING
# Set Shard Method and Shard Key property name
RUN if [ "$ENABLE_SHARDING" == "true" ] ; then \
sed -i '/^bash.*/i sed -i "'"s/shard.method=DB_ID/shard.method=${SHARDING_METHOD}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
if [ "$SHARDING_METHOD" == "DB_ID_RANGE" ]; then \
sed -i '/^bash.*/i echo "\nshard.range=${SHARD_RANGE}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
if [ "$SHARDING_METHOD" == "DATE" ]; then \
sed -i '/^bash.*/i echo "\nshard.key=${SHARD_KEY}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh && \
sed -i '/^bash.*/i echo "\nshard.date.grouping=${SHARD_DATE_GROUPING}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
if [ "$SHARDING_METHOD" == "PROPERTY" ]; then \
sed -i '/^bash.*/i echo "\nshard.key=${SHARD_KEY}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
if [[ ("$SHARDING_METHOD" == "EXPLICIT_ID") || ("$SHARDING_METHOD" == "EXPLICIT_ID_FALLBACK_LRIS") ]]; then \
sed -i '/^bash.*/i echo "\nshard.key=${SHARD_KEY}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
fi
# GZIP COMPRESSION
ARG COMPRESS_CONTENT
ENV COMPRESS_CONTENT $COMPRESS_CONTENT
RUN if [ "$COMPRESS_CONTENT" == "true" ] ; then \
sed -i '/^bash.*/i sed -i "'"s/solr.request.content.compress=false/solr.request.content.compress=true/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
# Useless for 'none'/'http' communications with Alfresco
RUN mkdir ${DIST_DIR}/keystore \
&& chown -R solr:solr ${DIST_DIR}/keystore
# Set the search log level if requested.
ARG SEARCH_LOG_LEVEL
ENV SEARCH_LOG_LEVEL $SEARCH_LOG_LEVEL
RUN if [ "$SEARCH_LOG_LEVEL" ] ; then \
sed -i '/^bash.*/i sed -i "'"s/log4j.rootLogger=WARN, file, CONSOLE/log4j.rootLogger=${SEARCH_LOG_LEVEL}, file, CONSOLE/g"'" ${DIST_DIR}/logs/log4j.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
VOLUME ["${DIST_DIR}/keystore"]

16
e2e-test/release-testing/insight-engine-sharding/db_id_range/share/Dockerfile Normal file
View File

@@ -0,0 +1,16 @@
ARG SHARE_TAG
FROM quay.io/alfresco/alfresco-share:${SHARE_TAG}
ARG TOMCAT_DIR=/usr/local/tomcat
# Install modules
RUN mkdir -p $TOMCAT_DIR/amps
COPY modules/amps/* $TOMCAT_DIR/amps/
COPY modules/jars/* $TOMCAT_DIR/webapps/share/WEB-INF/lib/
RUN java -jar $TOMCAT_DIR/alfresco-mmt/alfresco-mmt*.jar install \
$TOMCAT_DIR/amps $TOMCAT_DIR/webapps/share -directory -nobackup -force;
# Copy custom content forms to deployment folder
COPY model/* $TOMCAT_DIR/shared/classes/alfresco/web-extension/

27
e2e-test/release-testing/insight-engine-sharding/db_id_range/share/model/share-config-custom-dev.xml Normal file
View File

@@ -0,0 +1,27 @@
<alfresco-config>
<!-- Add custom configuration -->
<config evaluator="string-compare" condition="DocumentLibrary">
<aspects>
<visible>
<aspect name="shard:sharding"/>
</visible>
<addable>
</addable>
<removeable>
</removeable>
</aspects>
</config>
<config evaluator="aspect" condition="shard:sharding">
<forms>
<form>
<field-visibility>
<show id="shard:shardId" />
</field-visibility>
</form>
</forms>
</config>
</alfresco-config>

0
e2e-test/release-testing/insight-engine-sharding/db_id_range/share/modules/amps/empty Normal file
View File

0
e2e-test/release-testing/insight-engine-sharding/db_id_range/share/modules/jars/empty Normal file
View File

15
e2e-test/release-testing/insight-engine-sharding/explicit_id/.env Executable file
View File

@@ -0,0 +1,15 @@
ALFRESCO_TAG=6.2.2
ALFRESCO_CE_TAG=6.2.0-ga
SHARE_TAG=6.2.1
POSTGRES_TAG=11.4
TRANSFORM_ROUTER_TAG=1.2.0
TRANSFORM_CORE_AIO_TAG=2.2.1
TRANSFORMER_TAG=2.1.0
SHARED_FILE_STORE_TAG=0.7.0
ACTIVE_MQ_TAG=5.15.8
DIGITAL_WORKSPACE_TAG=1.5.0
ACS_NGINX_TAG=3.0.1
ACS_COMMUNITY_NGINX_TAG=1.0.0
SEARCH_TAG=1.4.3.0-RC1
ZEPPELIN_TAG=latest
ACA_TAG=master-latest

73
e2e-test/release-testing/insight-engine-sharding/explicit_id/alfresco/Dockerfile Executable file
View File

@@ -0,0 +1,73 @@
ARG ALFRESCO_TAG
FROM alfresco/alfresco-content-repository:${ALFRESCO_TAG}
ARG TOMCAT_DIR=/usr/local/tomcat
ARG ALF_DATA_DIR=${TOMCAT_DIR}/alf_data
# COMMS
ARG SOLR_COMMS
ENV SOLR_COMMS $SOLR_COMMS
# SSL
ARG TRUSTSTORE_TYPE
ARG TRUSTSTORE_PASS
ARG KEYSTORE_TYPE
ARG KEYSTORE_PASS
ENV TRUSTSTORE_TYPE=$TRUSTSTORE_TYPE \
TRUSTSTORE_PASS=$TRUSTSTORE_PASS \
KEYSTORE_TYPE=$KEYSTORE_TYPE \
KEYSTORE_PASS=$KEYSTORE_PASS
# Expose keystore folder
# Useless for 'none'/'http' communications with SOLR
VOLUME ["${ALF_DATA_DIR}/keystore"]
USER root
# Install modules and addons
RUN mkdir -p $TOMCAT_DIR/amps
COPY modules/amps/* $TOMCAT_DIR/amps/
COPY modules/jars/* $TOMCAT_DIR/webapps/alfresco/WEB-INF/lib/
RUN java -jar $TOMCAT_DIR/alfresco-mmt/alfresco-mmt*.jar install \
$TOMCAT_DIR/amps $TOMCAT_DIR/webapps/alfresco -directory -nobackup -force;
# Default value in "repository.properties" is "dir.keystore=classpath:alfresco/keystore"
RUN if [ "$SOLR_COMMS" == "https" ] ; then \
echo -e "\n\
dir.keystore=${ALF_DATA_DIR}/keystore\n\
alfresco.encryption.ssl.keystore.type=${TRUSTSTORE_TYPE}\n\
alfresco.encryption.ssl.truststore.type=${KEYSTORE_TYPE}\n\
" >> ${TOMCAT_DIR}/shared/classes/alfresco-global.properties; \
fi
# Enable SSL by adding the proper Connector to server.xml
RUN if [ "$SOLR_COMMS" == "https" ] ; then \
sed -i "s/\
[[:space:]]\+<\/Engine>/\n\
<\/Engine>\n\
<Connector port=\"8443\" protocol=\"org.apache.coyote.http11.Http11Protocol\"\n\
connectionTimeout=\"20000\"\n\
SSLEnabled=\"true\" maxThreads=\"150\" scheme=\"https\"\n\
keystoreFile=\"\/usr\/local\/tomcat\/alf_data\/keystore\/ssl.keystore\"\n\
keystorePass=\"${KEYSTORE_PASS}\" keystoreType=\"${KEYSTORE_TYPE}\" secure=\"true\"\n\
truststoreFile=\"\/usr\/local\/tomcat\/alf_data\/keystore\/ssl.truststore\"\n\
truststorePass=\"${TRUSTSTORE_PASS}\" truststoreType=\"${TRUSTSTORE_TYPE}\" clientAuth=\"want\" sslProtocol=\"TLS\">\n\
<\/Connector>/g" ${TOMCAT_DIR}/conf/server.xml; \
fi
# GZIP COMPRESSION
ARG COMPRESS_CONTENT
ENV COMPRESS_CONTENT $COMPRESS_CONTENT
RUN if [ "$COMPRESS_CONTENT" == "true" ] ; then \
sed -i "s/\
[[:space:]]\+connectionTimeout=\"20000\"/\n\
connectionTimeout=\"20000\"\n\
compression=\"on\"\n\
compressionMinSize=\"1\"\n\
/g" ${TOMCAT_DIR}/conf/server.xml; \
fi
# Copy custom content model to deployment folder
COPY model/* $TOMCAT_DIR/shared/classes/alfresco/extension/

15
e2e-test/release-testing/insight-engine-sharding/explicit_id/alfresco/model/sharding-content-model-context.xml Normal file
View File

@@ -0,0 +1,15 @@
<?xml version='1.0' encoding='UTF-8'?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.0.xsd">
<bean id="org.alfresco.sharding-content-model.dictionaryBootstrap" parent="dictionaryModelBootstrap" depends-on="dictionaryBootstrap">
<property name="models">
<list>
<value>alfresco/extension/sharding-content-model.xml</value>
</list>
</property>
</bean>
</beans>

27
e2e-test/release-testing/insight-engine-sharding/explicit_id/alfresco/model/sharding-content-model.xml Normal file
View File

@@ -0,0 +1,27 @@
<?xml version="1.0" encoding="UTF-8"?>
<model name="shard:contentModel" xmlns="http://www.alfresco.org/model/dictionary/1.0">
<description>Explicit Routing for Sharding Sample Model</description>
<imports>
<import uri="http://www.alfresco.org/model/dictionary/1.0" prefix="d"/>
</imports>
<namespaces>
<namespace uri="http://www.alfresco.org/model/sharding/1.0" prefix="shard"/>
</namespaces>
<aspects>
<aspect name="shard:sharding">
<title>Sharding</title>
<properties>
<!-- Shard number to index this content -->
<property name="shard:shardId">
<title>Shard Id</title>
<type>d:text</type>
</property>
</properties>
</aspect>
</aspects>
</model>

0
e2e-test/release-testing/insight-engine-sharding/explicit_id/alfresco/modules/amps/empty Normal file
View File

0
e2e-test/release-testing/insight-engine-sharding/explicit_id/alfresco/modules/jars/empty Normal file
View File

67
e2e-test/release-testing/insight-engine-sharding/explicit_id/config/nginx.conf Normal file
View File

@@ -0,0 +1,67 @@
worker_processes 1;
events {
worker_connections 1024;
}
http {
server {
listen *:8080;
client_max_body_size 0;
set $allowOriginSite *;
proxy_pass_request_headers on;
proxy_pass_header Set-Cookie;
# External settings, do not remove
#ENV_ACCESS_LOG
proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $host:$server_port;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass_header Set-Cookie;
# Alfresco Repository
location /alfresco/ {
proxy_pass http://alfresco:8080;
}
# REST API (Swagger)
location /api-explorer/ {
proxy_pass http://alfresco:8080;
}
# SOLR Web Console (Master)
location /solr/ {
proxy_pass http://solr6:8983;
# Basic authentication
auth_basic "Solr web console";
auth_basic_user_file /etc/nginx/conf.d/nginx.htpasswd;
}
# Alfresco Share Web App
location /share/ {
proxy_pass http://share:8080;
}
# Alfresco Content App
location / {
proxy_pass http://content-app:8080;
}
}
}

1
e2e-test/release-testing/insight-engine-sharding/explicit_id/config/nginx.htpasswd Normal file
View File

@@ -0,0 +1 @@
admin:YWG41BPzVAkN6

211
e2e-test/release-testing/insight-engine-sharding/explicit_id/docker-compose.yml Executable file
View File

@@ -0,0 +1,211 @@
# Using version 2 as 3 does not support resource constraint options (cpu_*, mem_* limits) for non swarm mode in Compose
version: "2"
services:
alfresco:
build:
context: ./alfresco
args:
ALFRESCO_TAG: ${ALFRESCO_TAG}
SOLR_COMMS: none
COMPRESS_CONTENT: "false"
mem_limit: 1800m
environment:
JAVA_OPTS: "
-Ddb.driver=org.postgresql.Driver
-Ddb.username=alfresco
-Ddb.password=alfresco
-Ddb.url=jdbc:postgresql://postgres:5432/alfresco
-Dsolr.host=solr6
-Dsolr.port.ssl=8983
-Dsolr.secureComms=none
-Dsolr.baseUrl=/solr
-Dsolr.useDynamicShardRegistration=true
-Dindex.subsystem.name=solr6
-Daos.baseUrlOverwrite=http://localhost:8080/alfresco/aos
-Dmessaging.broker.url=\"failover:(nio://activemq:61616)?timeout=3000&jms.useCompression=true\"
-Ddeployment.method=DOCKER_COMPOSE
-Dtransform.service.enabled=true
-Dtransform.service.url=http://transform-router:8095
-Dsfs.url=http://shared-file-store:8099/
-DlocalTransform.core-aio.url=http://transform-core-aio:8090/
-Dalfresco-pdf-renderer.url=http://transform-core-aio:8090/
-Djodconverter.url=http://transform-core-aio:8090/
-Dimg.url=http://transform-core-aio:8090/
-Dtika.url=http://transform-core-aio:8090/
-Dtransform.misc.url=http://transform-core-aio:8090/
-Dcsrf.filter.enabled=false
-Dalfresco.restApi.basicAuthScheme=true
-Xms1500m -Xmx1500m
"
solr6:
build:
context: ./search
args:
SEARCH_TAG: ${SEARCH_TAG}
SOLR_HOSTNAME: solr6
ALFRESCO_HOSTNAME: alfresco
ALFRESCO_COMMS: none
ENABLE_SHARDING: "true"
NUM_SHARDS: "2"
SHARD_ID: "0"
SHARD_KEY: "shard:shardId"
SHARDING_METHOD: EXPLICIT_ID
COMPRESS_CONTENT: "false"
SEARCH_LOG_LEVEL: WARN
mem_limit: 1200m
environment:
#Solr needs to know how to register itself with Alfresco
SOLR_ALFRESCO_HOST: "alfresco"
SOLR_ALFRESCO_PORT: "8080"
ALFRESCO_SECURE_COMMS: none
#Alfresco needs to know how to call solr
SOLR_SOLR_HOST: "solr6"
SOLR_SOLR_PORT: "8983"
#Create the default alfresco and archive cores
SOLR_CREATE_ALFRESCO_DEFAULTS: "alfresco,archive"
SOLR_JAVA_MEM: "-Xms1g -Xmx1g"
ports:
- 8083:8983
solr6secondary:
build:
context: ./search
args:
SEARCH_TAG: ${SEARCH_TAG}
SOLR_HOSTNAME: solr6secondary
ALFRESCO_HOSTNAME: alfresco
ALFRESCO_COMMS: none
ENABLE_SHARDING: "true"
NUM_SHARDS: "2"
SHARD_ID: "1"
SHARD_KEY: "shard:shardId"
SHARDING_METHOD: EXPLICIT_ID
COMPRESS_CONTENT: "false"
mem_limit: 1200m
environment:
#Solr needs to know how to register itself with Alfresco
SOLR_ALFRESCO_HOST: "alfresco"
SOLR_ALFRESCO_PORT: "8080"
ALFRESCO_SECURE_COMMS: none
#Alfresco needs to know how to call solr
SOLR_SOLR_HOST: "solr6secondary"
SOLR_SOLR_PORT: "8983"
#Create the default alfresco and archive cores
SOLR_CREATE_ALFRESCO_DEFAULTS: "alfresco,archive"
SOLR_JAVA_MEM: "-Xms1g -Xmx1g"
ports:
- 8084:8983
transform-router:
mem_limit: 512m
image: quay.io/alfresco/alfresco-transform-router:${TRANSFORM_ROUTER_TAG}
environment:
JAVA_OPTS: " -Xms256m -Xmx512m"
ACTIVEMQ_URL: "nio://activemq:61616"
CORE_AIO_URL : "http://transform-core-aio:8090"
FILE_STORE_URL: "http://shared-file-store:8099/alfresco/api/-default-/private/sfs/versions/1/file"
ports:
- 8095:8095
links:
- activemq
transform-core-aio:
image: alfresco/alfresco-transform-core-aio:${TRANSFORM_CORE_AIO_TAG}
mem_limit: 1536m
environment:
JAVA_OPTS: " -Xms256m -Xmx1536m"
ACTIVEMQ_URL: "nio://activemq:61616"
FILE_STORE_URL: "http://shared-file-store:8099/alfresco/api/-default-/private/sfs/versions/1/file"
ports:
- 8090:8090
links:
- activemq
shared-file-store:
image: alfresco/alfresco-shared-file-store:${SHARED_FILE_STORE_TAG}
mem_limit: 512m
environment:
JAVA_OPTS: " -Xms256m -Xmx512m"
scheduler.content.age.millis: 86400000
scheduler.cleanup.interval: 86400000
ports:
- 8099:8099
volumes:
- shared-file-store-volume:/tmp/Alfresco/sfs
share:
build:
context: ./share
args:
SHARE_TAG: ${SHARE_TAG}
mem_limit: 1g
environment:
REPO_HOST: "alfresco"
REPO_PORT: "8080"
JAVA_OPTS: "
-Xms500m
-Xmx500m
-Dalfresco.host=localhost
-Dalfresco.port=8080
-Dalfresco.context=alfresco
-Dalfresco.protocol=http
"
postgres:
image: postgres:${POSTGRES_TAG}
mem_limit: 512m
environment:
- POSTGRES_PASSWORD=alfresco
- POSTGRES_USER=alfresco
- POSTGRES_DB=alfresco
command: postgres -c max_connections=300 -c log_min_messages=LOG
ports:
- 5432:5432
activemq:
image: alfresco/alfresco-activemq:${ACTIVE_MQ_TAG}
mem_limit: 1g
ports:
- 8161:8161 # Web Console
- 5672:5672 # AMQP
- 61616:61616 # OpenWire
- 61613:61613 # STOMP
content-app:
image: quay.io/alfresco/alfresco-digital-workspace:${DIGITAL_WORKSPACE_TAG}
mem_limit: 128m
environment:
BASEPATH: ./
# HTTP proxy to provide HTTP Default port access to services
# SOLR API and SOLR Web Console are protected to avoid unauthenticated access
proxy:
image: nginx:stable-alpine
mem_limit: 128m
depends_on:
- alfresco
volumes:
- ./config/nginx.conf:/etc/nginx/nginx.conf
- ./config/nginx.htpasswd:/etc/nginx/conf.d/nginx.htpasswd
ports:
- 8080:8080
links:
- alfresco
- share
- solr6
- content-app
volumes:
shared-file-store-volume:
driver_opts:
type: tmpfs
device: tmpfs

163
e2e-test/release-testing/insight-engine-sharding/explicit_id/search/Dockerfile Executable file
View File

@@ -0,0 +1,163 @@
ARG SEARCH_TAG
FROM quay.io/alfresco/insight-engine:${SEARCH_TAG}
# COMMON
ARG ALFRESCO_HOSTNAME
ARG SOLR_HOSTNAME
ENV ALFRESCO_HOSTNAME $ALFRESCO_HOSTNAME
ENV SOLR_HOSTNAME $SOLR_HOSTNAME
# Configure Alfresco Service Name
RUN sed -i '/^bash.*/i sed -i "'"s/alfresco.host=localhost/alfresco.host=${ALFRESCO_HOSTNAME}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh && \
sed -i '/^bash.*/i sed -i "'"s/solr.host=localhost/solr.host=${SOLR_HOSTNAME}/g"'" ${DIST_DIR}/solrhome/conf/shared.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh
# COMMS
ARG ALFRESCO_COMMS
ENV ALFRESCO_COMMS $ALFRESCO_COMMS
# Configure SOLR cores to run in HTTPs mode from template
RUN if [ "$ALFRESCO_COMMS" == "https" ] ; then \
sed -i '/^bash.*/i sed -i "'"s/alfresco.secureComms=none/alfresco.secureComms=https/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
else \
sed -i '/^bash.*/i sed -i "'"s/alfresco.secureComms=https/alfresco.secureComms=none/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
# SSL
ARG TRUSTSTORE_TYPE
ENV TRUSTSTORE_TYPE $TRUSTSTORE_TYPE
ARG KEYSTORE_TYPE
ENV KEYSTORE_TYPE $KEYSTORE_TYPE
# Passwords using Env Vars
ARG PASSWORDS_AS_ENV_VARS
ENV PASSWORDS_AS_ENV_VARS $PASSWORDS_AS_ENV_VARS
# Set SSL properties
RUN if [ "$ALFRESCO_COMMS" == "https" ] ; then \
if [ "$PASSWORDS_AS_ENV_VARS" == "true" ]; then \
sed -i '/^bash.*/i \
sed -i "'"s/alfresco.encryption.ssl.keystore.location=.*/alfresco.encryption.ssl.keystore.location=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl-repo-client.keystore/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.keystore.passwordFileLocation=.*/alfresco.encryption.ssl.keystore.passwordFileLocation=/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.keystore.type=.*/alfresco.encryption.ssl.keystore.type=${KEYSTORE_TYPE}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.location=.*/alfresco.encryption.ssl.truststore.location=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl-repo-client.truststore/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.passwordFileLocation=.*/alfresco.encryption.ssl.truststore.passwordFileLocation=/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.type=.*/alfresco.encryption.ssl.truststore.type=${TRUSTSTORE_TYPE}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
else \
sed -i '/^bash.*/i \
sed -i "'"s/alfresco.encryption.ssl.keystore.location=.*/alfresco.encryption.ssl.keystore.location=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl.repo.client.keystore/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.keystore.passwordFileLocation=.*/alfresco.encryption.ssl.keystore.passwordFileLocation=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl-keystore-passwords.properties/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.keystore.type=.*/alfresco.encryption.ssl.keystore.type=${KEYSTORE_TYPE}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.location=.*/alfresco.encryption.ssl.truststore.location=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl.repo.client.truststore/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.passwordFileLocation=.*/alfresco.encryption.ssl.truststore.passwordFileLocation=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl-truststore-passwords.properties/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.type=.*/alfresco.encryption.ssl.truststore.type=${TRUSTSTORE_TYPE}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi \
fi
# REPLICATION
ARG ENABLE_MASTER
ARG ENABLE_SLAVE
ARG MASTER_HOST
ENV ENABLE_MASTER $ENABLE_MASTER
ENV ENABLE_SLAVE $ENABLE_SLAVE
ENV MASTER_HOST $MASTER_HOST
# Set Master / Slave configuration for this Node
RUN if [ "$ENABLE_MASTER" == "true" ] ; then \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
sed -i "/^bash.*/i sed -i '/^\\\\\s*<requestHandler name=\"\\\\/replication\".*/a \
<lst name=\"master\">\
<str name=\"replicateAfter\">commit</str>\
<str name=\"replicateAfter\">startup</str>\
<str name=\"confFiles\">schema.xml,stopwords.txt</str>\
</lst>' ${DIST_DIR}/solrhome/templates/rerank/conf/solrconfig.xml\n" ${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
RUN if [ "$ENABLE_SLAVE" == "true" ] ; then \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
sed -i "/^bash.*/i sed -i '/^\\\\\s*<requestHandler name=\"\\\\/replication\".*/a \
<lst name=\"slave\">\
<str name=\"masterUrl\">http://${MASTER_HOST}:8983/solr/alfresco</str>\
<str name=\"pollInterval\">00:00:60</str>\
</lst>' ${DIST_DIR}/solrhome/templates/rerank/conf/solrconfig.xml\n" ${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
# SHARDING
ARG ENABLE_SHARDING
ARG NUM_SHARDS
ARG SHARD_ID
ARG SHARDING_METHOD
ENV ENABLE_SHARDING $ENABLE_SHARDING
ENV NUM_SHARDS $NUM_SHARDS
ENV SHARD_ID $SHARD_ID
ENV SHARDING_METHOD $SHARDING_METHOD
# Set Port Number and Sharding ID for this Shard Service
RUN if [ "$ENABLE_SHARDING" == "true" ] ; then \
sed -i '/^bash.*/i echo "\nsolr.port.ssl=8983\nshard.instance=${SHARD_ID}" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh && \
if [[ ("$SHARDING_METHOD" != "DB_ID_RANGE") && ("$SHARDING_METHOD" != "EXPLICIT_ID_FALLBACK_LRIS") ]]; then \
sed -i '/^bash.*/i echo "\nshard.count=${NUM_SHARDS}" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
sed -i '/^bash.*/i echo "\nalfresco.port=8080\nalfresco.port.ssl=8443\nalfresco.baseUrl=/alfresco" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
# SHARDING METHODS
ARG SHARD_RANGE
ENV SHARD_RANGE $SHARD_RANGE
ARG SHARD_KEY
ENV SHARD_KEY $SHARD_KEY
ARG SHARD_DATE_GROUPING
ENV SHARD_DATE_GROUPING $SHARD_DATE_GROUPING
# Set Shard Method and Shard Key property name
RUN if [ "$ENABLE_SHARDING" == "true" ] ; then \
sed -i '/^bash.*/i sed -i "'"s/shard.method=DB_ID/shard.method=${SHARDING_METHOD}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
if [ "$SHARDING_METHOD" == "DB_ID_RANGE" ]; then \
sed -i '/^bash.*/i echo "\nshard.range=${SHARD_RANGE}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
if [ "$SHARDING_METHOD" == "DATE" ]; then \
sed -i '/^bash.*/i echo "\nshard.key=${SHARD_KEY}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh && \
sed -i '/^bash.*/i echo "\nshard.date.grouping=${SHARD_DATE_GROUPING}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
if [ "$SHARDING_METHOD" == "PROPERTY" ]; then \
sed -i '/^bash.*/i echo "\nshard.key=${SHARD_KEY}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
if [[ ("$SHARDING_METHOD" == "EXPLICIT_ID") || ("$SHARDING_METHOD" == "EXPLICIT_ID_FALLBACK_LRIS") ]]; then \
sed -i '/^bash.*/i echo "\nshard.key=${SHARD_KEY}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
fi
# GZIP COMPRESSION
ARG COMPRESS_CONTENT
ENV COMPRESS_CONTENT $COMPRESS_CONTENT
RUN if [ "$COMPRESS_CONTENT" == "true" ] ; then \
sed -i '/^bash.*/i sed -i "'"s/solr.request.content.compress=false/solr.request.content.compress=true/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
# Useless for 'none'/'http' communications with Alfresco
RUN mkdir ${DIST_DIR}/keystore \
&& chown -R solr:solr ${DIST_DIR}/keystore
# Set the search log level if requested.
ARG SEARCH_LOG_LEVEL
ENV SEARCH_LOG_LEVEL $SEARCH_LOG_LEVEL
RUN if [ "$SEARCH_LOG_LEVEL" ] ; then \
sed -i '/^bash.*/i sed -i "'"s/log4j.rootLogger=WARN, file, CONSOLE/log4j.rootLogger=${SEARCH_LOG_LEVEL}, file, CONSOLE/g"'" ${DIST_DIR}/logs/log4j.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
VOLUME ["${DIST_DIR}/keystore"]

16
e2e-test/release-testing/insight-engine-sharding/explicit_id/share/Dockerfile Normal file
View File

@@ -0,0 +1,16 @@
ARG SHARE_TAG
FROM quay.io/alfresco/alfresco-share:${SHARE_TAG}
ARG TOMCAT_DIR=/usr/local/tomcat
# Install modules
RUN mkdir -p $TOMCAT_DIR/amps
COPY modules/amps/* $TOMCAT_DIR/amps/
COPY modules/jars/* $TOMCAT_DIR/webapps/share/WEB-INF/lib/
RUN java -jar $TOMCAT_DIR/alfresco-mmt/alfresco-mmt*.jar install \
$TOMCAT_DIR/amps $TOMCAT_DIR/webapps/share -directory -nobackup -force;
# Copy custom content forms to deployment folder
COPY model/* $TOMCAT_DIR/shared/classes/alfresco/web-extension/

27
e2e-test/release-testing/insight-engine-sharding/explicit_id/share/model/share-config-custom-dev.xml Normal file
View File

@@ -0,0 +1,27 @@
<alfresco-config>
<!-- Add custom configuration -->
<config evaluator="string-compare" condition="DocumentLibrary">
<aspects>
<visible>
<aspect name="shard:sharding"/>
</visible>
<addable>
</addable>
<removeable>
</removeable>
</aspects>
</config>
<config evaluator="aspect" condition="shard:sharding">
<forms>
<form>
<field-visibility>
<show id="shard:shardId" />
</field-visibility>
</form>
</forms>
</config>
</alfresco-config>

0
e2e-test/release-testing/insight-engine-sharding/explicit_id/share/modules/amps/empty Normal file
View File

0
e2e-test/release-testing/insight-engine-sharding/explicit_id/share/modules/jars/empty Normal file
View File

15
e2e-test/release-testing/insight-engine-sharding/lris/.env Executable file
View File

@@ -0,0 +1,15 @@
ALFRESCO_TAG=6.2.2
ALFRESCO_CE_TAG=6.2.0-ga
SHARE_TAG=6.2.1
POSTGRES_TAG=11.4
TRANSFORM_ROUTER_TAG=1.2.0
TRANSFORM_CORE_AIO_TAG=2.2.1
TRANSFORMER_TAG=2.1.0
SHARED_FILE_STORE_TAG=0.7.0
ACTIVE_MQ_TAG=5.15.8
DIGITAL_WORKSPACE_TAG=1.5.0
ACS_NGINX_TAG=3.0.1
ACS_COMMUNITY_NGINX_TAG=1.0.0
SEARCH_TAG=1.4.3.0-RC1
ZEPPELIN_TAG=latest
ACA_TAG=master-latest

73
e2e-test/release-testing/insight-engine-sharding/lris/alfresco/Dockerfile Executable file
View File

@@ -0,0 +1,73 @@
ARG ALFRESCO_TAG
FROM alfresco/alfresco-content-repository:${ALFRESCO_TAG}
ARG TOMCAT_DIR=/usr/local/tomcat
ARG ALF_DATA_DIR=${TOMCAT_DIR}/alf_data
# COMMS
ARG SOLR_COMMS
ENV SOLR_COMMS $SOLR_COMMS
# SSL
ARG TRUSTSTORE_TYPE
ARG TRUSTSTORE_PASS
ARG KEYSTORE_TYPE
ARG KEYSTORE_PASS
ENV TRUSTSTORE_TYPE=$TRUSTSTORE_TYPE \
TRUSTSTORE_PASS=$TRUSTSTORE_PASS \
KEYSTORE_TYPE=$KEYSTORE_TYPE \
KEYSTORE_PASS=$KEYSTORE_PASS
# Expose keystore folder
# Useless for 'none'/'http' communications with SOLR
VOLUME ["${ALF_DATA_DIR}/keystore"]
USER root
# Install modules and addons
RUN mkdir -p $TOMCAT_DIR/amps
COPY modules/amps/* $TOMCAT_DIR/amps/
COPY modules/jars/* $TOMCAT_DIR/webapps/alfresco/WEB-INF/lib/
RUN java -jar $TOMCAT_DIR/alfresco-mmt/alfresco-mmt*.jar install \
$TOMCAT_DIR/amps $TOMCAT_DIR/webapps/alfresco -directory -nobackup -force;
# Default value in "repository.properties" is "dir.keystore=classpath:alfresco/keystore"
RUN if [ "$SOLR_COMMS" == "https" ] ; then \
echo -e "\n\
dir.keystore=${ALF_DATA_DIR}/keystore\n\
alfresco.encryption.ssl.keystore.type=${TRUSTSTORE_TYPE}\n\
alfresco.encryption.ssl.truststore.type=${KEYSTORE_TYPE}\n\
" >> ${TOMCAT_DIR}/shared/classes/alfresco-global.properties; \
fi
# Enable SSL by adding the proper Connector to server.xml
RUN if [ "$SOLR_COMMS" == "https" ] ; then \
sed -i "s/\
[[:space:]]\+<\/Engine>/\n\
<\/Engine>\n\
<Connector port=\"8443\" protocol=\"org.apache.coyote.http11.Http11Protocol\"\n\
connectionTimeout=\"20000\"\n\
SSLEnabled=\"true\" maxThreads=\"150\" scheme=\"https\"\n\
keystoreFile=\"\/usr\/local\/tomcat\/alf_data\/keystore\/ssl.keystore\"\n\
keystorePass=\"${KEYSTORE_PASS}\" keystoreType=\"${KEYSTORE_TYPE}\" secure=\"true\"\n\
truststoreFile=\"\/usr\/local\/tomcat\/alf_data\/keystore\/ssl.truststore\"\n\
truststorePass=\"${TRUSTSTORE_PASS}\" truststoreType=\"${TRUSTSTORE_TYPE}\" clientAuth=\"want\" sslProtocol=\"TLS\">\n\
<\/Connector>/g" ${TOMCAT_DIR}/conf/server.xml; \
fi
# GZIP COMPRESSION
ARG COMPRESS_CONTENT
ENV COMPRESS_CONTENT $COMPRESS_CONTENT
RUN if [ "$COMPRESS_CONTENT" == "true" ] ; then \
sed -i "s/\
[[:space:]]\+connectionTimeout=\"20000\"/\n\
connectionTimeout=\"20000\"\n\
compression=\"on\"\n\
compressionMinSize=\"1\"\n\
/g" ${TOMCAT_DIR}/conf/server.xml; \
fi
# Copy custom content model to deployment folder
COPY model/* $TOMCAT_DIR/shared/classes/alfresco/extension/

15
e2e-test/release-testing/insight-engine-sharding/lris/alfresco/model/sharding-content-model-context.xml Normal file
View File

@@ -0,0 +1,15 @@
<?xml version='1.0' encoding='UTF-8'?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.0.xsd">
<bean id="org.alfresco.sharding-content-model.dictionaryBootstrap" parent="dictionaryModelBootstrap" depends-on="dictionaryBootstrap">
<property name="models">
<list>
<value>alfresco/extension/sharding-content-model.xml</value>
</list>
</property>
</bean>
</beans>

27
e2e-test/release-testing/insight-engine-sharding/lris/alfresco/model/sharding-content-model.xml Normal file
View File

@@ -0,0 +1,27 @@
<?xml version="1.0" encoding="UTF-8"?>
<model name="shard:contentModel" xmlns="http://www.alfresco.org/model/dictionary/1.0">
<description>Explicit Routing for Sharding Sample Model</description>
<imports>
<import uri="http://www.alfresco.org/model/dictionary/1.0" prefix="d"/>
</imports>
<namespaces>
<namespace uri="http://www.alfresco.org/model/sharding/1.0" prefix="shard"/>
</namespaces>
<aspects>
<aspect name="shard:sharding">
<title>Sharding</title>
<properties>
<!-- Shard number to index this content -->
<property name="shard:shardId">
<title>Shard Id</title>
<type>d:text</type>
</property>
</properties>
</aspect>
</aspects>
</model>

0
e2e-test/release-testing/insight-engine-sharding/lris/alfresco/modules/amps/empty Normal file
View File

0
e2e-test/release-testing/insight-engine-sharding/lris/alfresco/modules/jars/empty Normal file
View File

67
e2e-test/release-testing/insight-engine-sharding/lris/config/nginx.conf Normal file
View File

@@ -0,0 +1,67 @@
worker_processes 1;
events {
worker_connections 1024;
}
http {
server {
listen *:8080;
client_max_body_size 0;
set $allowOriginSite *;
proxy_pass_request_headers on;
proxy_pass_header Set-Cookie;
# External settings, do not remove
#ENV_ACCESS_LOG
proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $host:$server_port;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass_header Set-Cookie;
# Alfresco Repository
location /alfresco/ {
proxy_pass http://alfresco:8080;
}
# REST API (Swagger)
location /api-explorer/ {
proxy_pass http://alfresco:8080;
}
# SOLR Web Console (Master)
location /solr/ {
proxy_pass http://solr6:8983;
# Basic authentication
auth_basic "Solr web console";
auth_basic_user_file /etc/nginx/conf.d/nginx.htpasswd;
}
# Alfresco Share Web App
location /share/ {
proxy_pass http://share:8080;
}
# Alfresco Content App
location / {
proxy_pass http://content-app:8080;
}
}
}

1
e2e-test/release-testing/insight-engine-sharding/lris/config/nginx.htpasswd Normal file
View File

@@ -0,0 +1 @@
admin:YWG41BPzVAkN6

209
e2e-test/release-testing/insight-engine-sharding/lris/docker-compose.yml Executable file
View File

@@ -0,0 +1,209 @@
# Using version 2 as 3 does not support resource constraint options (cpu_*, mem_* limits) for non swarm mode in Compose
version: "2"
services:
alfresco:
build:
context: ./alfresco
args:
ALFRESCO_TAG: ${ALFRESCO_TAG}
SOLR_COMMS: none
COMPRESS_CONTENT: "false"
mem_limit: 1800m
environment:
JAVA_OPTS: "
-Ddb.driver=org.postgresql.Driver
-Ddb.username=alfresco
-Ddb.password=alfresco
-Ddb.url=jdbc:postgresql://postgres:5432/alfresco
-Dsolr.host=solr6
-Dsolr.port.ssl=8983
-Dsolr.secureComms=none
-Dsolr.baseUrl=/solr
-Dsolr.useDynamicShardRegistration=true
-Dindex.subsystem.name=solr6
-Daos.baseUrlOverwrite=http://localhost:8080/alfresco/aos
-Dmessaging.broker.url=\"failover:(nio://activemq:61616)?timeout=3000&jms.useCompression=true\"
-Ddeployment.method=DOCKER_COMPOSE
-Dtransform.service.enabled=true
-Dtransform.service.url=http://transform-router:8095
-Dsfs.url=http://shared-file-store:8099/
-DlocalTransform.core-aio.url=http://transform-core-aio:8090/
-Dalfresco-pdf-renderer.url=http://transform-core-aio:8090/
-Djodconverter.url=http://transform-core-aio:8090/
-Dimg.url=http://transform-core-aio:8090/
-Dtika.url=http://transform-core-aio:8090/
-Dtransform.misc.url=http://transform-core-aio:8090/
-Dcsrf.filter.enabled=false
-Dalfresco.restApi.basicAuthScheme=true
-Xms1500m -Xmx1500m
"
solr6:
build:
context: ./search
args:
SEARCH_TAG: ${SEARCH_TAG}
SOLR_HOSTNAME: solr6
ALFRESCO_HOSTNAME: alfresco
ALFRESCO_COMMS: none
ENABLE_SHARDING: "true"
NUM_SHARDS: "2"
SHARD_ID: "0"
SHARDING_METHOD: LAST_REGISTERED_INDEXING_SHARD
COMPRESS_CONTENT: "false"
SEARCH_LOG_LEVEL: WARN
mem_limit: 1200m
environment:
#Solr needs to know how to register itself with Alfresco
SOLR_ALFRESCO_HOST: "alfresco"
SOLR_ALFRESCO_PORT: "8080"
ALFRESCO_SECURE_COMMS: none
#Alfresco needs to know how to call solr
SOLR_SOLR_HOST: "solr6"
SOLR_SOLR_PORT: "8983"
#Create the default alfresco and archive cores
SOLR_CREATE_ALFRESCO_DEFAULTS: "alfresco,archive"
SOLR_JAVA_MEM: "-Xms1g -Xmx1g"
ports:
- 8083:8983
solr6secondary:
build:
context: ./search
args:
SEARCH_TAG: ${SEARCH_TAG}
SOLR_HOSTNAME: solr6secondary
ALFRESCO_HOSTNAME: alfresco
ALFRESCO_COMMS: none
ENABLE_SHARDING: "true"
NUM_SHARDS: "2"
SHARD_ID: "1"
SHARDING_METHOD: LAST_REGISTERED_INDEXING_SHARD
COMPRESS_CONTENT: "false"
mem_limit: 1200m
environment:
#Solr needs to know how to register itself with Alfresco
SOLR_ALFRESCO_HOST: "alfresco"
SOLR_ALFRESCO_PORT: "8080"
ALFRESCO_SECURE_COMMS: none
#Alfresco needs to know how to call solr
SOLR_SOLR_HOST: "solr6secondary"
SOLR_SOLR_PORT: "8983"
#Create the default alfresco and archive cores
SOLR_CREATE_ALFRESCO_DEFAULTS: "alfresco,archive"
SOLR_JAVA_MEM: "-Xms1g -Xmx1g"
ports:
- 8084:8983
transform-router:
mem_limit: 512m
image: quay.io/alfresco/alfresco-transform-router:${TRANSFORM_ROUTER_TAG}
environment:
JAVA_OPTS: " -Xms256m -Xmx512m"
ACTIVEMQ_URL: "nio://activemq:61616"
CORE_AIO_URL : "http://transform-core-aio:8090"
FILE_STORE_URL: "http://shared-file-store:8099/alfresco/api/-default-/private/sfs/versions/1/file"
ports:
- 8095:8095
links:
- activemq
transform-core-aio:
image: alfresco/alfresco-transform-core-aio:${TRANSFORM_CORE_AIO_TAG}
mem_limit: 1536m
environment:
JAVA_OPTS: " -Xms256m -Xmx1536m"
ACTIVEMQ_URL: "nio://activemq:61616"
FILE_STORE_URL: "http://shared-file-store:8099/alfresco/api/-default-/private/sfs/versions/1/file"
ports:
- 8090:8090
links:
- activemq
shared-file-store:
image: alfresco/alfresco-shared-file-store:${SHARED_FILE_STORE_TAG}
mem_limit: 512m
environment:
JAVA_OPTS: " -Xms256m -Xmx512m"
scheduler.content.age.millis: 86400000
scheduler.cleanup.interval: 86400000
ports:
- 8099:8099
volumes:
- shared-file-store-volume:/tmp/Alfresco/sfs
share:
build:
context: ./share
args:
SHARE_TAG: ${SHARE_TAG}
mem_limit: 1g
environment:
REPO_HOST: "alfresco"
REPO_PORT: "8080"
JAVA_OPTS: "
-Xms500m
-Xmx500m
-Dalfresco.host=localhost
-Dalfresco.port=8080
-Dalfresco.context=alfresco
-Dalfresco.protocol=http
"
postgres:
image: postgres:${POSTGRES_TAG}
mem_limit: 512m
environment:
- POSTGRES_PASSWORD=alfresco
- POSTGRES_USER=alfresco
- POSTGRES_DB=alfresco
command: postgres -c max_connections=300 -c log_min_messages=LOG
ports:
- 5432:5432
activemq:
image: alfresco/alfresco-activemq:${ACTIVE_MQ_TAG}
mem_limit: 1g
ports:
- 8161:8161 # Web Console
- 5672:5672 # AMQP
- 61616:61616 # OpenWire
- 61613:61613 # STOMP
content-app:
image: quay.io/alfresco/alfresco-digital-workspace:${DIGITAL_WORKSPACE_TAG}
mem_limit: 128m
environment:
BASEPATH: ./
# HTTP proxy to provide HTTP Default port access to services
# SOLR API and SOLR Web Console are protected to avoid unauthenticated access
proxy:
image: nginx:stable-alpine
mem_limit: 128m
depends_on:
- alfresco
volumes:
- ./config/nginx.conf:/etc/nginx/nginx.conf
- ./config/nginx.htpasswd:/etc/nginx/conf.d/nginx.htpasswd
ports:
- 8080:8080
links:
- alfresco
- share
- solr6
- content-app
volumes:
shared-file-store-volume:
driver_opts:
type: tmpfs
device: tmpfs

163
e2e-test/release-testing/insight-engine-sharding/lris/search/Dockerfile Executable file
View File

@@ -0,0 +1,163 @@
ARG SEARCH_TAG
FROM quay.io/alfresco/insight-engine:${SEARCH_TAG}
# COMMON
ARG ALFRESCO_HOSTNAME
ARG SOLR_HOSTNAME
ENV ALFRESCO_HOSTNAME $ALFRESCO_HOSTNAME
ENV SOLR_HOSTNAME $SOLR_HOSTNAME
# Configure Alfresco Service Name
RUN sed -i '/^bash.*/i sed -i "'"s/alfresco.host=localhost/alfresco.host=${ALFRESCO_HOSTNAME}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh && \
sed -i '/^bash.*/i sed -i "'"s/solr.host=localhost/solr.host=${SOLR_HOSTNAME}/g"'" ${DIST_DIR}/solrhome/conf/shared.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh
# COMMS
ARG ALFRESCO_COMMS
ENV ALFRESCO_COMMS $ALFRESCO_COMMS
# Configure SOLR cores to run in HTTPs mode from template
RUN if [ "$ALFRESCO_COMMS" == "https" ] ; then \
sed -i '/^bash.*/i sed -i "'"s/alfresco.secureComms=none/alfresco.secureComms=https/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
else \
sed -i '/^bash.*/i sed -i "'"s/alfresco.secureComms=https/alfresco.secureComms=none/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
# SSL
ARG TRUSTSTORE_TYPE
ENV TRUSTSTORE_TYPE $TRUSTSTORE_TYPE
ARG KEYSTORE_TYPE
ENV KEYSTORE_TYPE $KEYSTORE_TYPE
# Passwords using Env Vars
ARG PASSWORDS_AS_ENV_VARS
ENV PASSWORDS_AS_ENV_VARS $PASSWORDS_AS_ENV_VARS
# Set SSL properties
RUN if [ "$ALFRESCO_COMMS" == "https" ] ; then \
if [ "$PASSWORDS_AS_ENV_VARS" == "true" ]; then \
sed -i '/^bash.*/i \
sed -i "'"s/alfresco.encryption.ssl.keystore.location=.*/alfresco.encryption.ssl.keystore.location=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl-repo-client.keystore/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.keystore.passwordFileLocation=.*/alfresco.encryption.ssl.keystore.passwordFileLocation=/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.keystore.type=.*/alfresco.encryption.ssl.keystore.type=${KEYSTORE_TYPE}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.location=.*/alfresco.encryption.ssl.truststore.location=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl-repo-client.truststore/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.passwordFileLocation=.*/alfresco.encryption.ssl.truststore.passwordFileLocation=/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.type=.*/alfresco.encryption.ssl.truststore.type=${TRUSTSTORE_TYPE}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
else \
sed -i '/^bash.*/i \
sed -i "'"s/alfresco.encryption.ssl.keystore.location=.*/alfresco.encryption.ssl.keystore.location=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl.repo.client.keystore/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.keystore.passwordFileLocation=.*/alfresco.encryption.ssl.keystore.passwordFileLocation=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl-keystore-passwords.properties/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.keystore.type=.*/alfresco.encryption.ssl.keystore.type=${KEYSTORE_TYPE}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.location=.*/alfresco.encryption.ssl.truststore.location=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl.repo.client.truststore/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.passwordFileLocation=.*/alfresco.encryption.ssl.truststore.passwordFileLocation=\\\/opt\\\/alfresco-insight-engine\\\/keystore\\\/ssl-truststore-passwords.properties/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties && \
sed -i "'"s/alfresco.encryption.ssl.truststore.type=.*/alfresco.encryption.ssl.truststore.type=${TRUSTSTORE_TYPE}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi \
fi
# REPLICATION
ARG ENABLE_MASTER
ARG ENABLE_SLAVE
ARG MASTER_HOST
ENV ENABLE_MASTER $ENABLE_MASTER
ENV ENABLE_SLAVE $ENABLE_SLAVE
ENV MASTER_HOST $MASTER_HOST
# Set Master / Slave configuration for this Node
RUN if [ "$ENABLE_MASTER" == "true" ] ; then \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
sed -i "/^bash.*/i sed -i '/^\\\\\s*<requestHandler name=\"\\\\/replication\".*/a \
<lst name=\"master\">\
<str name=\"replicateAfter\">commit</str>\
<str name=\"replicateAfter\">startup</str>\
<str name=\"confFiles\">schema.xml,stopwords.txt</str>\
</lst>' ${DIST_DIR}/solrhome/templates/rerank/conf/solrconfig.xml\n" ${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
RUN if [ "$ENABLE_SLAVE" == "true" ] ; then \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
sed -i "/^bash.*/i sed -i '/^\\\\\s*<requestHandler name=\"\\\\/replication\".*/a \
<lst name=\"slave\">\
<str name=\"masterUrl\">http://${MASTER_HOST}:8983/solr/alfresco</str>\
<str name=\"pollInterval\">00:00:60</str>\
</lst>' ${DIST_DIR}/solrhome/templates/rerank/conf/solrconfig.xml\n" ${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
# SHARDING
ARG ENABLE_SHARDING
ARG NUM_SHARDS
ARG SHARD_ID
ARG SHARDING_METHOD
ENV ENABLE_SHARDING $ENABLE_SHARDING
ENV NUM_SHARDS $NUM_SHARDS
ENV SHARD_ID $SHARD_ID
ENV SHARDING_METHOD $SHARDING_METHOD
# Set Port Number and Sharding ID for this Shard Service
RUN if [ "$ENABLE_SHARDING" == "true" ] ; then \
sed -i '/^bash.*/i echo "\nsolr.port.ssl=8983\nshard.instance=${SHARD_ID}" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh && \
if [[ ("$SHARDING_METHOD" != "DB_ID_RANGE") && ("$SHARDING_METHOD" != "EXPLICIT_ID_FALLBACK_LRIS") ]]; then \
sed -i '/^bash.*/i echo "\nshard.count=${NUM_SHARDS}" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
sed -i '/^bash.*/i echo "\nalfresco.port=8080\nalfresco.port.ssl=8443\nalfresco.baseUrl=/alfresco" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
# SHARDING METHODS
ARG SHARD_RANGE
ENV SHARD_RANGE $SHARD_RANGE
ARG SHARD_KEY
ENV SHARD_KEY $SHARD_KEY
ARG SHARD_DATE_GROUPING
ENV SHARD_DATE_GROUPING $SHARD_DATE_GROUPING
# Set Shard Method and Shard Key property name
RUN if [ "$ENABLE_SHARDING" == "true" ] ; then \
sed -i '/^bash.*/i sed -i "'"s/shard.method=DB_ID/shard.method=${SHARDING_METHOD}/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
if [ "$SHARDING_METHOD" == "DB_ID_RANGE" ]; then \
sed -i '/^bash.*/i echo "\nshard.range=${SHARD_RANGE}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
if [ "$SHARDING_METHOD" == "DATE" ]; then \
sed -i '/^bash.*/i echo "\nshard.key=${SHARD_KEY}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh && \
sed -i '/^bash.*/i echo "\nshard.date.grouping=${SHARD_DATE_GROUPING}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
if [ "$SHARDING_METHOD" == "PROPERTY" ]; then \
sed -i '/^bash.*/i echo "\nshard.key=${SHARD_KEY}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
if [[ ("$SHARDING_METHOD" == "EXPLICIT_ID") || ("$SHARDING_METHOD" == "EXPLICIT_ID_FALLBACK_LRIS") ]]; then \
sed -i '/^bash.*/i echo "\nshard.key=${SHARD_KEY}\n" >> ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi; \
fi
# GZIP COMPRESSION
ARG COMPRESS_CONTENT
ENV COMPRESS_CONTENT $COMPRESS_CONTENT
RUN if [ "$COMPRESS_CONTENT" == "true" ] ; then \
sed -i '/^bash.*/i sed -i "'"s/solr.request.content.compress=false/solr.request.content.compress=true/g"'" ${DIST_DIR}/solrhome/templates/rerank/conf/solrcore.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
# Useless for 'none'/'http' communications with Alfresco
RUN mkdir ${DIST_DIR}/keystore \
&& chown -R solr:solr ${DIST_DIR}/keystore
# Set the search log level if requested.
ARG SEARCH_LOG_LEVEL
ENV SEARCH_LOG_LEVEL $SEARCH_LOG_LEVEL
RUN if [ "$SEARCH_LOG_LEVEL" ] ; then \
sed -i '/^bash.*/i sed -i "'"s/log4j.rootLogger=WARN, file, CONSOLE/log4j.rootLogger=${SEARCH_LOG_LEVEL}, file, CONSOLE/g"'" ${DIST_DIR}/logs/log4j.properties\n' \
${DIST_DIR}/solr/bin/search_config_setup.sh; \
fi
VOLUME ["${DIST_DIR}/keystore"]

16
e2e-test/release-testing/insight-engine-sharding/lris/share/Dockerfile Normal file
View File

@@ -0,0 +1,16 @@
ARG SHARE_TAG
FROM quay.io/alfresco/alfresco-share:${SHARE_TAG}
ARG TOMCAT_DIR=/usr/local/tomcat
# Install modules
RUN mkdir -p $TOMCAT_DIR/amps
COPY modules/amps/* $TOMCAT_DIR/amps/
COPY modules/jars/* $TOMCAT_DIR/webapps/share/WEB-INF/lib/
RUN java -jar $TOMCAT_DIR/alfresco-mmt/alfresco-mmt*.jar install \
$TOMCAT_DIR/amps $TOMCAT_DIR/webapps/share -directory -nobackup -force;
# Copy custom content forms to deployment folder
COPY model/* $TOMCAT_DIR/shared/classes/alfresco/web-extension/

27
e2e-test/release-testing/insight-engine-sharding/lris/share/model/share-config-custom-dev.xml Normal file
View File

@@ -0,0 +1,27 @@
<alfresco-config>
<!-- Add custom configuration -->
<config evaluator="string-compare" condition="DocumentLibrary">
<aspects>
<visible>
<aspect name="shard:sharding"/>
</visible>
<addable>
</addable>
<removeable>
</removeable>
</aspects>
</config>
<config evaluator="aspect" condition="shard:sharding">
<forms>
<form>
<field-visibility>
<show id="shard:shardId" />
</field-visibility>
</form>
</forms>
</config>
</alfresco-config>

0
e2e-test/release-testing/insight-engine-sharding/lris/share/modules/amps/empty Normal file
View File

0
e2e-test/release-testing/insight-engine-sharding/lris/share/modules/jars/empty Normal file
View File

Some files were not shown because too many files have changed in this diff Show More