REPO-3176: Remove future tagged documentation as they may not live in this repo

This commit is contained in:
dcaruana
2018-01-12 15:01:14 +00:00
parent 0c08de0b75
commit 2629c27e21
34 changed files with 0 additions and 1250 deletions

View File

@@ -51,7 +51,6 @@
### Identity Provider ### Identity Provider
* [Authentication](./identity-provider/authentication) * [Authentication](./identity-provider/authentication)
* [Identity](./identity-provider/identity)
* LDAP Sync * LDAP Sync
### Deployment ### Deployment
@@ -61,7 +60,6 @@
### Messaging ### Messaging
* Messages and Topics * Messages and Topics
* [Queue](./messaging/queue) (FUTURE)
### Infrastructure ### Infrastructure
* Module Framework * Module Framework

View File

@@ -1,260 +0,0 @@
## Identity
![Completeness Badge](https://img.shields.io/badge/Document_Level-In_Progress-yellow.svg?style=flat-square)
![Version Badge](https://img.shields.io/badge/Version-Future-blue.svg?style=flat-square)
### Purpose
The Identity Component manages the directory of entities (users, groups and systems) that can use Alfresco services, and is also responsible for authenticating such entities.
***
### Overview
***
### Artifacts and Guidance
* Source Code Link: https://github.com/Alfresco/identity-service
* License: LGPL
* Issue Tracker Link: https://issues.alfresco.com/jira/projects/ID
* Documentation Link:
* Contribution Model: Alfresco Closed Source
***
### Prerequisite Knowledge
***
### Design
#### Open Source Offerings
Several Open Source offerings exist in this space. These can be reviewed as possible aternatives for us developing our own implementation.
***
##### Keystone
Keystone is the Identity Service used in the OpenStack Platform. It is written in Python. It provides Authentication and Authorization services, but not User Profile Management services.
Source: https://github.com/openstack/keystone
License: Apache 2.0
Documentation: http://docs.openstack.org/developer/keystone/
***
##### Keycloak
Keycloak is the Identity Service developed by Red Hat. It is written in Java. It provides Authentication, Authorization services, and User Profile Management services.
Source: https://github.com/keycloak/keycloak
License: Apache 2.0
Documentation: http://www.keycloak.org/documentation.html
***
##### Syncope
Syncope is an Identity Service from Apache.
* Written in Java
* Deployed as a set of three war files
* Tested with Tomcat, Glassfish, Payara, Wildfly app servers
* Tested with PostgreSQL, MariaDB, MySQL, Oracle and MS SQL
* Provides User Profile Management Services
* Provides User Authentication Services including both an internal (self-contained) directory as well as integration with external directory systems
* Integrates with Activiti for a user provisioning workflow (see https://syncope.apache.org/apidocs/2.0/org/apache/syncope/core/workflow/activiti/spring/package-frame.html)
* Provides a full REST API, with a Swagger definition file
* Provides an admin console
Source: https://git-wip-us.apache.org/repos/asf/syncope.git
License: Apache 2.0
Documentation: https://syncope.apache.org/docs/index.html
***
##### SCIM
An implementation of SCIM would be a UPM system that seems to satisfy the user stories defined in https://issues.alfresco.com/jira/browse/ID-1.
There are [a number of such implementations](http://www.simplecloud.info), 16 of which are open source implementations.
Greg Melahn assessed SCIM in 2013. See [Architecture Notes](https://docs.google.com/a/alfresco.com/document/d/1JmHrd1AYWn2dyU43_FnFg2XQ69ktn7EIPKpDE3YAjAA/edit?usp=sharing)
***
#### Component Model
There are three principal configurations supported by the Identity Service
1. Internal Directory
This is the out of the box configuration in which the service persists content in a database.
2. External Directory
In this configuration the Identity Service is configured to delegate authentication and user profile storage to an external
service such as LDAP. A database is still used to store configuration data.
3. SAML
In this configuration the Identity Service is configured to delegate authentication and user profile storage to a SAML IdP.
In this configuration the Identity Service is viewed by the IdP as a Service Provider.
These configurations are illustrated below.
![Configurations](./resource/component/is-ecm-bpm-components.png)
#### Data Model
The following are the elements of the Data Model
* **Entity**
This represents the actual person, identified by a guid issued by the Identity Service.
* **User**
This is the user as it is known to an instance of Alfresco or to a system, like Box, that is integrated with Alfresco. A *User* is associated with one, and only one, *Entity*. A *User* is a member of at least one *Group*, and may be a member of multiple *Groups*.
* **Group**
This is a collection of *Users* or other *Groups*. A *Group* may only be a member of one *Group*. Each *Domain* has at least one *Group*, known as *ALL*, which has no parent.
* **Domain**
This represents a Directory, which may either be external (such as an LDAP) or internal (Alfresco).
* **Token**
This is something issued to a *User* that certifies that the *User* has been authenticated
by a *Domain*. Typically the Domain is an external system such as an LDAP which is responsible for
issuing the token.
In the case where Alfresco is authenticating the user because there is no associated external service, the token is issued by the Identity Service
* **System**
This represents the system to which the *User* is associated, such as "Box", "Salesforce" or "Alfresco Cloud".
![Question Badge](https://img.shields.io/badge/Design-Question-red.svg?style=flat-square) Do we really need both *System* and *Domain*?
![Data Model](./resource/data/data-model.png)
#### Data Dictionary
Not applicable
#### Flows
##### Add a User using Alfresco Admin
![Create Alfresco User](./resource/sequence/create-user-alfresco.png)
##### OAuth Flow (Alfresco and Box for example)
![OAuth Flow](./resource/sequence/oauth-flow.png)
##### SAML Flow (The Identity Service uses a SAML IdP for authentication)
This is a somewhat simplified flow in that interactions with the Queuing system are
omitted
![SAML Flow](./resource/sequence/saml-is-flow.png)
##### Use of a Proxy to Manage Alfresco and Activiti tokens
As an interim approach, Mario has proposed the introduction of a proxy that sits in front of both Alfresco and
Activiti to trigger a flow that would cause the Identity service to collect credentials from the
user, collect an alf and act token from Alfresco and Activity using those credentials and then store the tokens
with a generated common token that the client could pass-back as proof of authentication.
The purpose of the proxy in this flow, is to allow the use of the Identity Service in *current* deployments of Alfresco
and Activity where there is no opportunity to update Alfresco or Activiti.
![Common SSO Flow with a Proxy](./resource/sequence/identity-proxy.png)
#### Class Diagram
***
### APIs and Interfaces
#### REST API
![Design](https://img.shields.io/badge/Design-TODO-orange.svg?style=flat-square) Gav to provide a proposal for the CRUD part of this API.
NOTE: This API does not match the data model above, alternative proposed data model diagram TBD.
GET /users
POST /users <- Something close to the [SCIM](http://www.simplecloud.info/specs/draft-scim-core-schema-01.html#anchor8) user representation
GET /users/{id} -> Something close to the [SCIM](http://www.simplecloud.info/specs/draft-scim-core-schema-01.html#anchor8) user representation
GET /users/{id}/groups
GET /users/{id}/identities
GET /users/{id}/tokens (validate token only, checks all identities for user)
PUT & PATCH /users/{id}
DELETE /users/{id}
GET /groups
POST /groups
GET /groups/members
POST /groups/members
POST /tokens
##### Examples
<pre>
POST /users
{
"userName": "gcornwell",
"name": {
"formatted": "Mr. Gavin Paul Cornwell",
"familyName": "Cornwell",
"givenName": "Gavin",
"middleName": "Paul",
"honorificPrefix": "Mr."
},
"displayName": "Gavin Cornwell",
"emails": [
{
"value": "gavin.cornwell@alfresco.com",
"type": "work",
"primary": true
}
],
"identities": [
{
"systemId": "box",
"externalId": "gavinc",
"token": "e9e30dba-f08f-4109-8486-d5c6a331660a"
}
]
}
</pre>
***
### Configuration
***
### Performance Considerations
***
### Security Considerations
***
### Cloud Considerations
***
### Design Questions and Decisions
| Question | Decision | Rationale | Date |
| :----------------|:-----------------------| --------------------------:| ------------:|
| Should we start with existing Open Source (e.g. Keycloak)
|
***
### Quiz

Binary file not shown.

Before

Width:  |  Height:  |  Size: 49 KiB

View File

@@ -1,59 +0,0 @@
@startuml
left to right direction
skinparam linetype ortho
rectangle "Internal Directory Configuration" {
actor User as U1
database "Interna]\nDirectory\n(e.g. NoSQL)" as D1
rectangle "Identity Service" as IS1
rectangle "Alfresco" as ALF1
rectangle "Activiti" as ACT1
U1---->ACT1
U1---->ALF1
ACT1---->IS1
ALF1---->IS1
IS1--->D1
}
rectangle "External Directory Configuration" {
actor User as U2
database "External\nDirectory\n(e.g. LDAP)" as D2a
database "Configuration\nData\n(e.g. NoSQL)" as D2b
rectangle "Identity Service" as IS2
rectangle "Alfresco" as ALF2
rectangle "Activiti" as ACT2
U2---->ACT2
U2---->ALF2
ACT2---->IS2
ALF2---->IS2
IS2--->D2a
IS2--->D2b
}
rectangle "External Directory Configuration with SAML" {
actor User as U3
database "External\nDirectory\n(e.g. LDAP)" as D3a
database "Configuration\nData\n(e.g. NoSQL)" as D3b
rectangle "Identity Service" as IS3
rectangle "Alfresco" as ALF3
rectangle "Activiti" as ACT3
rectangle "SAML\nIdentity\nProvider" as IDP
U3---->ACT3
U3---->ALF3
ACT3---->IS3
ALF3---->IS3
IS3-->IDP
IDP-->D3a
IS3-->D3b
}
@enduml

Binary file not shown.

Before

Width:  |  Height:  |  Size: 21 KiB

View File

@@ -1,52 +0,0 @@
@startuml
title Identity Data Model
sprite $primary [17x12/16z] bOqv3e1030CJRzPn9Fx_NWY7n4eqJ3TJs6OVa5pTpD-5tl3YyFHG-4DsqAOnWgawWp0r0KGagDuGMYMJxbMrBxzLPJ_O0G00
left to right direction
object Entity {
<$primary> id
name
email addresses
photo
other profile info
}
object Identity {
<$primary> id
}
object Group{
<$primary> id
name
}
object Domain {
<$primary> id
directoryType
directoryEndPoint
}
object Token {
<$primary> id
type
expirationTime
authority
}
object System {
<$primary> id
url
}
Group "1:n"----"0:n" Identity : members
Group "0:1"----"0:n" Group : members
Group "0:n"----"1:1" Domain : members
Identity "0:n"----"1:1" Domain : members
Identity "1:1"----"0:n" Token: granted
Domain "1:1"----"0:n" Token:issues
Identity "0:n"----"0:n" System : member
Identity "0:n"----"0:n" Entity : member
@enduml

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.3 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.4 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.2 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 55 KiB

View File

@@ -1,51 +0,0 @@
@startuml
@startuml
Title: Creating an Alfresco User
actor "End User\n <newuser>" as U
actor "Administrator\n" as A
participant "Admin Console" as AC
participant "API Gateway" as AG
participant "Identity Service" as IS
participant "User Topic" as UT
database "Directory\nNoSQL" as D
note left of U
The user created in this flow is an 'internal user', that is
one that is entirely managed by Alfresco Content Services
and not known to a Directory Service such as LDAP
end note
U->A: Request account via email
A->AC: create User
activate AC
AC->AG: HTTP POST /users/<newuser>
activate AG
note left of AG
public API
end note
AG->IS: HTTP POST ?identity service
note left of IS
internal API
end note
activate IS
IS->D: store the user account
activate D
D->IS: ok
deactivate D
IS->UT: AMQP POST userCreated message
activate UT
UT->IS: Message posted confirmation
deactivate UT
IS->AG: 201 created
deactivate IS
AG->AC: 201 created
deactivate AG
AC->A: user created\nconfirmation message
deactivate AC
A->U: Account created\nconfirmation email
@enduml
@enduml

Binary file not shown.

Before

Width:  |  Height:  |  Size: 122 KiB

View File

@@ -1,152 +0,0 @@
@startuml
Title: Interaction between Identity Service, Proxy, Activiti and Alfresco
participant "Browser" as B
participant "Identity Service" as IS
participant "Proxy" as P
participant "Alfresco Repository" as R
participant "Activiti" as A
== flow with Proxy ==
B->P: HTTP GET /something from Alfresco
activate B
activate P
note right of P
sees no authorization
header
end note
P->B: redirect
deactivate P
B->IS: authenticate
activate IS
B<-IS: login form
deactivate IS
note right of B
user enters
userid and
password
end note
note right of B
Assumption: common
password for Activiti
and Alfresco (such
as would be provided if
they shared an LDAP for
example (not pictured))
end note
B->IS: credentials
IS->R: credentials
activate R
R->R:checks credentials
R->R: generate alf_ticket
R->IS: alf_ticket
deactivate R
IS->IS: generate common_token
deactivate R
IS->IS:store common_token
IS->IS: store alf_ticket
note right of IS
common_token is
a key to retrieve
alf_ticket and
act_ticket
end note
IS->A: credentials
activate A
A->A:checks credentials
A->A: generate act_ticket
A->IS: act_ticket
deactivate A
IS->IS:store act_ticket
IS->B: redirect w/common_token
B->P: HTTP GET /something from Alfresco (with common_token)
activate P
note right of P
sees common_token in
authorization header
end note
P->IS:validate common_token and retrieve alf_ticket
activate IS
IS->P: alf_ticket
deactivate IS
P->R:HTTP GET /something (with alf_ticket)
R->R: validate alf_ticket
R->R: checks authorization of\nuser to something
R->P: something from Alfresco
deactivate R
P->B:something from Alfresco
deactivate P
B->P : HTTP GET something from Activiti (with common_token)
activate P
note right of P
sees common_token in
authorization header
end note
P->IS:validate common_token and retrieve act_ticket
activate IS
IS->P: act_ticket
deactivate IS
P->A:HTTP GET /something (with act_ticket)
A->A: validate act_ticket
A->A: checks authorization of\nuser to something
A->P: something from Activiti
deactivate A
P->B:something from Activiti
deactivate P
deactivate B
== flow without Proxy ==
B->R: HTTP GET /something from Alfresco
activate R
note right of R
sees no authorization
header
end note
R->B: redirect
deactivate R
B->IS: authenticate
activate IS
B<-IS: login form
deactivate IS
note right of B
user enters
userid and
password
end note
B->IS: credentials
IS->IS:checks credentials
IS->IS: generate common_token
IS->B:common_token
deactivate IS
B->R: HTTP GET /something from Alfresco (with common_token)
activate R
note right of R
sees common_token in
authorization header
end note
R->IS:validate common_token
activate IS
IS->R: OK
deactivate IS
R->R: checks authorization of\nuser to something
R->B: something from Alfresco
deactivate R
B->R: HTTP GET /something from Activiti (with common_token)
activate A
note right of A
sees common_token in
authorization header
end note
A->IS:validate common_token
activate IS
IS->A: OK
deactivate IS
A->A: checks authorization of\nuser to something
A->B: something from Activiti
deactivate A
deactivate B
@enduml

Binary file not shown.

Before

Width:  |  Height:  |  Size: 96 KiB

View File

@@ -1,110 +0,0 @@
@startuml
@startuml
Title: OAuth Flow
actor "End User" as U
participant "Alfresco App" as ALF
participant "External System" as EXT
participant "OAuth Gateway Component" as OG
participant "API Gateway" as AG
participant "Identity Service" as IS
participant "User Topic" as UT
database "Directory\nNoSQL" as D
U->ALF: Login
activate U
ALF->AG: HTTP authn user (URL?)
activate AG
note right of AG
public API
end note
AG->IS: HTTP authn user (URL?)
note right of IS
internal API
end note
activate IS
IS->D: Retrieve user account
activate D
D->IS: Return user and validate creds
deactivate D
IS->UT: AMQP POST userLogin message
activate UT
UT->IS: Message posted confirmation
deactivate UT
IS->AG: Return token
deactivate IS
AG->ALF: Return token
deactivate AG
ALF->U: Render Page
deactivate U
U->ALF: Authorize external system
activate U
ALF->OG: Authorize external system
activate OG
OG->EXT: Redirect browser
activate EXT
EXT->U: External Login / App Auth Page
U->EXT: Login / Approve App
EXT->OG: Redirect w/ authz code
OG->EXT: Get OAuth tokens
EXT->OG: Return OAuth tokens
deactivate EXT
OG->AG: HTTP POST\n/users/{id}/identities/{extsysid}/tokens
activate AG
note right of AG
public API
end note
AG->IS: HTTP POST\n/users/{id}/identities/{extsysid}/tokens
note right of IS
internal API
end note
activate IS
IS->D: Store OAuth tokens
activate D
D->IS: ok
deactivate D
IS->UT: AMQP POST userTokenStored message
activate UT
UT->IS: Message posted confirmation
deactivate UT
IS->AG: 201 created
deactivate IS
AG->OG: 201 created
deactivate AG
OG->U: Success Page
deactivate OG
deactivate U
ALF->AG: HTTP GET\n/users/{id}/identities/{extsysid}/tokens
activate ALF
activate AG
note right of AG
public API
end note
AG->IS: HTTP GET\n/users/{id}/identities/{extsysid}/tokens
note right of IS
internal API
end note
activate IS
IS->D: Retrieve OAuth tokens
activate D
D->IS: Return OAuth tokens
deactivate D
IS->AG: Return OAuth tokens
deactivate IS
AG->ALF: Return OAuth tokens
deactivate AG
ALF->EXT: API call w/ OAuth tokens
activate EXT
EXT->ALF: Return external data
deactivate EXT
deactivate ALF
@enduml
@enduml

Binary file not shown.

Before

Width:  |  Height:  |  Size: 68 KiB

View File

@@ -1,97 +0,0 @@
@startuml
Title: Interaction between Identity Service and SAML
participant "SAML IdP" as S
participant "Identity Service" as IS
participant "Browser" as B
participant "Repository" as R
== flow without SAML ==
B->R: HTTP GET /something
activate B
activate R
note right of R
sees no authorization
header
end note
R->B: redirect
deactivate R
B->IS: authenticate
activate IS
B<-IS: login form
note right of B
user enters
userid and
password
end note
B->IS: credentials
IS->IS: check credentials
IS->B: redirect w/token
deactivate IS
B->R: HTTP GET /something (with token)
activate R
note right of R
sees token in
authorization header
end note
R->IS: validate token
activate IS
IS->R: OK
deactivate IS
R->R: extracts userid from token
R->R: checks authorization of\nuser to something
R->B: something
deactivate R
deactivate B
== flow with SAML ==
B->R: HTTP GET /something
activate B
activate R
note right of R
sees no authorization
header
end note
R->B: redirect
deactivate R
B->IS: authenticate
activate IS
IS->S: get assertion
deactivate IS
activate S
S->B: login form
deactivate S
note right of B
user enters
userid and
password
end note
S<-B: credentials
activate S
S<->S: check credentials
S->S: generate assertion
S->B: assertion w/redirect
deactivate S
B->IS:get token
activate IS
IS->IS: extract user from assertion
IS->IS: generate token
IS->B: redirect w/token
deactivate IS
B->R: HTTP GET /something (with token)
activate R
note right of R
sees token in
authorization header
end note
R->IS: validate token
IS->R: OK
R->R: extracts userid from token
R->R: checks authorization of\nuser to something
R->B: something
deactivate R
deactivate B
@enduml

View File

@@ -1,244 +0,0 @@
## Queue
![Completeness Badge](https://img.shields.io/badge/Document_Level-In_Progress-yellow.svg?style=flat-square)
![Version Badge](https://img.shields.io/badge/Version-Future-blue.svg?style=flat-square)
### Purpose
The Queue Component is designed to meet the needs to asynchronous operations.
***
### Overview
The Queue Component manages a set of **Queues** and **Topics** that enable Alfresco components to communicate with each other asynchronously using Messages.
***
### Artifacts and Guidance
* Source Code Link: Not yet known
* License: Not yet known
* Issue Tracker Link: Not yet known
* Documentation Link: Not yet known
* Contribution Model: undetermined
***
### Prerequisite Knowledge
* [Apache Camel](http://camel.apache.org/)
* [Java Message Service](http://docs.oracle.com/javaee/6/tutorial/doc/bncdq.html)
* [AMQP](http://www.amqp.org/)
***
### Design
#### Technology Options
##### JMS and AMQP
There are two alternatives for client interactions with the Queue:
[JMS](http://docs.oracle.com/javaee/6/tutorial/doc/bncdq.html) and [AMQP](https://www.amqp.org/) .
JMS standardizes access at the API layer, thus making it easy to develop Java messaging clients.
AMQP standardizes the wire protocol, making it possible to develop clients in other programming languages.
The design is assuming the use of JMS by the Alfresco services for the following reasons:
1. Wider Adoption among brokers, in particular by AWS SNS should we decide to support that broker
2. Simpler client model for Java, which is an advantage since most of Alfresco is written in Java
However, ActiveMQ supports both JMS access and AMQP. So, when ActiveMQ is used as the message broker,
a client can use AMQP or JMS. This allows a wider range of clients to be developed, such as C and Python clients.
The [Apache Messaging Toolkit](http:/qpid.apache.org/proton/) provides a useful set of libraries that make it easy to
develop such clients,
##### Message Broker
There are a number of alternative implementations available for the Message Broker. These include ActiveMQ, Rabbit MQ, AWS SNS and others. This design is assuming ActiveMQ for the following reasons:
1. It has the most capability
2. Friendly License
3. We have the most knowledge of the technology
See the article [Distributed Queues](https://ts.alfresco.com/share/page/site/eng/wiki-page?title=Distributed_Queues) for a more complete discussion.
#### Delivery Mode
All Messages are sent as *persistent*. This means that the broker will assure that it survives the life of the broker by writing the message to disk (See *Security Considerations*).
#### Subscription Policy
Subscriptions to Topics are typically *durable* to assure that Message Consumers receive all messages, even when the Message Consumer is not active.
But no enforcement of that Subcription policy is done by this component.
#### Embedded Message Brokers
In addition to using this component for messaging, other components may embed a Message Broker in order to manage internal messages. These internal brokers are not part of the responsibility of this component.
#### Component Model
![Components](./resource/component/queue-components.png)
#### Data Model
##### Queues
*Queues* are used to send a Message from one Message Producer to one Message Consumer. Thus, they enable a *point-to-point* style of communication. This design does not use *Queues* (despite the component name).
##### Topics
*Topics* are used to send a Message from one Message Producer to multiple Message Consumers. A Message Producer writes the Message to a Topic. Message Consumers, which subscribe to the *Topic*, receive the Messages. Thus, *Topics* enable a *pub/sub* style of communication. This design assumes messages send via *Topics*
Topics are organized in the following hierarchy.
* System (alf)
* Content (alf.content)
* Node (alf.content.node)
* File (alf.content.file)
* Folder (alf.content.folder)
* Site (alf.content.site)
* Comment (alf.content.comment)
* Rating (alf.content.rating)
* Rendition (alf.content.rendition)
* Identity
* User (alf.identity.user)
* Group (alf.identity.group)
* System (alf.identity.system)
* Model (alf.model)
* Tag (alf.tag)
* Job (alf.job)
* Preference (alf.preference)
* Workflow (alf.workflow)
* Process (alf.workflow.process)
* Task (alf.workflow.task)
This will allow Message Consumers to be selective when deciding what Messages to listen for. For example, the Audit component can subscribe to all changes in the System by subcribing to **alf.*** whereas the Transform component may only subscribe to **alf.content.file.*** messages
##### Messages
###### Format
The Structure of Messages adhere to the *Alfresco Message Style Guidelines*.
![Design Note](https://img.shields.io/badge/Design-Note-orange.svg?style=flat-square) We need to define where that Guideline document is.
Each Message consists two parts:
* Header
* Body
![Message Structure](./resource/component/message-format.png)
###### Header
The header contains system information common to all Messages, such as the time it was sent and the kind of Message it is. Additionally, the header contains the following Alfresco-specific information, stored as keyword/value properties [**Note:** Some of this information will be provided in the *Alfresco Message Style Guidelines*.
* **alf.version**: The Version of the Message in x.y format where x is a major version and y is a minor version.
* **alf.sequenceNumber**: An integer generated by the component that can be used to order messages
###### Body
The body contains the Message payload specified as a JSON object. In a JMS client, that body is of type *javax.jms.TextMessage*. On the wire (AMQP) it is an *AmqpValue* holding a String.
#### Data Dictionary
Not Applicable
### Flows
#### Example of a message created by one service and used by two others
![Model Created Message Flow](./resource/sequence/model-created-flow.png)
#### Example of how a mix of AMQP and JMS clients can access a message topic
![Mixed JMS and AMQP Message Flow](./resource/sequence/jms-amqp-client.png)
#### Class Diagram
***
### APIs and Interfaces
#### Message Registry
The Message Registry is defined in a yaml file that conforms to the [OpenAPI Specification](https://github.com/OAI/OpenAPI-Specification).
See [Alfresco Message Explorer](https://github.com/Alfresco/message-explorer/blob/master/README.md)
##### REST API
![Design Note](https://img.shields.io/badge/Design-Note-orange.svg?style=flat-square)
We will use Swagger for specifying the REST API. For now, here are the services:
* **Get Topics**
* **Register Topic**
<pre>
POST /topics
{
"endpointName": "alf.content.file.v1.create",
"endpointType": "topic"
"messageSchema": "org.alfresco.message.file.v1.FileCreate"
}
</pre>
![Design](https://img.shields.io/badge/Design-TODO-orange.svg?style=flat-square) Do we need this REST API at all?
* rgauss: I believe in the current design consumers of messages are talking directly to the broker. Those consumers must also have been coded to know how to deal with the message schema of the endpoint they wish to listen to. Do we expect these topic names to change per deployment? If not, then I think they should be treated much like a REST URL in that it's helpful for developers to have documentation in the form of a topic explorer to learn about them, but that may not require that various components register their endpoints via another REST API.
* rgauss: If we do need a method of registering endpoints, we've also discussed the possibility of allowing that registration to happen via messaging rather than REST. A microservice could periodically send a type of heartbeat message containing its endpoint data to a known topic which the queue service could consume. This has a few advantages:
* The queue service would not be required to persist the list of endpoints. After queue service restart it would just gather the heartbeat data again and the microservices would not have to be notified of a queue service restart, asking them to re-register.
* Eliminates the need for a 'de-register' endpoint as the endpoint data can be expired if it's not seen in heartbeat messages after some time.
#### Client API
##### Java
[JMS provides the Java API](http://docs.oracle.com/javaee/1.4/api/javax/jms/package-summary.html)
##### Python
[Apache Messaging Toolkit - Python](http://qpid.apache.org/releases/qpid-proton-0.14.0/proton/python/api/index.html)
##### C/C++
[Apache Messaging Toolkit - C API](http://qpid.apache.org/releases/qpid-proton-0.14.0/proton/c/api/files.html)
![Design Note](https://img.shields.io/badge/Design-Note-orange.svg?style=flat-square)
More API Bindings are available, if the broker supports AMQP.
***
### Configuration
***
### Performance Considerations
![Design Note](https://img.shields.io/badge/Design-Note-orange.svg?style=flat-square) Add a link to a list of Requirement Specifications that describe the key performance goals for this component.
***
### Security Considerations
* Authentication
* Nothing should be made available without authentication. Message Producer and Consumers must be authenticated. Additionally, each Queue and Topic has an access control policy that controls who can read, write and delete messages. All stored
passwords will be stored in an encrypted manner (http://activemq.apache.org/encrypted-passwords.html).
* Authorisation
* The general principle here is that of least privilege. Each Queue and Topic has an access control policy that controls who can read, write and delete messages.
* Encryption
* Queue communications should be encrypted throughout. Additionally certificate based authentication also brings extra security gains (http://activemq.apache.org/how-do-i-use-ssl.html).
* Access
* Since Messages are sent as persisted, the Broker will write them to disk to assure they will survive a Broker crash.
Care must be taken to assure the disks the Message Broker uses to persist the messages are not accessible by unauthorized users.
***
### Cloud Considerations
An important design topic will be to decide what queue provider to support when the component is deployed to Cloud. When deploying to AWS, consideration should be given to using **SNS** and **SQS**.
***
### Design Decisions
| Decision | Rationale | Date |
| :----------------|:--------------------------| ------------:|
| Use JSON for the message body, not Java serialization| (a) Simpler for Engineers to understand <br>(b) more portable across multiple client languages | 22 Sept, 2016 |
| Clients will subscribe directly to the topics to listen for changes as opposed to providing an endpoint for Alfresco to write to| (a) Simpler for clients since they don't need to create an endpoint <br>(b) more scalable because Alfresco is not a potential bottleneck in needing to post events to multiple endpoints | 03 Oct, 2016 |
| Spring Boot will be used to build the Container | See https://issues.alfresco.com/jira/browse/TAG-11 and https://issues.alfresco.com/jira/browse/TAG-12 | 03 Oct, 2016 |
| Each topic will carry messsages pertaining to a single data type (e.g. User Object) | Simpler for tooling to serialize/deserialize objects from a topic (e.g. using the "@class" annotation) | 10 Oct, 2016 |
| Each topic will carry messsages pertaining to a version of the message format | Simpler for message consumers to only consume messages of a particular version | 10 Oct, 2016 |
| Each message will also carry a version id, nothwithstanding the fact that the topic should only carry messages of a particular type and version | We cannot be certain that message producers will be completely reliable | 10 Oct, 2016 |
| It will be possible for a microservice to register a Topic to the Queue service | (a)Allows a new microservice to be introduced dynamically<br>(b)Eliminates need for microservices to depend on a particular version of the Queue Service | 10 Oct, 2016 |
| The version will be encoded in the package name, e.g. **org.alfresco.message.file.v1.FileCreate** and in the header (to allow filtering)| | 13 Oct, 2016 |
***
### Quiz

Binary file not shown.

Before

Width:  |  Height:  |  Size: 8.5 KiB

View File

@@ -1,11 +0,0 @@
@startuml
entity "Object" as O
frame "AMQP Message" as M {
frame header as H
frame "body (JSON)" as B
}
O <.--.> B
@enduml

Binary file not shown.

Before

Width:  |  Height:  |  Size: 46 KiB

View File

@@ -1,42 +0,0 @@
@startuml
title Components Sharing a Model
left to right direction
node "Message Broker" as MB {
queue "Model Topic\nEndpoint" as MTE #white
}
database "NoSQL\n(Schema Data)" as DB1
database "NoSQL\n(Metadata)" as DB2
database "Solr 6.x" as Solr
node "API Gateway" as AG
node "Model\nService" as MS {
node "REST API" as API
node "Message Producer" as MP
}
node "Search Service" as SS {
node "Message Consumer" as L1
}
node "Repository\nService" as R {
node "Message Consumer" as L2
}
node "Model Manager App" as MMA
actor "user" as U
MS <-> DB1 : read/write model
MP -> MTE : model\nchanged message
R <-> DB2 : read/write metadata
SS <-> Solr : read/write
L1 <- MTE : model\nchanged message
L2 <- MTE : model\nchanged message
MMA -> AG : get/put model
AG -> API : get/put model
U <-> MMA : design model
@enduml

Binary file not shown.

Before

Width:  |  Height:  |  Size: 35 KiB

View File

@@ -1,41 +0,0 @@
@startuml
Title Queue Components
left to right direction
actor "Administrator" as A
node "Queue Service" as Q {
node "REST API" as R1 #white
node "Spring Boot" as SB #white
}
node "Message Broker" as MB {
queue "Topic\nEndpoint" as R2 #white
}
node "Message Producer" as MP
node "Message Consumer" as MC1
node "Message Consumer" as MC2
node "Message Consumer" as MC3
node "Message Consumer" as MC4
A-->SB: starts/stops
A-->R1: configures
Q-->R2: starts/stops broker
Q-->R2: configures
MP<--R1: get Topics
MP-->R2: sends message to Topic
R2-->MC1: gets message
R2-->MC2: gets message
R2-->MC3: gets message
R2-->MC4: gets message
R2<--MC1: subscribes
R2<--MC2: subscribes
R2<--MC3: subscribes
R2<--MC4: subscribes
'R2-T
@enduml

Binary file not shown.

Before

Width:  |  Height:  |  Size: 638 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.2 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.2 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.3 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.3 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.2 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 53 KiB

View File

@@ -1,52 +0,0 @@
@startuml
Title: Mix of AMQP and JMS Messaging Clients
actor "User\n" as U
participant "Python App\n(Apache Qpid App)" as PA
participant "Simple File Share" as SFS
participant "API Gateway" as AG
participant "Repository Service" as RS
participant "File Topic" as FT
database "Content Repository" as CR
autonumber
PA->FT: Subscribe to Topic
U->SFS: Uploads a File
activate SFS
SFS->AG: HTTP POST\nFile
activate AG
AG->RS: HTTP POST\nStore file
activate RS
RS->CR: Store file
activate CR
CR->RS: File stored
deactivate CR
RS->FT: Write message (JMS)
activate FT
note right of FT
alf-file-upload-message
end note
FT->RS: Message written
deactivate FT
RS->AG: 200 Content uploaded
deactivate RS
AG->SFS: 200 Content created
deactivate AG
SFS->U: File uploaded
deactivate SFS
group Message Processed\nBy Python App
autonumber 8
PA->FT: Read Message (AMQP)
note right of FT
alf-file-upload-message
end note
activate FT
FT->PA: Message content
deactivate FT
PA->PA: Process message
PA->FT: wait on next message (AMQP)
end
@enduml

Binary file not shown.

Before

Width:  |  Height:  |  Size: 76 KiB

View File

@@ -1,77 +0,0 @@
@startuml
Title: Flow for Model Creation Message
actor "User\n" as U
participant "Model Manager App" as MMA
participant "API Gateway" as AG
participant "Search Service" as SS
participant "Model Service" as MS
participant "Repository Service" as RS
participant "Model Topic" as MT
database "Solr Index" as SI
database "Content Repository" as CR
database "Model Database" as MD
autonumber
U->MMA: Creates a New Content Model
activate MMA
MMA->AG: HTTP POST\nStore model
activate AG
AG->MS: HTTP POST\nStore model
activate MS
MS->MD: Store model
activate MD
note right of MD
NoSQL Database
end note
MD->MS: Model stored
deactivate MD
MS->MT: Write message
activate MT
note right of MT
alf-model-created-message
end note
MT->MS: Message written
deactivate MT
MS->AG: 201 Content created
deactivate MS
AG->MMA: 201 Content created
deactivate AG
MMA->U: Model created
deactivate MMA
group Message Processed\nBy Repository Service
autonumber 7
RS->MT: Read message
activate MT
note right of MT
alf-model-created-message
end note
MT->RS: Message content
deactivate MT
RS->RS: Process model
RS->CR: Store model
activate CR
CR->RS: Model stored
deactivate CR
end
group Message Processed\nBy Search Service
autonumber 7
SS->MT: Read message
activate MT
note right of MT
alf-model-created-message
end note
MT->SS: Message content
deactivate MT
SS->SS: Process model
SS->SI: Store model
activate SI
SI->SS: Model stored
deactivate SI
end
@enduml