1. Overview
An ECSA is a service architecture for elearning based webservices. It provides mechanisms for communication and authorization between elearning systems among each other and management systems. This is implemented via a MOM.
An ECSA builds up of three primary components:
-
The ECS (elearning community server) serves the core functionionality of an ECSA network. It provides named message resources to allow communication between all participants.
-
An ECC (elearning community client) is a participant in an ECSA network. It has to be registered at ECS and must be able to talk to the ECS as a REST based client. This participant normally has a native implementation of the ECS interface. Our favourite ECCs are LMSs (learning management systems).
-
An ECP (elearning community proxy) represents a special kind of participant. It serves as a proxy for a none ECSA compliant system so that such a system is able to participate in an ECSA network without ever knowing about it.
1.1. Sample usage scenario
Suppose you have several LMSs (learning management systems) and want to share courses between them. You decide not to interchange the real courses but only course links which consist of some meta data of the appropriate course especially a link formed by an URL pointing to the real course so you can call it through the WWW e.g.:
http://ilias.freeit.de/goto.php?target=pg_26_43&client_id=ecs2
Now it’s possible for each LMS to communicate the released courses by the resources provided from the ECS to an explicit LMS (point to point) or to a community of LMSs (point to multipoint).
Because of the uniform application interface — there are only GET, PUT, DELETE and POST operations — receiving participants can fetch messages through a GET on the resource URL or sending messages by a POST on the resource URL (with some additional query parameters or header variables to point to the appropriate receivers).
To illustrate this we use the simple ECC application curl
to
send a message from one participant to another:
curl -i -H 'X-EcsAuthId: pid01' \ -H 'X-EcsReceiverMemberships: mid02' \ -H 'Content-Type: application/json' \ -X POST \ -d '{ "name": "Mathematics II", "url" : "http://ilias...?target=pg_26_43&client_id=ecs2", ... }' \ http://ecs.freeit.de/campusconnect/courselinks
In order to receive a message (in fifo mode) the receiving participant may call:
curl -i -H 'X-EcsAuthId: pid02' \ -H 'Accept: text/plain; application/json' \ -X GET \ http://ecs.freeit.de/campusconnect/courselinks/fifo
Of course, there are several ways to operate on a resource.
2. Participants
A particpant represents a legal client in an ECSA network.
2.1. Basic functionalities and requirements
2.1.1. Technology / Architecture
2.2. Authorization
A client should be able to use a simple "one touch token" authorization through
the ECS sys/auths
resource. This token could be used to accomplish deligated
authorization for accessing resources on participants of a common ECSA network.
E.g. in redirecting users clicking on course links or in
direct communicating of participants.
2.3. Ressource extensions / alterations
To make resource extensions and alteration possible the clients have to easily permit
-
additional ressources
-
extensible data formats
-
Postels’s Law (robustness principle): Be conservative in what you send; be liberal in what you accept.
-
versioning through request and response header (content negotiation)
-
Accept: application/vnd.my-format.v1+json
-
Accept: application/vnd.my-format.v2+json
-
3. ECS
The elearning community server (ECS) is designed as a message oriented middleware (MOM) and is implemented as a REST conform application.
Because the ECS was born in an elearning context the following definition shows consideration of that. Nevertheless the ECS could be used in other areas of responsibility.
The ECS groups its participants in so called communities. Participants could address each other only if they share a community. Therefore they could address an explicit participant, a participant list or the whole members of the community (see ECS API for details).
All participants have to register at the ECS. Every registered participant has
access to at least three system resources (/sys/memberships
,
/sys/events
, /sys/auths
) to get informed and take part at a ECSA
network. To design/map your specific application communication you can create as many
application resources you want.
3.1. HTTP Header
3.1.1. ECS specific headers
- X-EcsAuthId
-
Has to be a valid participant id. In a standard ECS configuration this HTTP header will be attached by the authentication process running on the proxy server.
- X-EcsReceiverCommunities
-
Has to be a valid community id/ids or community name/names. Adresses all participants joined the comimunity/communities. You are able to note multiple communities, either by name or by id, spaced by comma. Only allowed by POST.
- X-EcsReceiverMemberships
-
Has to be a valid membership id/ids. Adresses all listed memberships. You are able to note multiple memberships spaced by comma. Only allowed by POST.
- X-EcsSender
-
Describes the sender of a message. If you GET a resource this header variable shows the sender membership id. Additionally the ECS sets the X-EcsReceiverCommunities variable to the community from which you have received the message. If the message reach you from several communities X-EcsSender show you a comma separated list of membership ids representing the appropriate membership id of the sender in these communities. In this case the X-EcsReceiverCommunities variable would also represent a comma seperated list of a corresponding community ids.
- X-EcsQueryStrings
-
Used to provide querystrings.
3.1.2. HTTP standard header
- Accept
-
Content-Types that are acceptable.
- Content-Type
-
The mime type of the body of the request (used with POST and PUT requests).
- If-None-Match
-
Allows a 304 Not Modified to be returned if content is unchanged.
- Cookie
-
An HTTP cookie previously sent by the server with Set-Cookie (Wikipedia).
- Content-Type
-
The mime type of this content.
- ETag
-
An identifier for a specific version of a resource.
- Location
-
Used in redirection, or when a new resource has been created.
- Set-Cookie
3.2. HTTP return codes
- 200
-
Successful GET.
- 201
-
Successful POST.
- 304
-
A Not Modified response on a conditional GET. This means the requested resource has not been changed.
- 404
-
Resource not available.
- 4xx
-
General client side error.
- 5xx
-
General server side errors.
3.3. Addressing
In order to communicate to each other you have to provide a unique address. These addresses can either be a so called membership id or a community id or community name.
3.3.1. Membership IDs (mid)
These are unique ids in the scope of an ECS. They establish a relationship between a participant and a community:
pid (participants id), mid (memberships id) and cid (comunities id) are of type integer. |
Therefore a participant can be associated to different communities. Every participant can inquire his membership ids by calling his memberships resource.
3.3.2. Community IDs (cid) and names
A community can be referenced by his community id (cid, type integer) or his community name (type string). If you address a community you implicit address all members of the community. This applies also to the sender joining the receiver community if the sender has set his community_selfrouting flag (default off), otherwise the sender will be implicitly excluded from the receiver list. Every participant can inquire his communities and memberships by reading their memberships resource.
3.3.3. Create a resource
If you want to POST to a resource you have to provide either a X-EcsReceiverMemberships
or
X-EcsReceiverCommunities
header or both together.
If you want to address a single membership or a dedicated number of memberships
you have to set the
X-EcsReceiverMemberships
header. This header can have a
list of values, e.g.
X-EcsReceiverMemberships: 3,6,47
If you want to address a community you have to set the
X-EcsReceiverCommunities
header. This header can have a list of values,
either a community cid
(type integer) or a community name
(type string)
e.g.
X-EcsReceiverCommunities: SWS,23,25
You can get your valid community cid
or name
by reading your
memberships resource
or bei examining the ECS admin interface.
3.3.4. Get a resource
If you GET a resource then the ECS set the X-EcsSender
and the
X-EcsReceiverCommunities
header to show you from whom and where your received
message comes. If there is a list of X-EcsReceiverCommunities
values than
there is also a list of corresponding X-EcsSender
values (mid values, type
integer), i.e. the sending participant is member of multiple communities and
addressed his message to multiple communities also, e.g.
X-EcsSender: 3,19 X-EcsReceiverCommunities: 23,25
This means that this message is addressed to you through two communities
with cid=23
and cid=25
and the sender has the membership mid=3
in
community with cid=23
and membership mid=19
in community with cid=25
.
3.4. Community selfrouting
If community selfrouting is activated at the participant (administration area) you can decide if you also want to receive the message which you send to an appropriate community, i.e. you get an event notification (if events on this resource is activated) and you get it listed by its list resource and could access it through its queue resource. Of course, as sender of the message you can always access it by its message resource.
3.5. Authentication
All participants have to be authenticated in order to use ECS services. A
participant is deemed to be authenticated if the X-EcsAuthId
header is set and
the ECS knows it. The real authentication take place in front of the ECS,
normally at the Webserver. But this depends on configuration/installation of
ECS:
Currently supported authentication methods:
-
Basic Auth
-
X.509 certificates
3.6. Anonymous participants
The creation of a new anonymous participant automatically takes place by every
call to an ECS resource if the calling participant didn’t set X-EcsAuthId
or
Cookie
header, by setting a Set-Cookie
header in the response. On
subsequent calls the participant has to provide this cookie in a Cookie
header in order to be identified as the previously calling participant.
Additionally those participants were automatically joined to the public
community. Further their lifetime will be limited and all resources will be
silently deleted after this lifetime becomes zero. With succesional accesses to
ECS this lifetime will be refreshed. For general cookie handling see
Wikipedia
and
RFC6265.
See also ECS curl
examples.
3.7. System resources
3.7.1. Events
Provides a general queue which accumulates the resource tasks: creation, deletion and renewal. Available representations are application/json and application/xml. It’s recommended to use the events queue to supervise all your possible application specific resources. Further you only have to poll the events queue in order to supervise all your application specific resources and this further take down system load.
Remark: If you wisely decide to use the events queue to supervise your application specific resources you have to manage the validity of events queue yourself, i.e. you shouldn’t additionally poll your application specific resources directly, because then you will get stale events in the events queue.
- /sys/events
-
GET provides a list of events for the appropriate calling participant. Optionally the query string parameter count could be used to limit the amount of returned events.
- /sys/events/fifo
-
GET provides an event (the oldest one) for the appropriate calling participant. Optionally the query string parameter count could be used to extend the amount of returned events. POST provides an event (the oldest one) for the appropriate calling participant and removes it from the events queue. Optionally the query string parameter count could be used to extend the amount of returned events.
Following a sample representation in JSON:
[ { "status": "created", "ressource": "numlab/exercises/7" }, { "status": "destroyed", "ressource": "numlab/exercises/3" } ]
3.7.2. Memberships
Provides information of the affiliation of the calling participant to the
available communities. Available representations are application/json
and
application/xml
.
- /sys/memberships
-
GET provides a list of memberships for the appropriate calling participant. It implies all participants joining an appropriate community including the caller itself.
With the itsyou
key the caller of the /sys/memberships
resource will be
informed which participant in the different communities is assigned to him.
You can fetch your /sys/memberships
resource:
curl ... -X GET https://.../sys/memberships
Following a sample representation in JSON:
[ { "community": { "name": "Hochschulkooperation", "description": "Community zur Unterstützung hochschulübergreifender Zusammenarbeit", "cid": 1 }, "participants": [ { "name": "ILIAS-ECS Client 1", "itsyou": true, "org": { "name": "Uni Stuttgart", "abbr": "S" }, "mid": 1, "pid": 1, "description": "Development participant.", "dns": "n/a", "email": "admin@uni-stuttgart.de" }, { "name": "Freiburg Testparticipant", "itsyou": false, "org": { "name": "Uni Freiburg", "abbr": "FR" }, "mid": 2, "pid": 4, "description": "A general test participant.", "dns": "n/a", "email": "admin@uni-freiburg.de" }, { "name": "ILIAS-ECS Client 2", "itsyou": false, "org": { "name": "Uni Hohenheim", "abbr": "HOH" }, "mid": 3, "pid": 7, "description": "", "dns": "n/a", "email": "admin@uni-hohenheim.de" } ] } ]
3.7.3. Auths
This means authorization through one touch tokens. Provides a mechanism to grant each participant authorization to consume services from any service-providing-participant in an ECS network.
The interface is the same as for
application specific resources.
If you want to create an authorization token, you have to provide at least a
realm
(authorization context) or a url
(authorization context, DEPRECATED):
curl ... -X POST -d '{"realm":"authorization context string"}' https://.../sys/auths
and you will get back something like this:
{ "hash": "5a944e72346e6e3102d32ccfecc18862d23e1dc0", "sov": "2011-03-08T23:25:27+01:00", "eov": "2011-03-08T23:26:27+01:00", "url": "authorization context string", "realm": "authorization context string", "abbr": "LEI", "pid": 35 }
- hash
-
provides the authorization token (one touch token)
- sov
-
stands for start of validation
- eov
-
stands for end of validation
- url
-
provides the authorization context (DEPRECATED)
- realm
-
provides the authorization context (replaces url)
- abbr
-
provides an abbreviation of the participant which has been created the authorization token (DEPRECATED, use pid as reference key in
sys/memberships
representation to get participant information) - pid
-
provides the participant id of the participant which has been created the authorization token
You’re allowed to set the sov
and/or eov
to determine the validity
period of the authorization token. If you do not, the validity period is set to
one minute starting at current time.
The recommended way to fetch an authorization token when knowing the one touch hash:
curl .... -X DELETE https://.../sys/auths/<one touch hash>
This will return the auths representation (same structure/form as when
creating; see above) and delete it server side. If the authorization token is
outtimed, i.e. the current time is not between sov
and eov
, you will
get back a return code 409 (conflict) and following descriptional text in the
body: Authorization token outtimed.
3.8. Application specific resources
All application specific resources have to be configured at ECS. There are three types of application specific resources:
-
messages
-
lists
-
queues
Generally resources are an abtract concept:
-
clearly identifiable (in an HTTP context through URLs)
-
have one ore more representations (e.g. JSON, XML, text, …)
According to resources it plays no role how a representation is produced. It could be done by returning a static file or running a complex server side application, that doesn’t matter. Furthermore by looking at a resource you can’t conclude how the representation has been made. An evaluation of a resource based on internal operations and circumstances, it is thus also negligible, and even be inadmissible.
3.8.1. Resource structure
/<projectnamespace>/<name> /<projectnamespace>/<name>/details /<projectnamespace>/<name>/<id> /<projectnamespace>/<name>/<id>/receivers /<projectnamespace>/<name>/<id>/details /<projectnamespace>/<name>/fifo /<projectnamespace>/<name>/lifo
3.8.2. Message resource
A message resource receives/saves messages for each participant. The participant can fetch (GET) his messages from the resource. A message resource could hold its messages enduringly (see Postrouting), so new participants joining a community after a message has been sent to this community will also receive it.
- GET
-
Returns message with status code 200.
- DELETE
-
Deletes message and returns deleted resource representation with ststus code 200.
- PUT
-
Renew message and returns with status code 200.
- POST
-
Illegal call. Returns with status code 405 (Method Not Allowed).
Resource structure: /<projectnamespace>/<name>/<id>
Subresource details
You can ask for detailed (meta) information of a posted message. Only the original sender or a receiver can do that:
- GET
-
Returns details about the requested message.
Resource structure: /<projectnamespace>/<name>/<id>/details
You will get back something like this:
{ "receivers": [ { "itsyou": false, "mid": 1, "cid": 2 "pid": 19, }, { "itsyou": false, "mid": 4, "cid": 3 "pid": 29, } ], "senders": [ { "mid": 5 }, { "mid": 7 } ], "url": "courselinks/10", "content_type": "application/json" "owner": { "pid": 3, "itsyou": true } }
The "receivers" and "senders" have corresponding arrays: The first array entry in "senders" has been addressed the first array entry of "receivers" and so on.
3.8.3. List resource
- GET
-
Returns URI message list with status code 200. If there are no messages to list the HTTP body will be empty (
Content-Length: 0
). TheContent-Type
will betext/uri-list
. The URI list will be represented by relative references. URIs are specified in RFC3986. - DELETE
-
Illegal call. Returns with status code 405 (Method Not Allowed).
- PUT
-
Illegal call. Returns with status code 405 (Method Not Allowed).
- POST
-
Creates new message, returns with status code 201 and a HTTP header
Location:
providing the new message URI.
Resource structure: /<projectnamespace>/<name>
Subresource details
Now it’s possible to ask for detailed (meta) information of a list resource. All querystrings supported my normal list resources could be used. Only the original sender can do that:
- GET
-
Returns details about all resource URIs listed.
Resource structure: /<projectnamespace>/<name>/details
You will get back something like this:
[ { "senders": [ ], "receivers": [ ], "url": "courselinks/35", "content_type": "text/plain", "owner": { "pid": 3, "itsyou": true } }, { "senders": [ { "mid": 2 } ], "receivers": [ { "mid": 19, "cid": 2, "pid": 19, "itsyou": false } ], "url": "courselinks/36", "content_type": "text/plain", "owner": { "pid": 3, "itsyou": true } }, { "senders": [ { "mid": 2 } ], "receivers": [ { "mid": 19, "cid": 2, "pid": 19, "itsyou": false } ], "url": "courselinks/37", "content_type": "text/plain", "owner": { "pid": 3, "itsyou": true } } ]
The first element of the returned array of the details list subresource probably needs some explanation. Both senders and receivers are empty lists. This means that the appropriate message isn’t any more addressed to any participant. This further implies that all participants which had been addressed in the past have been received the message from their appropriate resource. But why was the message then not deleted ? Because the resource has been configured to be "postrouted". If that has not been the case, ECS would has been removed the message.
Querystrings
To affect the returned representation you could assign the following
querystrings to X-EcsQueryStrings
header variable:
- receiver
-
It’s possible to filter the returned index from a list resource to only those items to which the calling participant was formerly an addressed receiver (this is also the default, therefore it could be omited):
curl .... -H 'X-EcsQueryStrings: receiver=true' -X GET https://server/<namespace>/<name>
- sender
-
It’s possible to filter the returned index from a list resource to only those items to which the calling participant is the original sender:
curl .... -H 'X-EcsQueryStrings: sender=true' -X GET https://server/<namespace>/<name>
- all
-
It’s possible to filter the returned index from a list resource to show all messages either as addressed receiver or as original sender:
curl .... -H 'X-EcsQueryStrings: all=true' -X GET https://server/<namespace>/<name>
Using the X-EcsQueryStrings
header variable is the recommended way to use
querystrings. If you have to assign multiple querystrings please delimit the
querystrings by comma (,).
Of course you can also specify the querystring by appending it to the end of the resource url, e.g.
curl .... -X GET https://server/<namespace>/<name>?all=true
3.8.4. Queue resource
The queue resource is modelled as a subresource of a list resource and it can operate either in lifo (last in first out) or fifo (first in first out) mode.
- GET
-
Returns last (lifo) or first (fifo) message with status code 200. If there are no more messages in queue you will get an empty message (
Content-Length: 0
) and also status code 200. - DELETE
-
Illegal call. Returns with status code 405 (Method Not Allowed).
- POST
-
Returns last (lifo) or first (fifo) message with status code 200 and deletes it. If there are no more messages in queue you will get an empty message (
Content-Length: 0
) and also status code 200. - PUT
-
Illegal call. Returns with status code 405 (Method Not Allowed).
Resource structure: /<projectnamespace>/<name>/fifo
or /<projectnamespace>/<name>/lifo
3.8.5. Postrouting
If a resource has set its postroute flag, then all new participants will get postrouted this resource e.g. if you have posted some messages to a community named test-community and later joins a new participant to this community, he will get postrouted the former posted messages.
3.8.6. Fingerprinting
When a participant tries to create a resource representation fingerprinting allows the ECS to recognize if this representation is already available on ECS and therefor should not be created twice. In this situation the participant will be informed by a HTTP status code 409 and an informational text of "representation already sent".
Activate fingerprinting
Activate fingerprinting by setting the option of the appropriate resource in
Admin
→ Resources
. After that, all resource representations, that will be
created by participants, are subject to fingerprinting.
Deactivate fingerpriting
When fingerprinting is activated by resource
you can overrule this by setting
the deactivate fingerprint option of the appropriate participant in Admin
→
Participants
. After that, all resource representations that will be created
by this participant are not subject to fingerprinting.
Temporarily deactivating fingerprinting
It’s also possible to temporarily switch off activated fingerprinting by the
participant by setting the query string force_creation=true
or adding it to
the header variable X-EcsQueryStrings
:
curl ... -X POST ... https://...?force_creation=true curl ... -H 'X-EcsQueryStrings: force_creation=true' -X POST ...
The decision "new recipients?" evaluates to "no" only when the message for all
recipients has already been created. That means in reverse that if only one
recipient in the receiver list (X-EcsReceiverMemberships ,
X-EcsReceiverCommunities ) hasn’t been received the message in the past the
message will be created for all recipients.
|
3.9. JSON-Schemas
A json media type for describing the structure and meaning of json documents. It’s defined as an Internet-Draft working document of the IETF. There is also a homepage where you can start to discover more over JSON-Schemas.
All resource representations must have a Content-Type
header variable
containing an optional parameter profile
pointing to its describing
schema. For a respond on a /campusconnect/courses
request this could be:
Content-Type: application/json; \ profile=http://repo.or.cz/w/ecs.git/blob_plain/ \ e5cc81b2201ac24294d2ac3e732f9ddac954cc84:/ \ campusconnect/schemas/cc_courses.schema.json
It’s up to you to validate and check the received data against the provided
schema or to decide if you are able to process the format just receiving. There
is always a version id inbetween the profile
URL representing the commit id
of the git repository. For the last Content-Type
example this was
e5cc81b2201ac24294d2ac3e732f9ddac954cc84
. You can always ask for the latest
schema of an appropriate resourse by using HEAD
as the version id.
Of course you can use the schema of an appropriate resource for discovering the names and types of the data elements in order to match them dynamically to other internal meta data of your application.
3.10. Participant Cluster
The ECS is able to cluster participants. In the ECS network a cluster is seen as an ordinary participant.
3.10.1. Cluster building
First lets show the topology of a clustered ECS network:
-
The ECS registers a virtual participant. All cluster participants use this registration, i.e. the ECS doesn’t know which cluster participant is communicating. This way you can scale your cluster easily by attaching another cluster participant also using the previously generated virtual participant registration. You don’t have to make any further settings at ECS.
-
If you want to send a message to the cluster you only have to send it to the virtual participant. When all cluster participants compete against each other to get a message this would maybe the simplest resource access mode (message dispatching). Every cluster participant have to access the appropriate resource as a queue resource via DELETE method. This assures that every message could only be fetched by one cluster participant.
3.10.2. Cluster broadcasting
In order to explicitely communicate with a cluster participant we have to use a broadcasting mechanism. Every resource could be used as a broadcasting resource. It only depends on how the cluster participants access this resource. They have to do it like this:
-
Every cluster participant checks the broadcast resource as a queue resource with the idempotent GET method and decides by looking inside the message if this message is targeted to him. If it does belong to him he should compute and DELETE the message.
-
The ECS garbage collects the broadcast resource at a default time period.
4. Filter plugins
Messages could be changed at runtime by so called filter plugins. These filters could be attached to 5 different queues and triggered by one of the actions hereafter. The filter queues were mapped to a special path under the filesystem:
-
Show filter. Triggered when calling a message/queue resource with GET. Filter path:
filter/<project-name-space>/<resource-name>/show/filter-name>
-
Index filter. Triggered when calling a list resource with GET. Filter path:
filter/<project-name-space>/<resource-name>/index/filter-name>
-
Create filter. Triggered when calling a list resource with POST. Filter path:
filter/<project-name-space>/<resource-name>/create/filter-name>
-
Update filter. Triggered when calling a message/queue resource with PUT. Filter path:
filter/<project-name-space>/<resource-name>/update/filter-name>
-
Delete filter. Triggered when calling a message/queue resource with DELETE. Filter path:
filter/<project-name-space>/<resource-name>/delete/filter-name>
You’re able to create as many filters you want. They will be all queued/concatenated in lexical order:
If a filter was created and copied into the appropriate filesystem path, it would be automatically activated at runtime without additional configuration.
If there are any exceptions while reading (class loading) the filter the appropriate filter will be canceled and the processed message will be queued to the next one. If there are any exceptions while running the filter the appropriate filter will also be canceled and the processed message will also be queued to the next filter, but keep in mind, that all changes to the message prior to the occured exception will remain. You are always working with the original message (no copy). There will be error messages in the logfile of the form: "ERROR Filter Error: …".
4.1. Template
In the <filter-name> directory must be at least a file called "filter.rb" with following structure:
module Filter def self.start ... end end
ECS will call "Filter.start". From there its on you :) ECS will also load any file with ".rb" extention under the directory <filter-name> into the namespace of "Filter".
The ECS core provides the constant FILTER_API
as an API for accessing ECS messages:
- FILTER_API.params
-
It’s a hash to access the qureystrings of message call:
http://ecs.rus.uni-stuttgart.de/numlab/exercises/23?properties=name,description ... elements = FILTER_API.params["elements"].split(",") ...
- FILTER_API.record
-
This object provides access to the message body:
message = FILTER_API.record.body
Glossary
- DRY
-
don’t repeat yourself
- CRUD
-
create, read, update and delete
- ECC
-
elearning community client
- ECP
-
elearning community proxy
- ECS
-
elearning community server
- ECSA
-
elearning community service architecture
- IETF
-
internet engeneering task force
- LMS
-
learning management system
- MOM
-
message orientated middleware