Appendix: The VO Architecture

The Computing Infrastructure

The server powering the CPS-VO deployment is currently a Dell PowerEdge r515:

2 x 2.1ghz 4170HE AMD Opteron w/6 cores per, for 12 cores
64GB of memory
6.38TB of usable storage, configured as a RAID5 array
6 x gigabit network interfaces
protected by a Vanderbilt ITS firewall service (2 x Checkpoint 4600 configured as a HA pair)
backed up to an LTO tape using the ISIS/Vanderbilt backup service

This server hosts 4 Virtual Machines:

The Development Machine
where various new features and bug fixes are initially integrated together in an environment that is similar to the actual CPS-VO installation
(2 vCPU, 8GB memory, 96GB storage)
The Staging Machine
used for final testing before deploying new code; this is an exact replica of the actual CPS-VO
(6 vCPU, 16GB memory, 532GB storage)
The Production Machine
where http://cps-vo.org actually lives; this is the live site
(6 vCPU, 16GB memory, 532GB storage)
The Big Blue Button Server
hosts the VO's open-source web conferencing system (e.g. desktop sharing, video & voice over ip)
(2 vCPU, 4GB memory, 80GB storage)

The Virtual Organization Software Stack:

The software architecture of the VO comprises a large suite of open-source technologies.

In the following sections, we progressively walk through an overview of each of the layers, proceeding from the lowest systems level software, to the upper application-specific codes.

Virtualization and the VO

The CPS-VO collaboration framework is hosted within a VMWare virtual machine. Presently, we are running VMware ESXi version 5.

VMware, however, is not a strict requirement. There are several open-source, or otherwise free, alternative hypervisors on the market.

By using virtualization techniques the process of creating a clone of the VO is simply a matter of shutting down the virtual machine and copying a few files.

The LAMP Stack

LAMP - Linux, Apache, MySQL & PHP

The operating system driving the VO collaboration framework is Ubuntu, the world's most popular free, open-source, enterprise-level linux server derivative. In addition to the base operating system, the primary systems-level software packages are:

Apache - the world's most adopted open-source web server
MySQL - the world's most popular open-source database
PHP - the pervasive server-side web programming language

The Drupal CMS

The VO collaboration platform is built on top of the open-source Drupal content management platform, which powers millions of websites and applications. It's built, used, and supported by an active and diverse community of people around the world (including a dedicated Drupal Security Team, which is 40 strong). Presently, we use version 6 of the Drupal CMS.

Within any Drupal installation, there are three primary catagories of information that defines the look, feel, and features provided by the site. These categories are code, site configuration, & data.

Code

The code of your installation defines the fundamental behavior and feature sets provided by the site.

The architecture, installation, configuration and maintenance of Drupal-based content management systems is well documented at http://drupal.org, but we summarize the essential ideas here. The Drupal architecture is defined by a series of abstraction layers, along with an extensive and well documented API for extending and modifying default behaviors.

"Effective Web design is driven by the need to balance flexibility and simplicity. If a system is too simple, it can only be used for a single purpose - but if it is too flexible, it may be too difficult for new users to learn.

Drupal strives to reconcile these conflicting goals by providing its users with the tools they need to make their own content management solution, while still providing some pre-built components to help them get started. Thus, it can be described both as a content management system (CMS) and a content management framework (CMF) - one system which strives to have the strengths of both, without their deficiencies.

Most CMS's are like a toy boat or truck - specific assumptions have been made about their use, assumptions that would be hard for you to override. Frameworks, on the other hand, provide you with raw materials only - you need to know a programming language and have a clear design vision to put them together.

Drupal is like a Lego kit. Skilled developers have already made the building blocks - in the form of contributed modules - that you need to create a site that suits your needs, whether that is a news site, an online store, a social network, blog, wiki, or something else altogether."¹

Modules

A fundamental component of the Drupal Framework is the Modules system. Modules are Drupals term for it's plug-n-play extensibility architecture. By writing PHP code that conforms to Drupal's documented API, one may easily extend the core Drupal feature set. At the time of this writing, there are approximately 20,000 freely available, open-source Drupal modules, of which the VO infrastructure relies on approximately 400 modules² that were implemented by the open-source community. Additionally, there are custom modules implemented by the CPS-VO development team to specifically support the CPS-VO project. As dictated by Drupal's open-source contribution policies, customized installation instructions for each individual module are provided in an included readme.txt file.

Themes

Similarly, themes are a mechanism for customizing the user interface related behaviors of a Drupal-based website. The VO collaboration framework includes a single, highly customized theme designed specifically to support the CPS-VO community.

Views

Finally, in addition to writing customized PHP code, Drupal includes the concept of Views which provide a graphical user interface to system administrators for defining highly complex SQL queries that implement custom report types. The VO software infrastructure includes several custom views for generating common, re-usable [typically paramaterized] reports³.

Configuration

In addition to code, which is typically stored within the CPS-VO Subversion open-source version control and configuration management system, any Drupal installation has custom configuration data that is stored within the site's database tables.

Content Types

The first major category of configuration data defines a type system for information that may be posted on the website. As of this writing, the defined content types at http://cps-vo.org are:

Type	Description
Bibliographic Reference	Allows members to add bibliographic references to groups in which they have membership. Biblios can be added manually or imported from BibTex.
Event	Events are special because they have a start date/time and an optional end date/time as a first-class concept.
File	Files allow uploads of documents and other pre-existing content.
Forum topic	A forum topic is the initial post to a new discussion thread within a forum.
Group	The basic unit of organization for adding new virtual communities to the VO.
Group Project	Used for documenting research projects relevant to some Special Interest Group.
NSF CPS Project	Used to build the catalog of NSF CPS research projects.
News Item	News items are more ephemeral forms of information than Pages. They are ideal for creating and displaying content that informs or engages website visitors. Press releases, site announcements, and informal blog-like entries may all be created with a news item entry. News entries may be automatically featured on the site's or on a group's home page.
Online Meeting	A virtual teleconference/desktop sharing session.
Page	A simple web page for creating and displaying information that rarely changes, such as an "About us" section of a website. By default, a page entry does not allow visitor comments and is not featured on the site's initial home page.
Panel	A panel layout broken up into rows and columns. Used for building Dashboards with various content embedded within the rows and columns at specified locations.
Story	Deprecated
Tearline	A content type that differs from web pages in that there are separate sections, each with differing access permissions
User Profile	Used to track basic data and meta-data for individual users (e.g. place of employment).
Video	Used for embedding videos hosted at sites such as youtube or vimeo.
Webform	Create a new form or questionnaire accessible to users. Submission results and statistics are recorded and accessible to privileged users.
Wiki page	A version of web pages that supports hyperlinking by symbolic names, rather than explicit URLs.

Permissions

For each feature implemented on the VO, there is a role-based mapping between capabilities and classes of users that are allowed access to those capabilities. Presently the defined roles are:

anonymous users
authenticated users
trusted users
group moderators
group managers
site moderators
site administrators

Data

Finally, the last component of an instance of a VO is it's data. This includes, for example all of the web pages belonging to a site, as well as the list of validated users that may log onto the site. As with configuration settings, the data within an instance of a Drupal website is stored in a series of database tables.

Users

Every visitor to your site, whether they have an account and log in or visit the site anonymously, is considered a user to Drupal. Each user has a numeric user ID, and non-anonymous users also have a user name and an email address. Other information can also be associated with users.

Taxonomies

Taxonomies are Drupal's mechanism for providing meta-data that classifies site content along categories defined by Vocabularies.

Vocabularies may be either structured and defined by administrative personnel, or free-form and defined dynamically by users.

Nodes

A node in Drupal is the generic term for a piece of content on your web site. A node consists of several fields. The simplest node type is a web page, which contains a title field and a body field. Drupal allows site administrators to build a type system for content, which extends the web page with other, customized fields. For example Event nodes contain from, to, & URL fields in addition to their title and body. Some other examples of nodes are:

Pages in books
Discussion topics in forums
Entries in blogs
News article stories

Comments

Comments are another type of content you can have on your site (if you have enabled the core Comment module). Each comment is a typically small piece of content that a user submits, attached to a particular node. For example, each piece of discussion attached to a particular forum topic node is a comment.

Panels

One rather unique content type defined at the VO is what is termed Panels. Panels provide a drag-n-drop interface for laying out custom homepages or dashboards. Consult Appendix A of The VO User's Manual for an explanation of Panels.

1. Quoted from http://drupal.org/getting-started/before/overview

2. List of module pre-requisites for the VO collaboration framework

3. List of some of the Views defined by the VO collaboration framework

Support Staff and Roles

The VO collaboration and content management system is a large and complex software system consisting of many layers that require configuration and maintenance.

System Administration

Later, in the final chapter, we will discuss the process for an initial installation of a VO, which requires expertise in operating system administration, networking, and database management.

Site Development

Once the basic software is deployed, it is time to install, configure and customize the various modules and themes, which were reviewed in the previous chapter.

Using drupal.org's module search engine, a site developer can quickly find the official homepage of all of Drupal modules that have been published to the open-source community.

Each module comes with specific instructions for its installation and configuration procedures in a bundled README.txt file.

The above figure shows an example README for the node_clone module, which allows one to quickly make copies of existing content.

Once installed, however, it is frequently the case that, while the module is useful, its behavior needs to be extended in some way in order to better meet the needs of the CPS-VO community. In the case of the node_clone module, for example, while it can copy most simple content types, the concept of Groups has been greatly extended from the notion of groups that come with a vanilla, out of the box Drupal installation. Using Drupal's hook system, however, a software developer can define a new module that intercepts certain events generated by the website in order to override or complement the behavior of existing module(s). This is particularly useful for tying together multiple modules, which were developed in isolation and are not aware of one another, so that they behave in a more coherent fashion. The following figure show the info file for a custom CPS-VO module.

This info file declares that the cpsvo_groupclone module is dependent upon three other modules (viz. clone, og_vocab, & og_panels). In addition to the info file, the cpsvo_groupclone module contains a module file that contains PHP code, which defines the module's behavior.

In this example, we intercept Drupal's node_save event for the Group content type, and in addition to cloning the group node itself, we also look up all of that group's taxonomies and its panels from the database and iteratively clone each of those items as well, associating copies of each to the newly cloned group. This gives us the ability to have push-button initialization for a new group, based on previously defined template groups, without the tedious overhead of manually initializing the new group's taxonomies and its services (e.g. discussion forums, group homepage, membership listing, ...).

Site Management

While code developers typically work from the backend, using software development tools such as version control and PHP code editors, there is another class of support staff whose expertise falls along the lines of web design. These site administrators work primarily with Drupal's web-based administrative front-end to maintain basic configuration settings, and with CSS stylesheets and graphics files for maintaing the sites visual look and feel.

Below is an example screenshot of the web-based configuration page for the node_clone module.

Additionally, each module defines a set of permissions for its various features, which the site admin must map to the various roles assigned to each class of users. In the following screenshot, you can see that we restrict the ability to clone a group so that only site administrators have access to this feature.

Site Moderation

The next level of support staff required to maintain an evolving virtual organization is that of the individual group moderators and managers.

Group managers do not have the ability to perform site-wide configuration duties, but they do have full control of their own local group. This includes the ability to edit, delete, or roll back (i.e. to a prevous version) any content that resides within their group. The chapter titled Form & Manage Groups lays out the details for this role.

This covers the four layers of the archetypical support roles. Next, we delve into more details of the systems administration tasks at the lowest layer.

Deploying a VO

Joining the CPS-VO by founding a new group was covered previously in the chapter titled Form & Manage Groups. Here we discuss creating a new Virtual Organization from ground zero.

Setting Up a Development Environment

In the upcoming sections, we discuss options for instantiating a new Virtual Organization, which is based upon the CPS-VO project's open-source codebase. A detailed description of the process for replicating the CPS-VO on a local workstation or laptop to be used for software development is given here.

Multi-Site Installs

Drupal's multi-site features allow one to easily associate a new domain name to the existing CPS-VO infrastructure. Multi-site installations leverage all existing computing hardware, networking infrastructure and system administration staff resources. Additionally, since multi-sites share the same codebase, all future software development and code maintenance for the core CPS-VO modules are shared with sister sites with little additional overhead¹. Given that multi-sites share all resources all the way up to, but stopping short of, their Drupal configuration, the installation process for this mode of operation is far easier.

Deploying an Independent VO

Alternatively, one may instantiate a new virtual organization at a physically separate site, assuming the requisite computational infrastructure and administrative personell are available. In this case, however, a snapshot of the VO codebase is forked off, and there is no automated support for managing future software upgrades. The process for creating a new Virtual Organization is a subset of that of setting up a development environment outlined above, where the difference is that Drupal's configuration and site content will not be replicated from http://cps-vo.org.

1. It is still possible to develop new, customized features on a sister site that is isolated from the core CPS-VO activities.