SireAdvanced Molecular Simulation Framework

The bulk of Sire is now nearly complete, so I am now looking to the future of where the code will go next.
There are three main considerations;

(1) How can I make the code easier to install and maintain?
(2) How can I make the code easier to develop for new developers?
(3) What is the next major challenge that Sire needs to solve?

To answer points 1 or 2 I have been rewriting the underlying foundation of Sire so that
it has fewer dependencies and so that the object design is easy to understand and
completely consistent. The result is Siren - the Sire new object system. Siren
has been influenced by what I learned when I played with Java. I liked the
clean object design in Java, with a clear separation between implementations
and interfaces. I also liked the metaclass system and the single base class
for the entire hierarchy (which makes casting easier). However, the Java
object system didn't fit fully with the objects I use in Sire. The main
problem was that Sire has a clean distinction between const, read-only objects
(which are implicitly shared), read-write objects (which are also implicitly shared),
and read-write objects that are explicitly shared. In addition, using a virtual
class hierarchy for some classes is not efficient, both for space and
compute, e.g. numbers, vectors, spheres etc. To solve this I ended up creating
a class system with four bases;

(i) Primitive

These are non-virtual classes that are space and compute sensitive. These objects
must be movable in memory (e.g. memcpy will work). You create a primitive
by deriving from template<class T> Primitive<T>, e.g.

Note that Primitive<T> uses the curiously recurring template pattern so that
it can provide compile-time polymorphism.

(ii) Object

These are virtual classes which do not hold shared data (i.e. they are stand-alone
or implicitly shared). Because these classes are not shared, and do not hold
shared data, you can guarantee that your copy belongs just to you, and that
all of the functions are re-entrant (and accessing your copy is thus thread-safe).
By default classes derived from Object are const, meaning that all functions
are constant. You create an Object class by deriving from Extends or
Implements. Extends is to create a virtual class, while Implements creates
a concrete class. For example, Space is a virtual class representing simulation
spaces, while Cartesian is a concrete class that represents an infinite
cartesian space. These are derived using;

Again, the curiously recurring template pattern is used so that Extends and
Implements can implement functions needed to automatically fill in the
Object API.

(iii) Handle

These are classes than hold (handle) shared data, e.g. file handles,
handles to loggers, handles to compute nodes etc. These classes
are explicitly shared (obviously!), and contain functions that
allow the resource to be locked (via a mutex). Handles are read/write.
If we have a resource called MyResource, then we declare MyHandle to
hold that resource using ExtendsHandle or ImplementsHandle (where
Extends and Implements have the same meaning as for Object). To
declare MyHandle we would write;

(iv) Interface

Finally, there are cases where multiple inheritance is needed. However, I like
that Java only allows multiple inheritance using interfaces (pure virtual
classes with no member data). I also want to make that restriction, as it
makes multiple inheritance easier to use, and makes the programmers intention
clearer. To achieve this, I have created an Interface class, and use
Interfaces as a means of adding interfaces to an Object or Handle. One
such interface is Mutable, which provides the code needed to allow
Object classes become non-const (become mutable). For example,
Accumulator is a class that accumulates observations, e.g. it forms
that base class of Average and Median. It would be inefficient to implement
this as a read-only class, so it needs to be read/write. This means that
it is attached to the Mutable interface, via;

Having four types of objects can be a problem if the code has to
store or query objects of any type. To make things easier, there are
helper classes that allow easy and automatic conversion of Primitive
to Object (via PrimitiveObject<Primitive> classes) and to allow
objects to be held by a Handle. If a Handle can be detached, then
the Handle can also be held in a HandleObject (which changes
the explicit sharing to implicit sharing).

Each of the three primary class types (Primitive, Object and Handle)
have the same base API. The key functions are;

=== clone() ===

Return a reference (ObjRef or HanRef) to a clone of the object

=== toString() ===

Return a localised string that describes the object

=== test(), test(Logger &logger) ===

Runs a unit test of this object, optionally writing the results
to the passed Logger object. This function can be called by anyone,
and is also used by the program "sirentest" that runs the unit
tests of all public objects of a library.

=== copy(const Object&), copy(const Handle&) ===

Copy the passed object or handle into this object

=== equals(const Object&), equals(const Handle&) ===

Return whether the passed object or handle equals this object

=== hashCode() ===

Return a hash code for this object, suitable to allow this object
to be held in a hash or set

=== load(Stream&), save(Stream&), stream(Stream&) ===

Load, save or stream (load or save based on the stream) this object
to the passed Stream. A Stream is a virtual class that allows the
object to be saved or restored. Examples include DataStream
that saves the object in a platform and endian independent
binary format, and XMLStream, that saves the objects to the
Siren XML format.

The stream(Stream&) function is used for both loading and saving, and is quite
easy to implement, e.g.

=== isA<T>(), asA<T>() ===

Used for querying and casting, e.g. obj.isA<Space>() will
return whether or not obj is derived from Space, while
obj.asA<Cartesian>() will return obj cast as a Cartesian
(raising an invalid_cast exception if this can not be achieved).

=== getClass() ===

Finally, getClass() returns a Class object, which provides information
about the class of this object. A Class object provides functions like;

super() : get the Class representing the super class of this class
implements<T>() : return whether this class implements type T
isConcrete() : return whether this is a concrete class
interfaces() : return all of the interfaces for this class
name() : return the fully qualified typename of this class
createObject() : create a default constructed object of this class
createHandle() : create a default constructed Handle of this class

Class, together with getClass(), provides a full metatype system
for Sire, which replaces the many ad-hoc metatype systems that
evolved while Sire was written. In particular, this allows
for me to replace the old SireBase::Property class, which
provides something similar.

Siren provides a coherent foundation for Sire, making it easier
to see how a class is used based on how it is derived (e.g. is
it a Handle or Object, what does it Interface etc.). In addition,
Siren provides the basis for full unit testing by forcing the
creating of unit tests for all classes inline in the source,
and by providing a streaming API that has finally allowed me to
implement the XML streamer, and provides space to implement
other streamers without having to change the code. Finally,
Siren is completely independent of Sire and could be separated
out and used by anyone. I've ported about a third of Sire
across to Sire now (everything up to SireCluster). Once
it has finished I will have created what will effectively
be version 3 of Sire (version 0.5 was used for the SC07 demo
on the ClearSpeed supercomputer, version 1.0 was used for
my JCP paper and Katie's initial work, and version 2.0 was
the one with the new replica design, and which is fully working
now).

Finally (finally!) the eagle-eyed will wonder about my answer
to point (3);

What is the next major challenge that Sire needs to solve?

For the answer to that, I have one word; Inspire...