Smart Memory Object-Oriented Database
SMOOD is a system configuration data and model persistence implementation.
On this page
General
This document describes SMOOD itself, and then various IncrementalAccess
implementations based on SMOOD - Distributed Collaborative Smood Access (DCSA) and Collaborative Smood Access (CSA).
This means all these implementations use SMOOD as its query/apply manipulation engine/handler, and they only differ in terms of how they persist the actual data.
Let's now have a look at SMOOD itself, and then at the different access implementations.
SMOOD
SMOOD (Smart in-Memory Object Oriented Database) is a data structure holding a collection of entities (called population
), supporting queries and updates over this population. Indices are also supported, with id
and globalId
properties being indexed automatically, and indexing of other properties being configurable via metadata.
The main advantages of SMOOD are:
- no special infrastructure needed, it's all in-memory
- support for any model, regardless of its complexity (unlike e.g.
HibernateAccess
, where complex hierarchies with multiple inheritance or properties of typeGenericEntity
cannot be mapped).
This modeling flexibility is the main reason why we use SMOOD-based accesses for our core accesses, containing the configuration of our system.
SmoodAccess
This is the most simple and straightforward SMOOD-based access. Its persistence consists of a NonIncrementalAccess
, which is a persistence that stores and loads all the data at once. You cannot target an exact property for updating, nor can you evaluate a query against it.
For more information, see NonIncrementalAccess javadoc
So the lifecycle of the SMOOD access is that on startup, it loads the population from the NonIncrementalAccess
into SMOOD, and then uses it to evaluate queries. When it comes to write operations, SmoodAccess
uses an extra layer for storing manipulations, called a manipulation buffer, which appends applied manipulations, one after another.
When the threshold size of the buffer is reached, the entire population is persisted with the NonIncrementalAccess
, and the buffer is purged.
For more information, see ManipulationStorage javadoc
The standard NonIncrementalAccess
implementation is the XmlAccess
, which simply stores its data in an XML file on the file system.
For more information, see XmlAccess javadoc
CollaborativeSmoodAccess
In case of a collaborative SMOOD access, the persistence consists of stages, where each stage represents certain manipulations.
From this perspective it is similar to the distributed SMOOD access in that it doesn't consist of the data, but of incremental changes on how to build the data. It is, however, even more complex, because some stages might be read-only, not backed by persisted manipulations, but e.g. by Java code (or some script) performing the changes.
Other than the standard IncrementalAccess
functionality, collaborative SMOOD access also supports service requests for managing stages such as renaming, merging, or retrieving a date from a given stage.
For more information, see CollaborativeSmoodAccess javadoc
For example, a simple collaborative SMOOD access can consist of the following stages:
-
Classpath model:
some.org:my-access-model
Read-only stage which simply finds a given model on a classpath and clones it into the collaborative SMOOD access's SMOOD. -
Script:
groovy-stage
Read-only stage which executes a groovy script found with thegroovy-stage
folder. This script uses a variable called$context
to access the SMOOD's session ($context.getSession()
) and thus be able to create/access data. -
GMML:
trunk
Stage backed by GMML file(s), with newly applied manipulation being appended to the correct manipulation file. For most accesses there is only one file -data.man
, but specifically for cortex there are two of them -model.man
anddata.man
. Themodel
file stores manipulations where model elements are being modified (i.e. where they own the manipulations).
Every setup must end with a GMML stage as that is the one where all new manipulations are written to.
When it comes to actual implementation, collaborative SMOOD access is configured with a list of PersistenceInitializers
and the last stage must be a PersistenceAppender
.
Additionally, there are two ways to configure these stages. The dynamic part is the content of a config.json
file, containing a list of stages of the following types:
ManInitializer
- a GMML-based stage (model.man
/data.man
files)ScriptInitializer
- Groovy script (model.groovy
/data.groovy
files)PluginInitializer
- Initializer resolved via the plugin mechanism
The last entry in the config.json
must always be a ManInitializer
, called trunk
in Tribefire. New manipulations are always appended to this stage.
In addition to config.json
, we have statically configured initializers via Wire
, which can be any custom implementations of the PersistenceInitializer
interface. We have two options here:
- use the initializers that run before the dynamic ones
- use those running afterwards (post-initializers)
Initializer Priming
On top of the above, we have a mechanism to configure the initializer using a runtime property called TRIBEFIRE_MANIPULATION_PRIMING
. The format of the value is a comma-separated list of entries with the following syntax: ${manipulationFile path}[>${accessId}]
. If the optional part (in square brackets) is omitted, cortex
is used as a default accessId
.
Note that you can refer to an environment variable from TRIBEFIRE_MANIPULATION_PRIMING
using the env:VAR_NAME
parameter. Consider the below example, where we change the user password by calling the AUTH_INITIALIZER
variable:
TRIBEFIRE_MANIPULATION_PRIMING="cortex/data.man,workbench/data.man>workbench,env:AUTH_INITIALIZER>auth"
AUTH_INITIALIZER="$0=com.braintribe...User(`ENTER_USER_ID`).password='ENTER_NEW_PASSWORD'"
You can also address not only one particular access (via accessId
) but multiple accesses matching a certain accessId
pattern. You can configure this using the pattern
prefix before the accessId
. Whatever follows the pattern:
is interpreted as a regular expression to be matched against the accessId
. For example, the following expression addresses all accesses matching access.*wb
(e.g.: access.foo.wb
, access.bar.wb
, ...):
TRIBEFIRE_MANIPULATION_PRIMING="common/common.wb.primings/data.man>pattern:access.*wb"
Important Guidelines for Initializer Priming
-
Maniplation files can have only one of the following names:
data.man
ormodel.man
. The nature of the file is determined by its name. -
You must store manipulation files inside the Tribefire's
config
folder. For example, if your configuration directory is underuser/tribefire/config
, you can store your cortex manipulations as follows:tribefire/ config/ csa-priming/ cortex/ data.man model.man cortex-workbench/ data.man
Then, you can configure the
TRIBEFIRE_MANIPULATION_PRIMING
as follows:csa-priming/cortex/model.man,csa-priming/cortex/data.man,csa-priming/cortex-workbench/data.man>cortex.wb
Initializer Pre-priming
While the TRIBEFIRE_MANIPULATION_PRIMING
variable runs the priming after all other initializers, the TRIBEFIRE_MANIPULATION_PRIMING_PREINIT
variable can force a priming to run before the other initializers. This only takes effect for dynamically deployed CSAs (e.g. custom workbench accesses) and does not apply to HardwiredAccesses
(e.g.: cortex
).
DistributedCollaborativeSmoodAccess
Distributed collaborative SMOOD access, just like the distributed SMOOD access, is an implementation designed for a clustered environment, but this one is an extension of a regular collaborative SMOOD access - it stores its data and resources on a file system.
The reason we use this implementation, rather than a distributed SMOOD access, is that we also want to offer the stage management functionality found in collaborative SMOOD access.
When a distributed collaborative SMOOD access implementation is used in a cluster, each node will have an identical copy of a collaborative SMOOD access (aside from temporarily going out of sync), and a so called shared storage accessible by each of the nodes to coordinate the synchronization of the state.
Similar to the distributed SMOOD access, before every read operation we synchronize the local state with the shared state, so apply any operations that have been stored in the shared storage since the last check. In case of write operations, we use a distributed lock, and all local changes are also stored in the shared storage for all the other nodes to apply the next time they are syncing.
The first two implementations of the shared storage are JDBC - and etcd -based.
For more information, see DistributedCollaborativeSmoodAccess javadoc