DMLite: How to write a new plugin LOGO

The modular design of DMLite makes easy to write new plugins that can be loaded behind any front-end that uses DMLite behind.
In this tutorial we will develop a very simple plugin that is able to regiter itself and slightly alters the application logic.

For testing purposes you can use the "Get a physical location where to read" example. It doesn't matter if you choose the C, C++ or Python implementation.

Basic skeleton

Implementing a factory

As a first step, we will create a plugin that does nothing, except printing all the configuration parameters it receives, and then ignores them.

We create two files called, for instance, myplugin.h and myplugin.cpp.
As our plugin does nothing, we only need a basic factory that just receives configuration parameters, but does not create anything. For that, we will create a concrete implementation of dmlite::BaseFactory and overload the method configure.

///@file myplugin.h
#ifndef _MY_PLUGIN_H
#define _MY_PLUGIN_H

#include <dmlite/cpp/base.h>

class MyFactory: public dmlite::BaseFactory {
 public:
  MyFactory(void);
  ~MyFactory(void);

  void configure(const std::string& key,
                 const std::string& value) throw (dmlite::DmException);
};

#endif
///@file myplugin.cpp
#include <dmlite/cpp/dmlite.h>
#include <iostream>
#include "myplugin.h"

MyFactory::MyFactory(void)
{
  std::cerr << "MyFactory created" << std::endl;
}

MyFactory::~MyFactory(void)
{
  std::cerr << "MyFactory destroyed" << std::endl;
}

void MyFactory::configure(const std::string& key,
                          const std::string& value) throw (dmlite::DmException)
{
  std::cerr << "[CONF] " << key << " = " << value << std::endl;
  throw dmlite::DmException(DMLITE_CFGERR(DMLITE_UNKNOWN_KEY), "Unknown key %s", key.c_str());
}

You can compile now with

# gcc -fPIC -shared myplugin.cpp -o plugin_my.so

Registering the plugin

Right now we have a shared .so library with the basic functionality we added before, but that's not going to help unless we get our plugin into the DMLite stack. For that, we need to create a "register" function, and create a symbol that will act as an entry point for the DMLite loader.

For that, we just have to add at the end of our myplugin.cpp file the following code:

// Register method
static void registerMyPlugin(dmlite::PluginManager* pm) throw (dmlite::DmException)
{
  pm->registerConfigureFactory(new MyFactory());
}

// This is what DMLite will look for
dmlite::PluginIdCard plugin_my = {
  PLUGIN_ID_HEADER,
  registerMyPlugin
};

And we compile as before.

Now, to test it we will create a new configuration file that loads our plugin, and then the defaults /etc/dmlite.conf.d/*.conf.

# myplugin.conf
LoadPlugin plugin_my     ./plugin_my.so
LoadPlugin plugin_config /usr/lib64/dmlite/plugin_config.so


Include /etc/dmlite.conf.d/*.conf

And we run any of the test mentioned before, using our configuration file (the test executable must be in your PATH if you don't put the full path).

$ dmlite-init myplugin.conf /
MyFactory created
[CONF] DpmHost = localhost
[CONF] TokenPassword = test
[CONF] TokenId = ip
[CONF] TokenLife = 1000
[CONF] TokenPassword = test
[CONF] TokenId = ip
[CONF] TokenLife = 1000
[CONF] MySqlHost = localhost
[CONF] MySqlUsername = dpmdbuser
[CONF] MySqlPassword = changethis
[CONF] Include = /etc/dmlite.conf.d/*.conf
Could not get the final location for /
Reason: [#00.000021] Directories do not have replicas
MyFactory destroyed

Add a PoolManager decorator

The previous example is not able to actually create any concrete implementation of any interface, so its functionallity would be limited to interact with the PluginManager and with the configuration parameters. It could be useful to implement a configuration plugin (as the Include directive is implemented), but doesn't add functionality to the front-end.

Now, we are going to extend the plugin to be able to create a decorator for the PoolManager. Our implementation will do nothing, but delegate every call to the next plugin in the stack.

First, we need to inherit from dmlite::PoolManagerFactory and overload the method createPoolManager, and then create a new implementation of dmlite::PoolManager, inheriting from dmlite::DummyPoolManager, which is a convenient base class for decorator plugins, since it has a default implementation for all methods that just delegates.

///@file myplugin.h
#ifndef _MY_PLUGIN_H
#define _MY_PLUGIN_H

#include <dmlite/cpp/poolmanager.h>
#include <dmlite/cpp/dummy/DummyPool.h>

class MyFactory: public dmlite::PoolManagerFactory {
 public:
  MyFactory(dmlite::PoolManagerFactory* next);
  ~MyFactory(void);

  void configure(const std::string& key,
                 const std::string& value) throw (dmlite::DmException);

  dmlite::PoolManager* createPoolManager(dmlite::PluginManager* pm) throw (dmlite::DmException);

 private:
  dmlite::PoolManagerFactory* next;
};

class MyPoolManager: public dmlite::DummyPoolManager {
 public:
  MyPoolManager(dmlite::PoolManager* decorated) throw (dmlite::DmException);
  ~MyPoolManager();

  // Dummy does not implement this, so we need to do it ourselves
  std::string getImplId() const throw();
};

#endif

Then, we implement the methods:

///@file myplugin.cpp
#include <dmlite/cpp/dmlite.h>
#include <iostream>
#include "myplugin.h"

MyFactory::MyFactory(dmlite::PoolManagerFactory* next): next(next)
{
}

MyFactory::~MyFactory(void)
{
}

void MyFactory::configure(const std::string& key, const std::string& value) throw (dmlite::DmException)
{
}

dmlite::PoolManager* MyFactory::createPoolManager(dmlite::PluginManager* pm) throw (dmlite::DmException)
{
  // We got a pointer to the next factory in 'next', so we use the static protected method
  // to create a PoolManager from that factory.
  // We can not call directly next->createPoolManager(pm) because of the protection.
  // Then, we pass the PoolManager implementation to our own so it can be wrapped.
  return new MyPoolManager(PoolManagerFactory::createPoolManager(next, pm));
}

MyPoolManager::MyPoolManager(dmlite::PoolManager* next) throw (dmlite::DmException):
  dmlite::DummyPoolManager(next)
{
  std::cout << "Wrapping " << next->getImplId() << std::endl;
}

MyPoolManager::~MyPoolManager()
{
}

std::string MyPoolManager::getImplId() const throw ()
{
  return "MyPoolManager";
}

// Register method
static void registerMyPlugin(dmlite::PluginManager* pm) throw (dmlite::DmException)
{
  // What's happening here:
  // Since we are not yet registered, we are getting the previous top of the stack
  // and storing it in our factory so we can call it later.
  // Then, we register our factory.
  pm->registerPoolManagerFactory(new MyFactory(pm->getPoolManagerFactory()));
}

// This is what DMLite will look for
dmlite::PluginIdCard plugin_my = {
  PLUGIN_ID_HEADER,
  registerMyPlugin
};

We have to modify the configuration file too, since now we need to be loaded last, not first: our implementation rely on a previous PoolManager concrete implementation being there to be wrapped.

# myplugin.conf
LoadPlugin plugin_config /usr/lib64/dmlite/plugin_config.so

Include /etc/dmlite.conf.d/*.conf

LoadPlugin plugin_my ./plugin_my.so

After compiling and running, we should get something like this:

$ dmlite-init myplugin.conf /dpm/cern.ch/home/dteam/file.html
Wrapping mysql_pool_manager
Chunk 0: sl6vm.cern.ch:/dpm/cern.ch/home/dteam/dmlite03_test/test529.html (0-63214)
	token: UEkJNEvVepSVHYe8vVEkjikUnE0=@1348231193@0

You can see that the constructor of MyPoolManager is actually being code, but from the rest of the behaviour we see no difference because we are just delegating.
But this is a good starting point: our plugin is already in the stack, and everything it doesn't implement is being delegated

Override the 'whereToRead' method

Now we are going to actually decorate one method: whereToRead. Since doing something actually useful if beyond the scope of this tutorial, we are going to take advantage of the program printing the chunks to put something there that we can see.

For that, we only need to add whereToRead to our class declaration, and then implement the method delegating ourselves to the next plugin, and pushing some additional data.

class MyPoolManager: public dmlite::DummyPoolManager {
 public:
  MyPoolManager(dmlite::PoolManager* decorated) throw (dmlite::DmException);
  ~MyPoolManager();

  std::string getImplId() const throw();

  dmlite::Location whereToRead(const std::string& path) throw (dmlite::DmException);
};

Now, the implementation:

dmlite::Location MyPoolManager::whereToRead(const std::string& path) throw (dmlite::DmException)
{
  dmlite::Location where = DummyPoolManager::whereToRead(path);
  dmlite::Chunk myChunk;
  myChunk.host = "myhost.mydomain.com";
  myChunk.path = path;
  myChunk.offset = 0;
  myChunk.size   = 0;
  myChunk["arbitrary"] = std::string("data");
  where.push_back(myChunk);
  return where;
}

After compiling and executing as before, we would get something like:

# ../dmlite-init myplugin.conf /dpm/cern.ch/home/dteam/file.html
Wrapping mysql_pool_manager
Chunk 0: sl6vm.cern.ch:/dpm/cern.ch/home/dteam/dmlite03_test/test529.html (0-63214)
	token: J9dNueNkEpvmE7rr96BTn85h8oY=@1348231832@0
Chunk 1: myhost.mydomain.com:/dpm/cern.ch/home/dteam/file.html (0-0)
	arbitrary: data

So you can see how our PoolManager took over the request, pushing whatever data it needed to.

Summary

After following this tutorial, implementing new plugins that override additional methods should be relatively easy, since the steps are the same in all cases. The only thing that may change is the Factory and the Interface types to implement.
It could happen that your plugin doesn't need to delegate anything (i.e. a new database backend). In that case, you can inherit directly from the right interface, and not from the Dummy plugin, but in that case no default implementation is given, so using Dummy as a base is always useful when available, since it will give you a base where you can add methods one by one.