[Mageia-sysadm] Packaging puppet modules

[nicolas vigier]

On Fri, 25 Mar 2011, Buchan Milne wrote:

> 
> ----- "nicolas vigier" <boklm at mars-attacks.org> wrote:
> 
> > Hello,
> > 
> > We have now created a lot of puppet modules for different things used
> > on Mageia servers. And I think many of them could be useful for other
> > people installing servers. They are already available on our svn
> > server,
> > but I think we can do something better, to make them easier to use.
> > 
> > When trying to use puppet modules from other people that I found on
> > the
> > internet,
> 
> >From puppet forge (http://forge.puppetlabs.com/), or somewhere else?

Yes, from there. Modules available there are of various quality.

> 
> > I usually have the following problems :
> >  - modules are made for other distributions
> >  - modules have dependencies on other modules, and it is not easy to
> > see
> >    which ones
> 
> And puppet-module (https://github.com/puppetlabs/puppet-module-tool) isn't able to resolve them?

Oh, I didn't use this tool, I was only cloning modules git repositories.
Indeed it supports dependencies between modules. However it seems modules
developers need to list them manually.

> 
> >  - sometimes they can have dependency on an other module doing
> > something
> >    that you already do with a different module (for instance there is
> > many
> >    different apache httpd modules)
> >  - modules come from unknow people, so you have to check them
> > carefully
> >    before using them
> >  - you need to modify them to do what you want, because there often
> > is
> >    a lot of hardcoded values everywhere
> > 
> > So I think it could be interesting to provide some modules for
> > Mageia,
> > and distribute them as packages, doing the same as what we do for
> > other
> > software.
> 
> Well, the first question is, is servers a target use case of Mageia?

I don't know if target use cases have been defined yet. But it can
probably be a target, if enough people want to work on this.

> If so, should we not rather draw up requirements for all the requirements for increasing the efficiency of managing Mageia servers? Possibly draw up reference architectures for different types of deployments?

Yes, that's a good idea.

> 
> > We could do something like this :
> >  - one package per module
> >  - modules installed in /usr/share/puppet/modules
> >  - use rpm find-requires and find-provides to automatically manage
> >    dependencies between modules
> >  - remove any hardcoded values from modules, but use parameterised
> >    classes, so you don't have to modify modules to use them
> >  - setup puppet so that someone who wants to overwrite a module
> > installed
> >    by a package can create one with the same name in
> > /etc/puppet/modules.
> >    Or overwrite one template file.
> 
> +1
> 
> However, we may need to review what modules we would need, improve any of the ones we have developed (e.g. my Xymon one probably needs quite a bit of work).

Yes. Maybe we should also define some policies about how modules should
be written to allow packaging / standardized parameters.

> For quite some time I have been wanting to make a graphical tool for installing and configuring an LDAP server, using the online configuration (cn=config) capabilities of OpenLDAP, replacing 'openldap-mandriva-dit'. If it is possible to do this with puppet (I suspect not), I might consider a module for that (although, for the OpenLDAP project, it would probably be better to have a tool not dependent on puppet, so a Catalyst component was my original plan).
> 
> > Those module could also be used locally by some tools. For instance
> > if
> > we want to create a graphical tool like drakwizard, to setup some
> > software
> > on a machine. This graphical interface would ask some parameters,
> > then
> > install the corresponding puppet modules, generate a puppet file
> > containing the parameters entered and run puppet on it locally.
> 
> What is the capability of Puppet Dashboard (http://www.puppetlabs.com/puppet/related-projects/dashboard/) ?

I never used it, so I don't know. But they mention "classes and parameters
can be applied to nodes directly or inherited from groups".

> 
> > However compared to what we do currently :
> >  - it will be more work, to avoid hardcoding anything
> >  - modules can become more complexe if they support more things
> >  - we will have to be careful when changing API of released modules,
> > or
> >    write it in release notes so users of the packages know what they
> > need
> >    to change when updating to a newer release of the distribution.
> 
> Well, we may want to upstream modules as well, especially ones that we didn't write from scratch.

Yes, we can also share them on puppet forge so people using other
distributions can use them.

> 
> > So it is more work, but I think it can be worth it. We can maybe
> > convince
> > other Mageia packagers to help write puppet modules for the software
> > they package.
> > 
> > A distribution providing puppet modules for many software would allow
> > someone to install one or many servers very quickly and easily, and I
> > think it would be very nice.
> > 
> > Maybe this is something that could be done for Mageia release 2 or
> > later.
> > So I plan to try to see if/how this could be done.
> > 
> > What do you think ?
> 
> I think we can try and package some modules we use that are well-tested, or any of ours that we believe are of sufficient quality now. 
> 
> But, while some of the contributors on this list may want to contribute to such a project, maybe it would be better to have a separate team/project/special interest group looking at the requirements/features for a server release. And, I think mandating puppet for this would be premature (from a design perspective), as there may be other better architectures/tools (e.g. Chef?).

Yes, that should be a separate team/project. I posted this on sysadmin
list because I think that's where most interested people are, but
anybody using Mageia could be interested to participate.

And Chef should also be considered.

> 
> Also, would there be any issue regarding support? Would we point support requests to Puppet Labs?

What kind of issue regarding support ? I think it would be the same as
other open source projects we include that are maintained by a company
providing support.