[Mageia-sysadm] Packaging puppet modules

[Michael Scherer]

Le vendredi 25 mars 2011 à 19:47 +0100, nicolas vigier a écrit :
> On Fri, 25 Mar 2011, Buchan Milne wrote:
> 
> > 
> > ----- "nicolas vigier" <boklm at mars-attacks.org> wrote:

> > 
> > > I usually have the following problems :
> > >  - modules are made for other distributions
> > >  - modules have dependencies on other modules, and it is not easy to
> > > see
> > >    which ones
> > 
> > And puppet-module (https://github.com/puppetlabs/puppet-module-tool) isn't able to resolve them?
> 
> Oh, I didn't use this tool, I was only cloning modules git repositories.
> Indeed it supports dependencies between modules. However it seems modules
> developers need to list them manually.

First step, let's do a package of that.

> > Well, the first question is, is servers a target use case of Mageia?
> 
> I don't know if target use cases have been defined yet. But it can
> probably be a target, if enough people want to work on this.

Personally, I would be interested ( and slightly more in this space than
on the desktop side ).

> > If so, should we not rather draw up requirements for all the requirements 
> > for increasing the efficiency of managing Mageia servers? Possibly draw up 
> > reference architectures for different types of deployments?
> 
> Yes, that's a good idea.

I think usage of puppet directly in the distribution was something that
as discussed by Ubuntu Server Team. Not sure it went far.

> > 
> > > We could do something like this :
> > >  - one package per module
> > >  - modules installed in /usr/share/puppet/modules
> > >  - use rpm find-requires and find-provides to automatically manage
> > >    dependencies between modules
> > >  - remove any hardcoded values from modules, but use parameterised
> > >    classes, so you don't have to modify modules to use them
> > >  - setup puppet so that someone who wants to overwrite a module
> > > installed
> > >    by a package can create one with the same name in
> > > /etc/puppet/modules.
> > >    Or overwrite one template file.
> > 
> > +1
> > 
> > However, we may need to review what modules we would need, improve any of 
> > the ones we have developed (e.g. my Xymon one probably needs quite a
> > bit of work).
> 
> Yes. Maybe we should also define some policies about how modules should
> be written to allow packaging / standardized parameters.

We also have done some choices that are not easy to change :

- mostly postgresql rather than mysql ( with a high integration like
remote database setup and more to come ( I plan to be able to manage the
initial sql setup with it )

- web applications are on application.example.org rather than
www.example.org/application/

- fastcgi to cope with the fact we have lots of interpreter rather than 
using only ruby, python or perl

- highly dependent on the ldap and on the group structure ( ie, there is
group policy everywhere and this can be quite hard to abstract )

- the domain name of web application are everywhere ( could be
abstracted like password however )

The repository is quite free of mageia.org reference, hopefully.

> > > Those module could also be used locally by some tools. For instance
> > > if
> > > we want to create a graphical tool like drakwizard, to setup some
> > > software
> > > on a machine. This graphical interface would ask some parameters,
> > > then
> > > install the corresponding puppet modules, generate a puppet file
> > > containing the parameters entered and run puppet on it locally.
> > 
> > What is the capability of Puppet Dashboard (http://www.puppetlabs.com/puppet/related-projects/dashboard/) ?
> 
> I never used it, so I don't know. But they mention "classes and parameters
> can be applied to nodes directly or inherited from groups".

That just a web interface, and I fear we would lose the vcs feature if
we use this to manage our server. 
On the other hand, this could help on the reporting side. 

And there is also theforeman, that does the same.

> > 
> > > However compared to what we do currently :
> > >  - it will be more work, to avoid hardcoding anything
> > >  - modules can become more complexe if they support more things
> > >  - we will have to be careful when changing API of released modules,
> > > or
> > >    write it in release notes so users of the packages know what they
> > > need
> > >    to change when updating to a newer release of the distribution.
> > 
> > Well, we may want to upstream modules as well, especially ones that we didn't write from scratch.
> 
> Yes, we can also share them on puppet forge so people using other
> distributions can use them.

Then it would be easier to split our svn into self contained module,
like 1 git/svn/whatever per modules in modules/

And one big one with svn:external, git-slave, whatever to keep the
current way of work.

I do like the idea of sharing them on puppet forge. Not specially for
being reused in all case, but at least to be useful as a reference and
example. Because if we want to be reused, we need to take in account
different package name. I do manage 1 gentoo, 3 mandriva, and 1 fedora
with puppet, and the openssh modules I wrote is full of "if". ( and I
didn't add os x or debian to the mix yet, but for fun I am thinking  ).

> > I think we can try and package some modules we use that are well-tested, or any of ours that we believe are of sufficient quality now. 
> > 
> > But, while some of the contributors on this list may want to contribute to such a project, maybe it 
> > would be better to have a separate team/project/special interest group looking at the requirements/features 
> > for a server release. And, I think mandating puppet for this would be premature (from a design perspective), 
> > as there may be other better architectures/tools (e.g. Chef?).
> 
> Yes, that should be a separate team/project. I posted this on sysadmin
> list because I think that's where most interested people are, but
> anybody using Mageia could be interested to participate.
> 
> And Chef should also be considered.

The first step would be to package chef, but I am far from being happy
when instruction start by "use gem install".
( for the record, I also considered looking at bcfg2 ).

-- 
Michael Scherer