On opening day, I chose to post an entry on adding
a system call to OpenSolaris. Considering the feedback, I thought I’d
continue with brief “How-To add to OpenSolaris” documents for a while.
There’s a lot to choose from here, so I’ll just pick them off as quick as I can.
Todays topic as adding a new kernel module to OpenSolaris.
For the sake of discussion, we will be adding a new module that does nothing
apart from print a message on load and unload. It will be architecture-neutral,
and be distributed as part of a separate package (to give you a taste of our
packaging system). We’ll continue my narcissistic
tradition and name this the “schrock” module.
1. Adding source
To begin, you must put your source somewhere in the tree. It must be put
somewhere under usr/src/uts/common,
but exactly where depends on the type of module. Just about the only real rule
is that filesystems go in the “fs” directory, but other than that there are no
real rules. The bulk of the modules live in the “io” directory, since the
majority of modules are drivers of some kind. For now, we’ll put ‘schrock.c’ in
the “io” directory:
#include <sys/modctl.h>
#include <sys/cmn_err.h>
static struct modldrv modldrv = {
&mod_miscops,
"schrock module %I%",
NULL
};
static struct modlinkage modlinkage = {
MODREV_1, (void *)&modldrv, NULL
};
int
_init(void)
{
cmn_err(CE_WARN, "OpenSolaris has arrived");
return (mod_install(&modlinkage));
}
int
_fini(void)
{
cmn_err(CE_WARN, "OpenSolaris has left the building");
return (mod_remove(&modlinkage));
}
int
_info(struct modinfo *modinfop)
{
return (mod_info(&modlinkage, modinfop));
}
The code is pretty simple, and is basically the minimum needed to add
a module to the system. You notice we use ‘mod_miscops’ in our modldrv.
If we were adding a device driver or filesystem, we would be using a
different set of linkage structures.
2. Creating Makefiles
We must add two Makefiles to get this building:
usr/src/uts/intel/schrock/Makefile usr/src/uts/sparc/schrock/Makefile
With contents similar to the following:
UTSBASE = ../.. MODULE = schrock OBJECTS = $(SCHROCK_OBJS:%=$(OBJS_DIR)/%) LINTS = $(SCHROCK_OBJS:%.o=$(LINTS_DIR)/%.ln) ROOTMODULE = $(ROOT_MISC_DIR)/$(MODULE) include $(UTSBASE)/intel/Makefile.intel ALL_TARGET = $(BINARY) LINT_TARGET = $(MODULE).lint INSTALL_TARGET = $(BINARY) $(ROOTMODULE) CFLAGS += $(CCVERBOSE) .KEEP_STATE: def: $(DEF_DEPS) all: $(ALL_DEPS) clean: $(CLEAN_DEPS) clobber: $(CLOBBER_DEPS) lint: $(LINT_DEPS) modlintlib: $(MODLINTLIB_DEPS) clean.lint: $(CLEAN_LINT_DEPS) install: $(INSTALL_DEPS) include $(UTSBASE)/intel/Makefile.targ
3. Modifying existing Makefiles
There are two remaining Makefile chores before we can continue. First, we have
to add the set of files to usr/src/uts/common/Makefile.files:
KMDB_OBJS += kdrv.o SCHROCK_OBJS += schrock.o BGE_OBJS += bge_main.o bge_chip.o bge_kstats.o bge_log.o bge_ndd.o \ bge_atomic.o bge_mii.o bge_send.o bge_recv.o
If you had created a subdirectory for your module instead of placing it in
“io”, you would also have to add a set of rules to usr/src/uts/common/Makefile.rules.
If you need to do this, make sure you get both the object targets and the
lint targets, or you’ll get build failures if you try to run lint.
You’ll also need to modify the usr/src/uts/intel/Makefile.intel
file, as well as the corresponding SPARC version:
MISC_KMODS += usba usba10 MISC_KMODS += zmod MISC_KMODS += schrock # # Software Cryptographic Providers (/kernel/crypto): #
4. Creating a package
As mentioned previously, we want this module to live in its own package. We
start by creating usr/src/pkgdefs/SUNWschrock and adding it to the list
of COMMON_SUBDIRS in usr/src/pkgdefs/Makefile:
SUNWsasnm \
SUNWsbp2 \
SUNWschrock \
SUNWscpr \
SUNWscpu \
Next, we have to add a skeleton package system. Since we’re only adding a
miscellaneous module and not a full blown driver, we only need a simple
skeleton. First, there’s the Makefile:
include ../Makefile.com .KEEP_STATE: all: $(FILES) install: all pkg include ../Makefile.targ
A ‘pkgimfo.tmpl’ file:
PKG=SUNWschrock NAME="Sample kernel module" ARCH="ISA" VERSION="ONVERS,REV=0.0.0" SUNW_PRODNAME="SunOS" SUNW_PRODVERS="RELEASE/VERSION" SUNW_PKGVERS="1.0" SUNW_PKGTYPE="root" MAXINST="1000" CATEGORY="system" VENDOR="Sun Microsystems, Inc." DESC="Sample kernel module" CLASSES="none" HOTLINE="Please contact your local service provider" EMAIL="" BASEDIR=/ SUNW_PKG_ALLZONES="true" SUNW_PKG_HOLLOW="true"
And ‘prototype_com’, ‘prototype_i386’, and ‘prototype_sparc’ (elided) files:
# prototype_i386 !include prototype_com d none kernel/misc/amd64 755 root sys f none kernel/misc/amd64/schrock 755 root sys
# prototype_com i pkginfo d none kernel 755 root sys d none kernel/misc 755 root sys f none kernel/misc/schrock 755 root sys
5. Putting it all together
If we pkgadd our package, or BFU to the resulting archives, we can see our
module in action:
halcyon# modload /kernel/misc/schrock Jun 19 12:43:35 halcyon schrock: WARNING: OpenSolaris has arrived halcyon# modunload -i 197 Jun 19 12:43:50 halcyon schrock: WARNING: OpenSolaris has left the building
This process is common to all kernel modules (though packaging is simpler for
those combined in SUNWckr, for example). Things get a little more complicated
and a little more specific when you begin to talk about drivers or filesystems
in particular. I’ll try to create some simple howtos for those as
well.
Technorati Tag: OpenSolaris
10 Responses
Nice! Thanks for posting this, now the world is only one step away from having the KitchenSink module, which is really what OpenSolaris is missing 🙂
Thanks Eric, this is great info! I’m looking forward to seeing your fs howto, since I’m toying with the idea to add the NetBSD ffs variant to OpenSolaris…
I am tempted to open up a bug at bugs.opensolaris.org – “Solaris – Even the kernel module creation process takes longer than the Linux One ” ! 🙂 But I don’t feel too encouraging after looking into the Solaris bug database – number of performance bugs vs. Linux are pending since 1999!! 😉
Also, why is that bugs.opensolaris.org so closed and anti-dialogue? Meaning why are there no traces of open discussions on how the bug was fixed? You can’t even know the bug submitter’s comments from developers if they are present.
Anonymous Coward –
First off, you should note that the Linux documentation you link to doesn’t describe how to actually integrate the module into the kernel source, just what the source code should be. On a source comparision it does take slightly more code (~20 lines), but we also have a well-defined DDI in Solaris, which has many advantages beyond “Hello, World!”.
As for performance bugs, we’ve fixed hundreds of performance bugs during Solaris 10 development, and continue to do so now. Every bug has to be prioritized and resourced accordingly, so you’re always going to be able to find some bugs that have languished for long periods of time.
Finally, the comments are not available on bugs.opensolaris.org because they contain proprietary information, including confidential customer data. We don’t have the resources to “scrub” every bug that’s ever been filed against Solaris, so we made available only what we could. We’re working on making this better for the future.
Hmmm, I should’ve pressed the ‘next’ link for the Linux example. Yeah, their Makefile is also smaller. Our Makefile system could certainly do with some love. And a proper packaging system also has a lot of other advantages…
First off, you should note that the Linux documentation you link to doesn’t describe how to actually integrate the module into the kernel source
Best part is Linux doesn’t require you to integrate the module into kernel source – you write your module anywhere with a simple make file and you are done. And even if you wanted to integrate the module into the linux kernel, it is far more simpler than what you described.
Also, I did not understand what you meant by “Well defined DDI” – Linux also has a well defined interface for kernel modules/drivers. It’s another thing that they keep on evolving it but none the less all in-tree drivers are fixed almost automatically.
Regarding bugs.opensolaris.org – Do you mean to say that your customers will continue to enter bugs into a closed system even after opensolaris? That would be less than ideal.
Thanks for answering, BTW.
Anonymous Coward –
Solaris does not require you to integrate your changes, either. You can come up with a signficantly simpler Makefile to build a module (it’s just one ‘cc’ line, anyway). But the purpose of the post was to demonstrate how to add a module to the OpenSolaris tree – not simply build a standalone module.
Linux doesn’t actually have a well-defined DDI, AFAICT. You are free to call every function in the Linux kernel, without regard for any perceived stability. And from what I’ve read on various linux mailing lists, the developers enjoy breaking the supposed ‘interfaces’, because it gives them more ammunition to get the source integrated into the main tree. But I do not want to rekindle this old flamewar in my blog comments – suffice to say that Solaris and Linux have different approaches to device drivers.
Regarding bugs.opensolaris.org – the system itself is always open; there is no separate ‘closed system’. That being said, if supporting data for a bug contains confidential customer information, that will continue to be witheld. It’s just that showing source is no longer a valid reason to keep something confidential 😉
If you have further questions, I encourage you to head over to the OpenSolaris forums, or join us on IRC (#opensolaris on irc.freenode.net). Or email me if you want to continue the discussion off-line. And yes, the bug database is certainly one area that still needs a lot of work.
Linux doesn’t actually have a well-defined DDI, AFAICT. You are free to call every function in the Linux kernel, without regard for any perceived stability.
Sorry as this is mis-information – I would prefer to correct it here. With Linux – You can only call exported Kernel functions from modules. Kernel doesn’t export internal private functions and data structures. Modules for example cannot fiddle with the sys call table since it is not exported. (Sure there are ways to work around but that’s true with any piece of software.) And the part about “… the developers enjoy breaking the supposed ‘interfaces’, because it gives them more ammunition to get the source integrated into the main tree.” is equally false. I am a regular LKML reader and I haven’t seen a single instance where a public kernel interface was broken when there was no solid reason to do so and they don’t break user space compatibility at all.
Sigh. So much for no flamewar. Please point me to any documentation, anywhere, which tells me which functions are part of the stable driver interface and which are not. You’re basically saying “all exported kernel symbols are stable”. This is NOT a DDI. In Solaris, for example, the ‘unix’ module exports a lot of functions, many which are needed by other kernel subsystems that are not, in fact, drivers. No driver should ever be calling <tt>hat_probe()</tt>, but it must remain a global symbol because other modules (i.e. genunix) must use it. As a driver writer, I need to know that whatever function I call is designed for used by driver developers, and will not change in future versions. We can debate the value of a DDI (interface commitment, cost of change, etc) and whether its a worthwhile use of time, but plese don’t try and argue that the Linux kernel actually has a device driver interface.
And yes, despite whatever you might believe, I have been personally involed in discussions (both online and in person) where Linux developers (including major branch maintainers) defend the lack of a DDI precisely because it discourages users from maintaining drivers outside the linux source tree (among other reasons, of course). I did not mean to suggest that people intentionally change interfaces, only that their reaction to such change in the light of external drivers is usually one of pleasure or indifference. And I know that the syscall interface is relatively stable; but that is neither here nor there in this discussion.
Sadly, these comments have gotten out of control. If you want to continue this discussion with me, please follow up with email. If you want to continue this discussion with a larger audience, I encourage to head over the OpenSolaris forums or chat on IRC.
On personal opinion, I find this very helpful.
Guys, I have also posted some more relevant info further on this, not sure if you find it useful: http://www.bidmaxhost.com/forum/