Play With Lua!

I’m not an expert on Lua. I’ve been using it for less than a year; most of the time before that I wrote Ruby, JavaScript, Scheme, etc. It’s not a terribly large language. I usually tell people that if they know JavaScript they can learn Lua in a weekend, and that’s true as far as the syntax goes. But, there are several Lua idioms that are either complicated or just not explained well, and one of those is modules. You can get a long way coding in Lua without ever using a module, but eventually you’ll want to know how they work.

I’m going to explain what modules are for, how to use them, and most importantly why they work that way, and I’m going to do that by implementing Lua’s module system myself. Let’s begin:

What’s a module?

Imagine you’re writing some code and you want to use a very generic name for a function, like init(). Or maybe you want to have a set of functions that work together to provide something (like drawing on the screen, managing a combat system, whatever) but have some of them only be visible to functions also in that set, so that only resolve_combat() can be called by other code, not modifier_for_weapon(). For whatever reason, you want to draw a big box around part of your program and say “these things all work together, and they should be treated as a unit”. That’s what a module is.

What you want, in more specific terms, is a table containing a bunch of functions. These functions should all be defined in the same scope, so they can see local variables (other, private functions) that aren’t in the table. If you were going to define this in an ad-hoc way, it would look like this:

function make_dice_module()
   local dice = {}
   local rand = math.random
 
   dice.roll = function(number, sides, modifier)
                  modifier = modifier or 0
                  if number <= 0 then return modifier
                  else
                     return dice.roll(number-1, sides, modifier + rand(sides))
                  end
               end
 
   return dice
end
 
dice = make_dice_module()
print(dice.roll(2,6))

Make a function, so we can define stuff in its scope. Then, create a table, put some functions in it, and return the table. This is essentially the only way to do modular programming in Javascript, by the way: the primitives that Lua provides to make module() work don’t exist there.

Some improvements

This function is a little ugly because it’s responsible for two things: it creates and manages the dice table, and it also puts all the functions in it. Let’s write the first version of our module function by extracting that first part out:

function make_module(definer)
   local module = {}
   definer(module)
   return module
end
 
dice = make_module(
   function(dice)
      local rand = math.random
 
      dice.roll = function(number, sides, modifier)
                     -- ... etc ...
                  end
   end)

Now we call make_module and pass it a function that adds all the stuff. That separates out the defining-a-module part from the actual-code-in-the-module part. It’s a good first step. Now, let’s see if we can tie this inte the loader some.

Loading packages

One of my favorite areas of Lua is its dependency system. This is a place where a lot of otherwise good languages fall flat, forcing you to jump through hoops to make changes to how it loads things (or jump through flaming hoops in order to let other people run your code). Lua doesn’t make any assumptions about the environment it’s running in, though, and offers easy hooks to customize everything.

The general way you load code in Lua is, you use require to load a module (used here as a generic term). As far as it’s concerned, a module is just any variable that it can stick in package.loaded; it has a list of places that it looks (given by functions called searchers) and it stops as soon as one of those places finds a value. So, let’s have our make_module function, in addition to returning the module, add it to package.loaded. This will mean we also have to know what the module’s name is:

function make_module(name, definer)
   local module = {}
   definer(module)
   package.loaded[name] = module
   return module
end

This by itself doesn’t do much, but if we then put the make_module stuff in a file called “dice.lua” and call this from the Lua prompt:

dice = require 'dice'
print(dice.roll(2,6))

we’ll have loaded the code, because one of the default searchers knows to look in a file named the same as the module. We can even go one better, and have make_module automatically make the variable for us: all globally-scoped variables in Lua are actually entries in a table called _G, so if we just stick it in there then we’ll have it available as soon as we require it:

function make_module(name, definer)
   local module = {}
   definer(module)
   package.loaded[name] = module
   _G[name] = module
   return module
end
 
-- -- --
 
require 'dice'
print(dice.roll(2,6))

Function environments

We’re pretty close to having a convenient module system. From the calling side it’s now totally transparent; you just require('dice') and you’re done. Actually implementing the module, though, you have that call to make_module still, and there’s no way to get rid of that, right? You need to make a scope to put the private variables in, and the only way to do that is to make them local variables of a function.

Well, not quite. Take a look at the description of load() in the Lua reference: it takes a function that returns code as a string, and it returns that code compiled into a function. loadfile uses load, meaning that there’s an invisible, anonymous function wrapping everything in dice.lua.

So what good does that do us? Well, for one thing, it means that if we declare a variable as local in dice.lua, it won’t be visible to other files. It also means that any operation that manipulates scopes that we can do to a function, we can do to a file. Like call setfenv().

setfenv() changes the environment of a running function, identified by its position on the call stack. The environment is just the table of global variables that function uses. So, what if we had make_module set the environment of the function that called it to the new module? Then, anything that function defines as global (or that file defines as global, since files are functions) gets stuck in the module. Writing dice.lua turns into this:

function make_module(name)
   local module = {}
   package.loaded[name] = module
   _G[name] = module
   setfenv(2, module)
   return module
end
 
local rand = math.random
make_module('dice')
 
function roll(number, sides, modifier)
   modifier = modifier or 0
   if number <= 0 then return modifier
   else
      return roll(number-1, sides, modifier + rand(sides))
   end
end

Now we don’t have to do anything special when defining roll, we just tell it it’s in a module and it goes there. The only tricky bit is that we can’t refer to math.random after we do it. Hm.

Outside the module

The module can’t write to the globals table because we set the environment, but that also means it can’t read from it. In order to bring in a function from the actual globals in to the module so we can use it, we have to create a local variable (locals aren’t touched by setfenv) that points to it, before we make the module. That’s why we have that local rand = math.random up there. But, sometimes, that’s a pain in the ass. What if we could have make_module set the module’s metatable to use the original _G as its index? Then the module could still see everything outside itself, while protecting its own stuff:

function make_module(name)
   local module = {}
   package.loaded[name] = module
   _G[name] = module
   setmetatable(module, {__index=_G})
   setfenv(2, module)
   return module
end

Sometimes, though, we don’t want to do that. The boundary between the module and the rest of the program goes both ways: it’s nice to be sure that the module isn’t going to accidentally overwrite some global that something else defines, but it’s also nice to have a list of every outside thing that the module requires. So let’s make it not always set the metatable. In fact, let’s make it as generic as possible, and have it take a set of functions that it will apply to the module, one of which will be “let this module see all outside stuff”:

function make_module(name, ...)
   local module = {}
   package.loaded[name] = module
   _G[name] = module
   setfenv(2, module)
   for _,fn in ipairs{...} do fn(module) end
   return module
end
 
function seeall(module)
   setmetatable(module, {__index=_G})
end

Now we make the module like this:

make_module('dice', seeall)

And now we can define our function in the module, and it can see the math table just like normal:

function roll(number, sides, modifier)
   modifier = modifier or 0
   if number <= 0 then return modifier
   else
      return roll(number-1, sides, modifier + math.random(sides))
   end
end

Lua’s built-in module system

Lua’s built-in module function is used in a strikingly similar fashion to the one we just built:

module('dice', package.seeall)

In fact, if you go look at the list of things that the Lua reference manual says it does, it looks strikingly similar to what we just wrote:

Creates a new table t and sets it as the value of the global name . . . Finally, module sets t as the new environment of the current function and the new value of package.loaded[name] . . . This function can receive optional options after the module name, where each option is a function to be applied over the module.

That’s exactly what ours does. In fact, make_module is a drop-in replacement for module. This is exactly how Lua’s standard module system works.

How to use modules

So that brings us back around to the original problem, which was how do we use modules in Lua. Simple: you stick module('whatever') in the top of “whatever.lua”, and then any global variable x in that file becomes whatever.x. Which you could probably have figured out anyway, but now you know how it was made. Which means you can make your own that works slightly differently, if you ever need to.