Haskell/Lenses and functional references

This chapter is about functional references. By "references", we mean they point at parts of values, allowing us to access and modify them. By "functional", we mean they do so in a way that provides the flexibility and composability we came to expect from functions. We will study functional references as implemented by the powerful lens library. lens is named after lenses, a particularly well known kind of functional reference. Beyond being very interesting from a conceptual point of view, lenses and other functional references allow for several convenient and increasingly common idioms, put into use by a number of useful libraries.

A taste of lenses
As a warm-up, we will demonstrate the simplest use case for lenses: as a nicer alternative to the vanilla Haskell records. There will be little in the way of explanations in this section; we will fill in the gaps through the remainder of the chapter.

Consider the following types, which are not unlike something you might find in a 2D drawing library:

Record syntax gives us functions for accessing the fields. With them, getting the coordinates of the points that define a segment is easy enough:

Updates, however, are clunkier...

... and get downright ugly when we need to reach a nested field. Here is what it takes to double the value of the y coordinate of the end point:

Lenses allow us to avoid such nastiness, so let's start over with them:

The only real change here is the use of, which automatically generates lenses for the fields of   and   (the extra underscores are required by the naming conventions of  ). As we will see, writing lenses definitions by hand is not difficult at all; however, it can be tedious if there are lots of fields to make lenses for, and thus automatic generation is very convenient.

Thanks to, we now have a lens for each field. Their names match that of the fields, except with the leading underscore removed:

The type  tells us that   is a reference to a   within a. To work with such references, we use the combinators provided by the lens library. One of them is, which gives us the value pointed at by a lens, just like a record accessor:

Another one is, which overwrites the value pointed at:

One of the great things about lenses is that they are easy to compose:

Note that when writing composed lenses, such as, the order is from large to small. In this case, the lens that focuses on a point of the segment comes before the one that focuses on a coordinate of that point. While that might look a little surprising in contrast to how record accessors work (compare with the equivalent lens-less example at the beginning of this section), the  used here is just the function composition operator we know and love.

Composition of lenses provide a way out of the nested record update quagmire. Here is a translation of the coordinate-doubling example using, through which we can apply a function to the value pointed at by a lens:

These initial examples might look a bit magical at first. What makes it possible to use one and the same lens to get, set and modify a value? How come composing lenses with  just works? Is it really so easy to write lenses without the help of ? We will answer such questions by going behind the curtains to find what lenses are made of.

The scenic route to lenses
There are many ways to make sense of lenses. We will follow a sinuous yet gentle path, one which avoids conceptual leaps. Along the way, we will introduce a few different kinds of functional references. Following lens terminology, from now on we will use the word "optics" to refer collectively to the various species of functional references. As we will see, the optics in lens are interrelated, forming a hierarchy. It is this hierarchy which we are now going to explore.

Traversals
We will begin not with lenses, but with a closely related optic: traversals. The Traversable chapter discussed how  makes it possible to walk across a structure while producing an overall effect:

With, you can use any   you like to produce the effect. In particular, we have seen how  can be obtained from   simply by picking   as the applicative functor, and that the same goes for   and , using  :

lens takes this idea and lets it blossom.

Manipulating values within a  structure, as   allows us to, is an example of targeting parts of a whole. As flexible as it is, however,  only handles a rather limited range of targets. For one, we might want to walk across structures that are not  functors. Here is an entirely reasonable function that does so with our  type:

is a traversal of. It looks a lot like a typical implementation of, and can be used in pretty much the same way. The only difference is that  has a fixed type inside  instead of being a polymorphic type. Here is an adaptation of the  example from the Traversable chapter:

This generalised notion of a traversal that  exemplifies is captured by one of the core types of lens:.

With the  synonym, the type of   can be expressed as:

Let's have a closer look at what became of each type variable in :


 * becomes :   is a traversal of a.
 * becomes :   produces a   (in some   context).
 * becomes :   targets   values in a   (the X and Y coordinates of the points).
 * becomes : the targeted   values become   values (possibly different than the original ones).

In the case of,   is the same as  , and   is the same as. does not change the type of the traversed structure, or that of the targets in it, but that need not be the case. One example is good old, whose type can be expressed as:

is able to change the types of the targeted values in the  structure and, by extension, the type of the structure itself.

The module includes generalisations of  functions and various other tools for working with traversals.

Setters
Next in our programme comes the generalisation of the links between,   and. We shall begin with.

To recover  from , we picked   as the applicative functor. That choice allowed us to modify the targeted values without producing any extra effects. We can reach similar results by picking the definition of a ...

... and specialising  to  :

In lens parlance, that is how you get a. For technical reasons, the definition of  in  is a little different...

... but if you dig into the documentation you will find that a  functor is either   or something very much like it, so the difference need not concern us.

When we take  and restrict the choice of   we actually make the type more general. Given that a  works with any   functor, it will also work with , and therefore any   is a   and can be used as one. The reverse, however, is not true: not all setters are traversals.

is the essential combinator for setters. It works a lot like, except that you pass a setter as its first argument in order to specify which parts of the structure you want to target:

In fact, there is a  called   that allows us to recover  :

Another very important combinator is, which replaces all targeted values with a constant. , analogously to how :

Folds
Having generalised the -as-traversal trick, it is time to do the same with the  -as-traversal one. We will use  to go from...

... to:

Since the second parameter of  is irrelevant, we replace   with   and   with   to make our life easier.

Just like we have seen for  and ,  uses something slightly more general than  :

is both a  and an. Thanks to the functor and contravariant laws, anything that is both a  and a   is, just like , a vacuous functor, with both   and   doing nothing. The additional  constraint corresponds to the  ; it allows us to actually perform the fold by combining the  -like contexts created from the targets.

Every  can be used as a , given that a   must work with any  , including those that are also. The situation parallels exactly what we have seen for  and.

offers analogues to everything in. Two commonly seen combinators from that module are, which produces a list of the   targets...

... and, which extracts the first target of a   using the   monoid from.

Getters
So far we have moved from  to more general optics (  and  ) by restricting the functors available for traversing. We can also go in the opposite direction, that is, making more specific optics by broadening the range of functors they have to deal with. For instance, if we take ...

... and relax the  constraint to merely , we obtain  :

As  still has to be both   and , it remains being a  -like vacuous functor. Without the  constraint, however, we can't combine results from multiple targets. The upshot is that a  always has exactly one target, unlike a   (or, for that matter, a , or a  ) which can have any number of targets, including zero.

The essence of  can be brought to light by specialising   to the obvious choice,  :

Since a  value can be losslessly converted to a   value and back, the type above is equivalent to:

An  function, however, is just an   function in disguise (the camouflage being continuation passing style):

Thus we conclude that a  is equivalent to a   function. From this point of view, it is only natural that it takes exactly one target to exactly one result. It is not surprising either that two basic combinators from are , which makes a   out of an arbitrary function, and  , which converts a   back to an arbitrary function.

Lenses at last
If we go back to ...

... and relax the  constraint to , just as we did when going from   to  ...

... we finally reach the  type.

What changes when moving from  to  ? As before, relaxing the  constraint costs us the ability to traverse multiple targets. Unlike a, a   always focuses on a single target. As usual in such cases, there is a bright side to the restriction: with a, we can be sure that exactly one target will be found, while with a   we might end up with many, or none at all.

The absence of the  constraint and the uniqueness of targets point towards another key fact about lenses: they can be used as getters. plus  is a strictly more specific constraint than just , and so   is strictly more general than. As every  is also a   and therefore a , we conclude that lenses can be used as both getters and setters. That explains why lenses can replace record labels.

Here is a quick demonstration of the flexibility of lenses using, a lens that focuses on the first component of a tuple:

Composition
The optics we have seen so far fit the shape...

... in which:


 * is a  of some sort;
 * is the type of the whole, that is, the full structure the optic works with;
 * is the type of what the whole becomes through the optic;
 * is the type of the parts, that is, the targets within  that the optic focuses on; and
 * is the type of what the parts becomes through the optic.

One key thing those optics have in common is that they are all functions. More specifically, they are mapping functions that turn a function acting on a part into a function acting on the whole. Being functions, they can be composed in the usual manner. Let's have a second look at the lens composition example from the introduction:

An optic modifies the function it receives as argument to make it act on a larger structure. Given that  composes functions from right to left, we find that, when reading code from left to right, the components of an optic assembled with   focus on progressively smaller parts of the original structure. The conventions used by the <tt>lens</tt> type synonyms match this large-to-small order, with  and   coming before   and. The table below illustrates how we can look at what an optic does either a mapping (from small to large) or as a focusing (from large to small), using  as an example:

The types behind synonyms such as  and   only differ in which functors they allow in place of. As a consequence, optics of different kinds can be freely mixed, as long as there is a type which all of them fit. Here are some examples:

Operators
Several <tt>lens</tt> combinators have infix operator synonyms, or at least operators nearly equivalent to them. Here are the correspondences for some of the combinators we have already seen:

<tt>lens</tt> operators that extract values (e.g.,   and  ) are flipped with respect to the corresponding prefix combinators, so that they take the structure from which the result is extracted as the first argument. That improves readability of code using them, as writing the full structure before the optics targeting parts of it mirrors how composed optics are written in large-to-small order. With the help of the  operator, which is defined simply as , the structure can also be written first when using modifying operators (e.g.   and  ). is particularly convenient when there are many fields to modify:

A Swiss army knife
Thus far we have covered enough of <tt>lens</tt> to introduce lenses and show that they aren't arcane magic. That, however, is only the tip of the iceberg. <tt>lens</tt> is a large library providing a rich assortment of tools, which in turn realise a colourful palette of concepts. The odds are that if you think of anything in the core libraries there will be a combinator somewhere in <tt>lens</tt> that works with it. It is no exaggeration to say that a book exploring every corner of <tt>lens</tt> might be made as long as this one you are reading. Unfortunately, we cannot undertake such an endeavour right here. What we can do is briefly discussing a few other general-purpose <tt>lens</tt> tools you are bound to encounter in the wild at some point.

manipulation
There are quite a few combinators for working with state functors peppered over the  modules. For instance:


 * from  is an analogue of   from   that takes a getter instead of a plain function.
 * includes suggestive-looking operators that modify parts of a state targeted a setter (e.g.  is analogous to ,   to   and   to  ).
 * offers the remarkably handy  combinator, which uses a traversal (or a lens) to zoom into a part of a state. It does so by lifiting a stateful computation into one that works with a larger state, of which the original state is a part.

Such combinators can be used to write highly intention-revealing code that transparently manipulates deep parts of a state:

Isos
In our series of  and   examples, we have been using the   function as a convenient way to make a   out of   pair.

The X and Y coordinates of the resulting  correspond exactly to the two components of the original pair. That being so, we can define an  function...

... so that  and   are a pair of inverses, that is, they undo each other:

In other words,  and   provide a way to losslessly convert a pair to a point and vice-versa. Using jargon, we can say that  and   form an isomorphism.

might be made into a. Symmetrically. would give rise to a, and the two lenses would be a pair of inverses. Lenses with inverses have a type synonym of their own,, as well as some extra tools defined in.

An  can be built from a pair of inverses through the   function:

s are es, and so the familiar lens combinators work as usual:

Additionally, s can be inverted using  :

Another interesting combinator is. As the name suggests, it is just like, except that it uses the inverted   that   would give us. We will demonstrate it by using the  isomorphism to play with the   representation of  s without using   and   from   explicitly:

s and other single-constructor types give rise to isomorphisms. exploits that fact to provide -based tools which, for instance, make it unnecessary to remember record label names for unwrapping  s...

... and that make  wrapping for instance selection less messy:

Prisms
With, we have reached for the first time a rank below   in the hierarchy of optics: every   is a  , but not every   is an. By going back to, we can observe how the optics get progressively less precise in what they point to:


 * An  is an optic that has exactly one target and is invertible.
 * A  also has exactly one target but is not invertible.
 * A  can have any number of targets and is not invertible.

Along the way, we first dropped invertibility and then the uniqueness of targets. If we follow a different path by dropping uniqueness before invertibility, we find a second kind of optic between isomorphisms and traversals: prisms. A  is an invertible optic that need not have exactly one target. As invertibility is incompatible with multiple targets, we can be more precise: a  can reach either no targets or exactly one target.

Aiming at a single target with the possibility of failure sounds a lot like pattern matching, and prisms are indeed able to capture that. If tuples and records provide natural examples of lenses,,   and other types with multiple constructors play the same role for prisms.

Every  is a , and so the usual combinators for traversals, setters and folds all work with prisms:

A  is not a , though: the target might not be there. For that reason, we use  rather than   to retrieve the target:

For inverting a, we use   and   from. is analogous to, though it gives merely a. is equivalent to  with the inverted prism.

Just like there is more to lenses than reaching record fields, prisms are not limited to matching constructors. For instance, defines , which encodes equality tests as a  :

The  and   functions allow us to build our own prisms. Here is an example using  from  :

is available from <tt>lens</tt>, in the module.

Laws
There are laws specifying how sensible optics should behave. We will now survey those that apply to the optics that we covered here.

Starting from the top of the taxonomy,  does not have laws, just like the   class. does not have laws either, which is not surprising, given that any function can be made into a  via.

, however, does have laws. is a generalisation of, and is therefore subject to the functor laws:

As, a consequence of the second functor law is that:

That is, setting twice is the same as setting once.

laws, similarly, are generalisations of the  laws:

The consequences discussed in the Traversable chapter follow as well: a traversal visits all of its targets exactly once, and must either preserve the surrounding structure or destroy it wholly.

Every  is a   and a , and so the laws above also hold for lenses. In addition, every  is also a. Given that a lens is both a getter and a setter, it should get the same target that it sets. This common sense requirement is expressed by the following laws:

Together with the "setting twice" law of setters presented above, those laws are commonly referred to as the lens laws.

Analogous laws hold for s, with   instead of   and   instead of  :

s are both lenses and prisms, so all of the laws above hold for them. The prism laws, however, can be simplified, given that for isomorphisms  (that is,   never fails):

Polymorphic updates
When we look at optic types such as  and   we see four independent type variables. However, if we take the various optic laws into account we find out that not all choices of,  ,   and   are reasonable. For instance, consider the "setting twice" law of setters:

For "setting twice is the same than setting once" to make sense, it must be possible to set twice using the same setter. As a consequence, the law can only hold for a  if   can somehow be specialised so that it becomes equal to   (otherwise the type of the whole would change on every , leading to a type mismatch).

From considerations about the types involved in the laws such as the one above, it follows that the four type parameters in law-abiding s,  s,  s and  es are not fully independent from each other. We won't examine the interdependency in detail, but merely point out some of its consequences. Firstly,  and   are cut from the same cloth, in that even if an optic can change types there must be a way of specialising   and   to make them equal; furthermore, the same holds for   and. Secondly, if  and   are equal then   and   must be equal as well.

In practice, those restrictions mean that valid optics that can change types usually have  and   parametrised in terms of   and. Type-changing updates in this fashion are often referred to as polymorphic updates. For the sake of illustration, here are a few arbitrary examples taken from <tt>lens</tt>:

At this point, we can return to the question left open when we presented the  type. Given that  and   allow type changing while   and   do not, it would be indeed rash to say that every   is a , or that every   is a. However, the interdependence of the type variables mean that every lawful  can be used as a , and every lawful   can be used as a  , as lawful lenses and traversals can always be used in non type-changing ways.

No strings attached
As we have seen, we can use <tt>lens</tt> to define optics through functions such as  and auto-generation tools such as. Strictly speaking, though, these are merely convenience helpers. Given that,   and so forth are just type synonyms, their definitions are not needed when writing optics − for instance, we can always write   instead of. That means we can define optics compatible with <tt>lens</tt> without using <tt>lens</tt> at all! In fact, any,  ,   or   can be defined with no dependencies other than the <tt>base</tt> package.

The ability to define optics without depending on the <tt>lens</tt> library provides considerable flexibility in how they can be leveraged. While there are libraries that do depend on <tt>lens</tt>, library authors are often wary of acquiring a dependency on large packages with several dependencies such as <tt>lens</tt>, especially when writing small, general-purpose libraries. Such concerns can be sidestepped by defining the optics without using the type synonyms or the helper tools in <tt>lens</tt>. Furthermore, the types being only synonyms makes it possible to have multiple optic frameworks (i.e. <tt>lens</tt> and similar libraries) that can be used interchangeably.