« Oatmeal

lil

Over the last few December adventures I’ve spent time with Decker and the lovely scripting language, lil. Lil is a mishmash of all kinds of great stuff — it is a little Lua, a splash of Scheme, a pinch of perl, and quite a lot of Q. One of the most interesting parts of lil for me is the array oriented, or vector programming, facilities it supports.

Vector programming involves applying operations to entire arrays or vectors of data all at once. A few key concepts that make this possible include:

• Arrays and vectors, a data structure, or type that collects together a bunch of elements
• Element-wise operations, or those operations that can be applied individually to each element of a data structure, like an array, in sequence or all at once
• Reduction (also known as folding), recursively operates on elements of an array to combine or recombine them
• Broadcasting, which is applying functions to arrays of different shapes at the same time
• Higher-order functions that take in other functions as arguments or return functions as their result
• Functional composition, where smaller, simple functions can be assembled to build up complex operations

Now, while you don’t need to know any of this stuff to use lil effectively, I’ve found lil to be a great way into using these sorts of tools in day-to-day programming.

Arrays and vectors

Before I dive into some preliminary examples let’s get to know lil’s data structures — lists, dictionaries and tables. All of which are a kind of array.

From lil’s documentation:

Lists are ordered sequences of values which can be indexed by numbers. The empty list is written as (), and the unary operator list constructs a single-element list from any value. Lists can be joined with the , operator.

Dictionaries are ordered sequences of pairings between keys and values, both of which can be any type. Dictionaries can be made by the binary operator dict, or a list can be promoted to a dictionary by assigning it non-numeric keys. Dictionaries can be unioned together with the , operator. The keys operator gets the keys of a dictionary, and the range operator gets the values of a dictionary.

Tables are a rectangular array of values for which every column has a string as its key. Tables can be made with the unary operator table or with the insert statement. The binary operators , (append), join (natural join), and cross (cartesian join) offer a selection of tabular joins.

So, we can create a list as easily as (1,2,3,4,5,6,7) — we can then access information in the list by index, (1,2,3,4,5,6,7)[3] returns 4. Note! Unlike Lua, but like…most other programming languages, lil is 0 indexed, meaning you start to count from 0.

There’s much more too it, too, but for now that’ll be enough for us to start exploring with — the only other item I’ll walk through before we continue is variable assignment:

To assign our list to a variable we do a:(1,2,3,4,5,6,7) where a is any variable name we want. So, a[3] returns 4! Note! We’re using : (I like to pronounce it as “gets”) for variable assignment, not =.

Element-wise operations

Element-wise operations are those operations that take 2 array of equal length and sort of line them up, so, say we have 2 arrays:

a:(1,2,3,4)
b:(5,6,7,8)

If we were to preform element-wise addition on these two arrays we’d add the value of index 0 from array a to the value of index 0 from array b and so on through the entire length of the array. The result would be a new, 3rd array of equal length to our starting arrays, (6,8,10,12).

Reduction

Reduction allows us to combine all the values of a single array into a scalar value:

a:(1,2,3,4,5,6,7)
sum a

sum is a special sort of reduction in lil, and in our example returns 28 which is the value of index 0 plus the value of index 1 through the entire length of our array all added up.

Broadcasting

Higher-order functions

Function composition

Super powers

A quick aside to explore lil’s super operator and, the one that I find myself missing the most when I go back to other languages, @ — which you can think of as kinda like a map or an each.

This operator is one of the most useful in lil when working with arrays. You can use it to apply a function, or to retrieve elements from a list.

A basic example of using it to access elements at specific indexes,

"ABC" @ 0,0,1,2,1,2,0
# ("A","A","B","C","B","C","A")

"ABC" @ 0,0,1,2,1,2,0 returns elements at specified indexes from the string “ABC”.

…which is sort of boring, you can also use @ to apply a function across every element of a list:

on triple x do x,x,x end
triple @ 1,2,3
# ((1,1,1),(2,2,2),(3,3,3))

This is getting more exciting, this is sorta like using an each, which is also a keyword available to you in lil, but a bit fancier!

The triple function is applied to each element of the list (1,2,3), creating a new list where each original element is repeated three times.

@ can also be combined with unary operators in order to apply them across all elements of a list or dictionary:

first "Hi","Elegant","Lion","Perched"
# "Hi"

first @ "Hi","Elegant","Lion","Perched"
# ("H","E","L","P")

BUT! Where @ really starts to show its stuff is when working with higher-dimensional data structures like matrices.

flip(27,19)+flip 2 cross 3
# ((27,19),(28,19),(27,20),(28,20),(27,21),(28,21))

flip and @ are used to apply addition across a matrix, showing off how you can perform element-wise operations on more complicated structures with lil.