Made the be/de macros for (co)monadic expression extensible.

14 files changed, 212 insertions, 1 deletions

diff --git a/documentation/book/the_lux_programming_language/chapter_8.md b/documentation/book/the_lux_programming_language/chapter_8.md
index 11b56faf8..fc3e9c4a9 100644
--- a/documentation/book/the_lux_programming_language/chapter_8.md
+++ b/documentation/book/the_lux_programming_language/chapter_8.md
@@ -311,5 +311,5 @@ Hopefully, you'll be able to get a feel for them in the next chapters, because w
 
 So, buckle-up, cowboy. This ride is about to get bumpy.
 
-See you in the next chapter!
+See you in [the next chapter](chapter_9.md)!
 
diff --git a/documentation/book/the_lux_programming_language/chapter_9.md b/documentation/book/the_lux_programming_language/chapter_9.md
new file mode 100644
index 000000000..d79057d63
--- /dev/null
+++ b/documentation/book/the_lux_programming_language/chapter_9.md
@@ -0,0 +1,168 @@
+# Chapter 9: Metaprogramming
+
+_Where we go meta. For real._
+
+---
+
+Metaprogramming is the art of making programs... that make programs.
+
+There are many techniques and tools for achieving this, but one that is very familiar to _Lisp_ fans is to use macros to generate code at compile-time.
+
+However, we're not going to talk about macros in this chapter.
+
+Instead, I'll reveal the infrastructure that makes macros possible, and we'll discuss macros in the next chapter.
+
+## The `Lux` type
+
+	Yeah, I'm aware that it's weird there's a type with the same name as the language, but I couldn't figure out a better name.
+
+The Lux compiler was designed to integrate very well with the language itself.
+
+Most compilers are just programs that take source code and emit some binary executable or some byte-code. But the Lux compiler opens itself for usage within Lux programs and provides Lux programmers with a wealth of information.
+
+The `Lux` type enters the stage.
+
+```
+(type: #export Lux
+  {#info            Info
+   #source          Source
+   #location        Location
+   #current_module  (Maybe Text)
+   #modules         (List [Text Module])
+   #scopes          (List Scope)
+   #type_context    Type_Context
+   #expected        (Maybe Type)
+   #seed            Nat
+   #scope_type_vars (List Nat)
+   #extensions      Any
+   #host            Any})
+```
+
+	By the way, the `Lux` type and other weird types you may not recognize there are all defined in the `library/lux` module. Check the documentation for the Standard Library for more details.
+
+The `Lux` type represents the state of the Lux compiler at any given point.
+
+It iss not a reflection of that state, or a subset of it. It is `the` state of the Lux compiler; and, as you can see, it contains quite a lot of information about compiled modules, the state of the type-checker, the lexical and global environments, and more.
+
+Heck, you can even access the yet-unprocessed source code of a module at any given time.
+
+That's pretty neat.
+
+You can actually write computations that can read and even modify (_careful with that one_) the state of the compiler. This turns out to be massively useful when implementing a variety of powerful macros.
+
+For example, remember the `open:` and `\` macros from [chapter 7](chapter_7.md)?
+
+They actually look up the typing information for the structures you give them to figure out the names of members and generate the code necessary to get that functionality going.
+
+And that is just the tip of the iceberg.
+
+The possibilities are really vast when it comes to using the information provided by the `Lux` compiler state.
+
+## The `Meta` type
+
+_But, how do I use it?_
+
+Well, that is where the `Meta` type and the `library/lux/meta` module come into play.
+
+The `library/lux/meta` module houses many functions for querying the `Lux` compiler state for information, and even to change it a little bit (in safe ways).
+
+I won't go into detail about what's available, but you'll quickly get an idea of what you can do if you read the documentation for it in the Standard Library.
+
+However, one thing I _will_ say is that those functions rely heavily on the `Meta` type, which is defined thusly:
+
+```
+(type: #export (Meta a)
+  (-> Lux (Either Text [Lux a])))
+```
+
+	The `Meta` type is defined in the `library/lux` module, although most functions that deal with it are in the `library/lux/meta` module.
+
+The `Meta` type has a `Functor`, and a `Monad`, but they are a bit rather complicated.
+
+You saw some functor/monad examples in the last chapter, but this is more colorful.
+
+`Meta` instances are functions that given an instance of the `Lux` compiler state, will perform some calculations which may fail (_with an error message_); but if they succeed, they yield a value, plus a (_possibly updated_) instance of the `Lux` compiler state.
+
+Lux metaprogramming is based heavily on the `Meta` type, and macros themselves rely on it for many of their functionalities, as you'll see in the next chapter.
+
+## Where do `Lux` instances come from?
+
+Clearly, `Lux` instances are data, but the compiler is not available at all times.
+
+The compiler is only ever present during... well... compilation.
+
+And that is precisely when all of your `Lux`-dependant code will execute.
+
+Basically, in order for you to get your hands on that _sweet_ compiler information, your code must be run at compile-time. But only macro code can ever do that, so you will have to wait until the next chapter to learn how this story ends.
+
+## Definition annotations
+
+Another important piece of information you should be aware of is that definitions don't just have values and types associated with them, but also arbitrary meta-data which you can customize as much as you want.
+
+The relevant types in the `library/lux` module are:
+
+```
+(type: #export Location
+  {#module Text
+   #line   Nat
+   #column Nat})
+
+(type: #export (Ann m v)
+  {#meta  m
+   #datum v})
+
+(type: #export (Code' w)
+  (#Bit Bit)
+  (#Nat Nat)
+  (#Int Int)
+  (#Rev Rev)
+  (#Frac Frac)
+  (#Text Text)
+  (#Identifier Name)
+  (#Tag Name)
+  (#Form (List (w (Code' w))))
+  (#Tuple (List (w (Code' w))))
+  (#Record (List [(w (Code' w)) (w (Code' w))])))
+
+(type: #export Code
+  (Ann Location (Code' (Ann Location))))
+```
+
+You can add annotations to definitions in the many definition macros offered in the standard library.
+You can also annotate modules when using the `module:` macro.
+
+All you need to do is pass in some record syntax, with tags signaling the type of annotation you want, and the associated value being either an explicit variant `Code`, or some function or macro call that would produce such a value.
+
+Here's an example from `library/lux`:
+
+```
+(def: #export (is? reference sample)
+  {#.doc (doc "Tests whether the 2 values are identical (not just 'equal')."
+              "This one should succeed:"
+              (let [value +5]
+                (is? value value))
+
+              "This one should fail:"
+              (is? +5 (+ +2 +3)))}
+  (All [a] (-> a a Bit))
+  ("lux is" reference sample))
+```
+
+	The (_optional_) annotations always goes after the declaration or name of the thing being defined.
+
+Note that all tag usage within annotation records should be fully qualified, to avoid potential confusions, as different modules could be using annotation tags with similar names.
+
+The `library/lux/meta/annotation` module contains various functions for reading and exploring annotations, and some modules in the standard library (for example, the `lbrary/lux/ffi` module) make heavy use of annotations to figure out properties of definitions which may be useful during code-generation and parsing in macros.
+
+And also, as you can appreciate from the previous example, some macros may be designed to be used during annotation specification.
+
+---
+
+This chapter feels a little empty because the topic only makes sense within the context of macros. But macros by themselves are a huge subject, and involve more machinery than you've seen so far.
+
+However, I wanted to give you a taste of what's possible in order to whet your appetite, while keeping the chapter focused.
+
+In the next chapter, I'll complete this puzzle, and you'll be given access to a power greater than you've ever known (_unless you've already been a lisper for a while_).
+
+See you in the next chapter!
+
diff --git a/documentation/bookmark/compilation/invariant.md b/documentation/bookmark/compilation/invariant.md
new file mode 100644
index 000000000..a8af7e993
--- /dev/null
+++ b/documentation/bookmark/compilation/invariant.md
@@ -0,0 +1,5 @@
+# Reference
+
+1. [William J Bowman: Compilation as Multi Language Semantics](https://www.youtube.com/watch?v=RfVhUPkAEKo)
+1. [Do compilers respect programmers?](https://www.williamjbowman.com/resources/wjb-talk-respinv.pdf)
+
diff --git a/documentation/bookmark/database.md b/documentation/bookmark/database.md
index 5b0784c07..3ef227087 100644
--- a/documentation/bookmark/database.md
+++ b/documentation/bookmark/database.md
@@ -123,6 +123,7 @@
 
 # Exemplar
 
+1. [Debunking “Purpose-Built Data Systems”: Enter the Universal Database](https://tiledb.com/blog/debunking-purpose-built-data-systems-enter-the-universal-database-2021-08-04)
 1. [Irmin](https://irmin.org/)
 1. [Database of Databases](https://dbdb.io/)
 
diff --git a/documentation/bookmark/database/relational.md b/documentation/bookmark/database/relational.md
new file mode 100644
index 000000000..1977a1cec
--- /dev/null
+++ b/documentation/bookmark/database/relational.md
@@ -0,0 +1,4 @@
+# Reference
+
+1. [Implicit ordering in relational languages](https://scattered-thoughts.net/writing/implicit-ordering-in-relational-languages)
+
diff --git a/documentation/bookmark/floating_point.md b/documentation/bookmark/floating_point.md
index 238112f51..47527b43e 100644
--- a/documentation/bookmark/floating_point.md
+++ b/documentation/bookmark/floating_point.md
@@ -1,3 +1,7 @@
+# Half-precision
+
+1. [Supporting half-precision floats is really annoying](https://futhark-lang.org/blog/2021-08-05-half-precision-floats.html)
+
 # Debugging
 
 1. [Keynote: William Kahan - Debugging Tools for Floating-Point Code](https://www.youtube.com/watch?v=qHddEkfQBrA)
diff --git a/documentation/bookmark/game/math.md b/documentation/bookmark/game/math.md
new file mode 100644
index 000000000..708a77871
--- /dev/null
+++ b/documentation/bookmark/game/math.md
@@ -0,0 +1,4 @@
+# Reference
+
+1. [A Bestiary of Functions for Systems Designers](https://brunodias.dev/2021/03/19/functions-for-system-designers.html)
+
diff --git a/documentation/bookmark/inspiration.md b/documentation/bookmark/inspiration.md
index 3e5841ba7..5331f9a0b 100644
--- a/documentation/bookmark/inspiration.md
+++ b/documentation/bookmark/inspiration.md
@@ -1,3 +1,11 @@
+# Design
+
+1. [Good Design is Imperfect Design Part 1: Honest Names](https://www.domainlanguage.com/articles/good-design-is-imperfect-design-part-1-honest-names/)
+
+# Programming
+
+1. [Reflections on 10,000 Hours of Programming](https://matt-rickard.com/reflections-on-10-000-hours-of-programming/)
+
 # Feature
 
 1. [About Adding Features (part 1)](https://kele.codes/2021/07/about-adding-features-1/)
diff --git a/documentation/bookmark/machine_learning/probabilistic_soft_logic.md b/documentation/bookmark/machine_learning/probabilistic_soft_logic.md
new file mode 100644
index 000000000..8ded7ebf1
--- /dev/null
+++ b/documentation/bookmark/machine_learning/probabilistic_soft_logic.md
@@ -0,0 +1,4 @@
+# Reference
+
+1. [Probabilistic soft logic (PSL)](https://psl.linqs.org/)
+
diff --git a/documentation/bookmark/platform/jvm.md b/documentation/bookmark/platform/jvm.md
index f804b80e2..4ea87de28 100644
--- a/documentation/bookmark/platform/jvm.md
+++ b/documentation/bookmark/platform/jvm.md
@@ -1,5 +1,6 @@
 # Reference
 
+1. [Journey to the Centre of the JVM — Daniel Spiewak](https://www.youtube.com/watch?v=EFkpmFt61Jo)
 1. [String concatenation, redux](https://cl4es.github.io/2019/05/14/String-Concat-Redux.html)
 1. [Beware of computation in static initializer](https://pangin.pro/posts/computation-in-static-initializer)
 1. [JVM Internals](https://blog.jamesdbloom.com/JVMInternals.html)
diff --git a/documentation/bookmark/rendering.md b/documentation/bookmark/rendering.md
index 4222fea4b..793367868 100644
--- a/documentation/bookmark/rendering.md
+++ b/documentation/bookmark/rendering.md
@@ -1,3 +1,8 @@
+# Ray Tracing
+
+1. [Ray Tracing Gems II Available Today as Free Digital Download](https://developer.nvidia.com/blog/ray-tracing-gems-ii-available-today-as-free-digital-download/)
+1. [Ray Tracing Gems II](https://link.springer.com/book/10.1007/978-1-4842-7185-8)
+
 # Physically Based Rendering
 
 1. [Physically Based Rendering](https://www.pbr-book.org/3ed-2018/contents)
diff --git a/documentation/bookmark/security.md b/documentation/bookmark/security.md
index 620c37e1f..524d1651c 100644
--- a/documentation/bookmark/security.md
+++ b/documentation/bookmark/security.md
@@ -63,6 +63,7 @@
 
 # Vulnerability
 
+1. [SAML is insecure by design](https://joonas.fi/2021/08/saml-is-insecure-by-design/)
 1. [Against Cipher Agility in Cryptography Protocols](https://paragonie.com/blog/2019/10/against-agility-in-cryptography-protocols)
 1. [Padding the struct: How a compiler optimization can disclose stack memory](https://www.nccgroup.trust/us/about-us/newsroom-and-events/blog/2019/october/padding-the-struct-how-a-compiler-optimization-can-disclose-stack-memory/)
 1. [PCG generators are easily “crackable”](https://news.ycombinator.com/item?id=21475210)
diff --git a/documentation/bookmark/security/programming.md b/documentation/bookmark/security/programming.md
new file mode 100644
index 000000000..75cb2fb4e
--- /dev/null
+++ b/documentation/bookmark/security/programming.md
@@ -0,0 +1,4 @@
+# Reference
+
+1. [Secure Programming in C](https://web.mit.edu/6.s096/www/lecture/lecture03/secure-C.pdf)
+
diff --git a/documentation/bookmark/user_interface/color.md b/documentation/bookmark/user_interface/color.md
index 80595d1b1..a4de0c8e3 100644
--- a/documentation/bookmark/user_interface/color.md
+++ b/documentation/bookmark/user_interface/color.md
@@ -1,5 +1,7 @@
 # Reference
 
+1. [Building a color scheme: A foundational overview of how to establish a dynamic and configurable color scheme](https://web.dev/building-a-color-scheme/)
+1. [Web-safe colors](https://en.wikipedia.org/wiki/Web_colors#Web-safe_colors)
 1. [CSS Gradient Fixer](https://www.da.vidbuchanan.co.uk/widgets/css-gradient-fixer/)
 1. [Towards richer colors on the Web](https://darker.ink/writings/Towards-richer-colors-on-the-Web)
 1. [How software gets color wrong](https://bottosson.github.io/posts/colorwrong/)