Logging In Effects
When the word Logging is mentioned, many people immediately think about de-forestation but if you are a computer programmer this word has an entirely different meaning. In the world of computers, logging is a quintessential activity because you allow others a glimpse into this blackbox via the log files via the log generators.
The world’s first professional log generator was also known as the
System.out.println if you are (or were) a Java programmer. This programming
language statement allowed the programmer to be able to tell the reader of the
running program what exactly is going on when it is executing (without
inspecting the computer program, line by line ☺). You might not be
surprised that entire libraries (for Java, for Python) have been devoted to analysing log
files and other data spewed by log generators. The science of logging, if i can
call it that, has the unremarkable appearance of being heuristic (or guesswork) and
the evidence it offers is when you run into conversations like the following :
2 ficticious programmers : John and Don
John: The system crashed
Don : Do you know how it crashed ?
John: Nope, but i'll insert some log statements and let's restart this ...
...
Don : It crashed again. Did the logs reveal anything?
John: (browsing thru the logs ... nothing) Some parts of the program is not
covered by the log statements, i'll add that and let's check it out ..
...
Don : It crashed again (frustration starting to rise). There MUST be something
in the logs right?
John: Yeah i saw it .. the system crashed between lines 2234 and 2236 while
executing this function call at line 2235 (Silence ensues ...)
Logging, to me, remains an art form and our industry have come a long way by inventing conventions by focusing on what to log, how to log and finally giving appropriate tags (e.g. DEBUG, WARNING, ERROR) to alert the human user and drive more detailed insights by interpreting the collected data logs. There are dozens of businesses that specialised around this idea like Splunk, Logstash (its really a technology), Datadog (this is a company).
The libraries of 2020 are very good (i suspect they are likely to get better) and they allow the developer to write modular functions and allow them to be re-used in the computer programs but a thing that made me wonder (for a long time, i’d admit) is how much compositionality do these libraries offer? Composing functions are a norm, for functional programmers (be it you are into Haskell, Scala, OCaml, Clojure etc) and this question comes naturally. When i was working with Java at the earlier years, logging libraries were kinda meh and the idioms are largely well understood, you can still get away with logging as a low-level art form in the cats but once you programmed in Haskell, the expectations for compositionality not only rose and i wished we had a higher level of abstraction to work with instead of writing our own.
There are a lot of libraries and frameworks in Haskell in which you can leverage to build systems that interact with various external systems and when you do that (you will, eventually) you would have to choose some kind of Effects library and in 2020, there are two primary ones i’ve experimented:
Fused-effects is a really cool performant effects library but its not the Gfocus for this post, i’m also not going to discuss Polysemy either but the thing i do want to show you a little more is about co-log and its derivatives like co-log-polysemy
Main ideas of Colog
This is the part where i direct you to go off and take some time to read Dimitrii Kovanikov’s excellent introduction to the topic over here. You might want to go over the introductory posts on Github here and here.
Learning By Working Through An Example
Once you have read about the main ideas of Co-log, come back here and i’ll work out an example. As always, i’ve chosen a incredibly small problem to illustrate how it works.
The ficitious problem is here : Given a list (finite, of course) of integers, i want to be able to detect odd and even integers and log them to the standard output.
First, let’s take a look at how we can use co-log. Co-log builds on the data
type LogAction m msg and what its saying is that you need to figure which
monad m you would like the logging to happen and the data input’s type msg.
Here’s how i would define a logger that logs within the IO monad:
-- This is how you can craft your own logger
logToStdout :: LogAction IO String
logToStdout = LogAction putStrLn
-- I prefer the following definition since we allow the input data type 'a' to
-- implement the `Show` typeclass.
logToStdout2 :: Show a => LogAction IO a
logToStdout2 = LogAction print
When i load these 2 functions into GHC repl and ran it (just to make sure
things are basically okay) via the unLogAction, i get the following output:
> unLogAction logToStdout "2"
> 2
> unLogAction logToStdout2 2 -- hopefully you see why this definition is my preferred one
> 2
Admittedly, the de-constructor unLogAction is going to be a little clunky as
you move along as it becomes unwieldy when you write bigger expressions and the
meaning of this complex expression is likely lost. The nicest thing about the
co-log-core (base) library is that it provides shortcuts (You get this in the co-log library as well);
there’re a tonne of things we can do with them and i’ll get into a little later but if you want to know
what they are, they are \&> and <\&.
Combinators (Which is building higher-order functions using smaller functions)
Once the logger(s) have been defined, the next thing is to create suitable
combinators that allows me to re-use the previously defined functions and the
thing i’m interested to use is the cfilter and cfilterM. Let’s take a look
at their type signatures to see what they do:
cfilter :: Applicative m => (msg -> Bool) -> LogAction m msg -> LogAction m msg
cfilterM :: Monad m => (msg -> m Bool) -> LogAction m msg -> LogAction m msg
Now, it should be clear that it consumes a predicate function that allows the
runtime to evaluate it and depending on the truth level it would execute the
associated logger. What is not evidently clear from here is that LogActions
can be combined as loggers in co-log are both Semigroups and Monoids. We’ll
take a look at how they can be combined.
The next thing to do is to craft the desired combinators and this time, i’m not
using the unLogAction and instead leverage the shorter symbol &> and once
again the type signature gives a hint on how this symbol operator can be used:
(&>) :: msg -> LogAction m msg -> m () | infix 5 |
The final function for this simple exercise is to create a combinator that allows me to solve the exercise.
-- | Determines whether a number is even
onlyEven :: Int -> Bool
onlyEven x = even x
-- | Determines whether a number is odd
onlyOdd :: Int -> Bool
onlyOdd x = not . even $ x
-- | Customized loggers
logE = LogAction (\f -> print ("Even => " ++ show f))
logO = LogAction (\f -> print ("Odd => " ++ show f))
-- | My first combinator which iterates through a finite list, looking for odd
-- numbers
discoverOddOnly = forM datum (\x -> x &> cfilter onlyOdd logO)
-- | My first combinator which iterates through a finite list, looking for even
-- numbers
discoverEvenOnly = forM datum (\x -> x &> cfilter onlyEven logE)
-- | Our fictitous list
datum = [1..10]
-- | foldActions basically folds through the Foldable structure and creates a
-- single (mega, loosely speaking) logger
discoverBothOddEven =
forM datum (\x -> x &> foldActions [cfilter onlyOdd logO, cfilter onlyEven logE])
The other aspect about this logging framework is really about leveraging the contravariant aspect but i won’t go through it in this post since the API documentation has done a respectable job here; the example given in that API documentation is likely going to be very useful for you
So, the co-log-core has other combinators that i suspect you’ll love which
provides you much versatility and there’s a good chance you will grow to rely
and depend on it. The other package, co-log builds on co-log-core which
gives you other loggers and combinators which i found enjoyable ☺ which brings
me to the final “part” of this short post.
When it comes to logging, one activity that will almost come up is the ability
to flush the data i care about to files; a nicety about co-log-core is the
fact that it allows a developer to be able to plug-in formatters of your
choice (the hint is given in the contravariant support given by this
library).
A standard file logger which is provided by the co-log-core library is the
withLogStringFile which no longer abstracts the type of the monad it operates in and
in fact the actual type is the MonadIO (that’s fancy speak for monads in
which IO actions can be embedded in). The code snippets i’m going to show you
is simple and its likely going to be hidden behind some substrate of your
choice since its pretty much boiler-plate.
-- | The functions 'logger' and 'flushToLog' are
-- pretty much boiler plate.
--
logger :: LogAction m msg -> msg -> m ()
logger action d = d &> action
flushToLog :: FilePath -> String -> IO ()
flushToLog filePath d = withLogStringFile filePath (flip logger $ d)
-- | This is an obvious way to pipe "even" numbers to one log and "odd"
-- numbers to another.
sieveAndFlushToFile lLog rLog = forM [1..100] (\x -> if onlyEven x
then flushToLog lLog (show x)
else flushToLog rLog (show x))
When sieveAndFlushToFile was ran with the arguments of even.log and
odd.log respectively, you would find that the data is output as normal and
what is great about it is that if i ever wished to add more decorations or
embellishments to the datum that’s about to the spewed to an external file,
what i needed to do is to inspect the logger function and add whatever i
needed. Here’s an example of this below and the interesting thing here is to
keep in mind that the continuation function actually has a IO handle to the
file that is pointing to which means that all the transformations to the data
should be done prior.
logger :: (Monad m) => LogAction m String -> String -> m ()
logger action d = if (even . read $ d) then "Even => " &> action >> d &> action
else "Odd => " &> action >> d &> action
References :
- co-log-core light weight package with basic data types and generally depends on
base - co-log tagless final implementation of logging library based on the
co-log-core