PBDirect – Protobuf without the .proto files

Protocol Buffer (aka Protobuf) is an efficient and fast way to serialise data into a binary format. It is much more compact than Java serialisation or any text-based format (Json, XML, CSV, …).

Protobuf is schema based – it needs a description (in a .proto file) of the data structures to be serialised/deserialised.

On the JVM, protoc (the Protobuf compiler) reads the .proto description files and generates corresponding classes.
For Scala there is a very good sbt plugin “scalaPB” that follows the same process and generates case classes corresponding to the .proto files definitions.

The .proto files are an easy way to describe a protocol between 2 components (e.g. services). However there are some cases (e.g. writing to persistent storage) where the .proto files definition are just unnecessary and add superfluous complexity. (Who likes to read auto-generated code?).

In such cases it would be much easier to serialise an object directly into protobuf (using its class definition as a schema). Afterall this is what the protobuf java binding does: it serialises (auto-generated) java classes into protobuf binary format.

To that matter, let me introduce – PBDirect – a scala library to directly encode scala objects into protobuf. Continue reading “PBDirect – Protobuf without the .proto files”

Rethinking logging on the JVM with Logoon

Logging has been around on the JVM for a while now. It all started with Log4J back in 2001. Log4J was the first logging framework and it is still around today (in its version 2). It provides a simple and efficient API (compare to System.out.println that was in use before).

  1. Get a logger for a class
  2. Use that logger to log messages
val logger = Logger.getLogger(classOf[MyClass])
...
logger.log(Level.DEBUG, "I am doing something right now")
...
logger.error("Oops, something went wrong", theException)
...

Today there are a few more frameworks on the JVM but they all provide similar APIs as Log4J:

  • JUL(2002): java.util.logging provides a standardisation of Log4J and of course provides a similar API
  • Commons-logging (2002): Apache project providing a façade over Log4J, JUL, … still the same API
  • SLF4J (2005): Another façade over Log4J (1&2), JUL, JCL, … no much changes in the API
  • Logback (2006): Brings structured logging with an API compatible (and similar) to SLF4J (and Log4J)
  • Log4J2 (2012): Rewrite of Log4J inspired by Log4J and Logback with improved performances. The API does not change much though.

As you can see the logging APIs available on the JVM haven’t changed much over the last 15 years. The most interesting additions are structured logging and the Mapped Dependent Context (MDC) as we shall see later.

In this post I am going to look at the current limitations of these APIs and see how we can overcome them while still relying on this frameworks to actually write the logs. Continue reading “Rethinking logging on the JVM with Logoon”

Fluent – A deep dive into Shapeless and implicit resolution

As promised in my previous post we’re going to explore to internal of Fluent and how it uses Shapeless and implicit resolution to transform case classes.

Fluent started as an experiment (and still is), the code is rather small (about 300 lines of code) and yet I am still impressed by the variety of cases it can handle.

Before working with Shapeless I’ve often heard that is pure magic and I got the impression that most people (including me) don’t really know how it works. It turns out that the principles used in Shapeless are not really difficult to understand – especially if you read the well-written Type Astronaut’s guide to Shapeless.

Understanding how Shapeless works doesn’t mean it’s easy to work with. Actually Shapeless makes a heavy use of implicits and working with implicits is hard. Remember that implicits resolution is performed at compile time so when it fails, there is nothing to debug, no log messages or stack trace. We are just left with rather blunt messages like could not find implicit value for parameter ...

In this post I am going to explain the concept used in Fluent, the problem I faced during implementation and hopefully by the end of the post, you’ll know enough to understand and edit the code (Pull requests welcomed!). Continue reading “Fluent – A deep dive into Shapeless and implicit resolution”

Introducing Fluent – the seamless translation layer

In Domain Driven Design (DDD) it is recommended to introduce a translation layer (aka anticorruption layer) between 2 bounded contexts. The role of the anticorruption layer is to avoid any concepts to leak from one domain into the other.

This is a sound idea as it keeps the domains isolated from each other ensuring they can evolve independently. After having implemented several anticorruption layers I realised that, although useful, they also introduced a lot of boilerplate code that doesn’t add much value to the business.

To this extent, let me introduce Fluent, a library that aims at getting rid of this boilerplate code by leveraging all the power of Shapeless and its generic programming. Continue reading “Introducing Fluent – the seamless translation layer”

Reducing type class boilerplate with Shapeless

If you followed our previous post on forging a DSL using type classes, you surely notice that writing type class instances is a rather repetitive task.

In today’s post we’re going to get rid of this by derivating type class instances automatically using Shapeless. We also use this post as an excuse to experiment with Shapeless and try to understand all the “magic” that’s happening. Continue reading “Reducing type class boilerplate with Shapeless”

Forging a DSL using Scala type classes

In this post we’re going to explore how to build a DSL (Domain Specific Language) with a user-friendly syntax while maintaining as much type-safety as possible. We want that any operations that is not allowed by the business rules fail at compile time. This would be really nice as it makes sure that no one writes such forbidden logic (even by mistake).

More over Scala provides really nice syntactic sugar that can make a DSL syntax pretty neat.

If you don’t know what type classes or don’t feel very comfortable with this concept, follow along as we’ll also explore how we can use them to dissociate data and behaviours (always a good practice). Continue reading “Forging a DSL using Scala type classes”

The free monad without the boilerplate

The free monad is really neat for creating DSL as it allows to completely separate the business logic from the implementation.

It leaves a great freedom for the implementation choices and should you change your implementation you just need to rewrite your interpreter without changing any of the business logic.

That’s great! However after playing a little around with the free monad I was a bit skeptical about the amount of boilerplate required to plug everything together. Writing smart constructors and injectors is not the most trivial thing (depending on how your team is familiar with functional programming).

To conclude this serie on the free monad we’ll have a look at some solutions to remove this boilerplate code, especially freasy and freek.
Continue reading “The free monad without the boilerplate”

The free monad in (almost) real life

Now that we have a fairly good understanding of what the free monad is and how it works. Let’s see what it looks like to write a real program based on this concept. This time we won’t implement Free ourselves but use the cats library for that. If you don’t know all the theory behind the free monad and just want to see how to use it then read on as this post builds an application from scratch.

In this post we’ll write simple DSLs, lift them into the free monad and even combine them into one program. There is some boiler plate around but I want to get a feeling of what it looks like to combine more DSLs and see how we can architecture a whole application around the free monad. Continue reading “The free monad in (almost) real life”

The Free monad explained – part 2

Remember in part 1 we implemented a little program to interact with the user. We nicely separated the business logic from the implementation. And as good as it is we now need to add more functionality to our application and this exactly what we’re going to cover here.

The idea is to define a new language for user authentication and then combine it with the user interaction language to build a more complex program.
Continue reading “The Free monad explained – part 2”

The Free monad explained – part 1

My next series of blog posts will be on the Free monad.  Don’t be scared by the name, it’s not so difficult to grasp it (if explains correctly) and your code could greatly improve.

Why ? Because it allows a clear separation between the program definition (i.e. the business logic) and its implementation. That means if one day you decide to change your implementation (e.g change your persistent storage) you just need to re-write your interpreter – leaving the business logic untouched.
Continue reading “The Free monad explained – part 1”