Breaking Apart the MVC Application

So far we have built a prototype MVC application for an Address Book using Grails 3 and Apache Camel. We first started the Grails 3 application using scaffolding of our Domain Classes. We later found that with Apache Camel are our publicly facing RESTful route on port 7777, we could still leverage the Grails scaffolding as a development space on port 8080. Now we are going to break the monolithic MVC application into two micro-services, a Back-End-For-Frontend (BFF) and a core service to house the persistency of types Person, Address, and PhoneNumber. We will take our RESTful service and migrate that into a Maven based Spring Boot only application and instead of directly calling a service layer to handle our request, pass the request off to a message queue in which our original Grails application is a subscriber.

Installation of RabbitMQ

We will use RabbitMQ as our message queue. To install it, simply in your terminal type:

brew update && brew install rabbitmq

Once downloaded, you will need to open a terminal, which needs to remain open, and type:

rabbitmq-server

Creating the BFF

As already stated, we will use Spring Boot to run our BFF. The reason Spring Boot is chosen over a full Grails 3 application is because we will not need the persistency. The only thing we need to accomplish in our BFF for this application is providing the RESTful routes. So open IntelliJ, and create a new Project, selecting Spring Initializer as our template.

Once you hit Next, you will be prompted with a series of values you can update. We will only be concerned with Group, Artifact, Language, and Description

Setting the Group

The Group, or Group ID, will be our base package naming schema. Typically we want to at least name this com.saasforge. For this application, setting this to com.saasforge is adequate.

Setting the Artifact

The Artifact, or Artifact ID, will be our packages artifact. The artifact we want to describe in this case is addressbook.

Choosing the Language

I like Groovy. So let's choose Groovy for this project.

Setting the Description

Descriptions always help later when we revisit a project. Set a description that will help you fully understand the project when you look at it in a year from now. Once you are done with that, hit Next

Picking Dependencies

After you hit Next in the last wizard page, IntelliJ will prompt you for which dependencies you wish to use. Do not worry, if you miss one now, you can add them manually later. For our project we are going to use :

• Cloud Messaging -> Stream Rabbit
• I/O -> Apache Camel
• Ops -> Actuator

Once those three are selected, hit Next. This will prompt you with another screen to update the default paths. If you need to update, feel free, and when done, hit Finish

When built, open up the pom.xml file and add the following dependencies.

<dependency>
    <groupId>org.apache.camel</groupId>
    <artifactId>camel-jackson</artifactId>
    <version>2.18.1</version>
</dependency>
<dependency>
    <groupId>org.apache.camel</groupId>
    <artifactId>camel-netty4-http</artifactId>
    <version>2.18.1</version>
</dependency>
<dependency>
    <groupId>org.apache.camel</groupId>
    <artifactId>camel-swagger-java-starter</artifactId>
    <version>2.18.1</version>
</dependency>
<dependency>
    <groupId>org.apache.camel</groupId>
    <artifactId>camel-rabbitmq</artifactId>
    <version>2.18.1</version>
</dependency>
<dependency>
    <groupId>org.apache.camel</groupId>
    <artifactId>camel-gson</artifactId>
    <version>2.18.1</version>
</dependency>
<dependency>
    <groupId>com.google.code.gson</groupId>
    <artifactId>gson</artifactId>
    <version>2.8.0</version>
</dependency>

Configuring the Application

The first thing we want to do is change application.properties to application.yaml. This way we save some typing and use easy to read yaml. Once changed, add the following configuration.

# set poirt
server:
  port: 8081

# rabbitmq configs
rabbitmq:
  username: guest
  password: guest

# camel/spring
camel:
  springboot:
    main-run-controller: true

# messaging routes
addressbook:
  person:
    findAll: addressbook.person.findAll
    findOne: addressbook.person.findOne

Server Port

We are setting the server port to 8081 as not to conflict with our other local running application on port 8080. Later, when we put these applications in Docker containers, we can have both of them on port 8080.

RabbitMQ

While using RabbitMQ on localhost, we can use the username and password pair of guest.

Camel / SpringBoot

This configuration tells the Spring Boot application to remain alive for with the Camel routes. Without it, the application would start and exit.

Address Book Messaging Routes

We will only have two routes for this application. One is to find all Person and the other is to find one Person. These are not the RESTful endpoints, but rather the topics we will publish to on RabbitMQ.

Bring Over the DTOs

We will need Data Transfer Objects (DTOs) to be able to send across the messaging queue. Simply copy/paste the monolithic applications DTOs into a package named com.saasforge.utils.addressbook.dtos.

AddressDTO

package com.saasforge.utils.addressbook.dtos

class AddressDTO {
    Integer id
    String address
}

ListPersonDTO

package com.saasforge.utils.addressbook.dtos

class ListPersonDTO {
    Long id
    String name
}

PersonDTO

package com.saasforge.utils.addressbook.dtos

class PersonDTO {
    Long id
    String name
    List<PhoneNumberDTO> phoneNumbers
    List<AddressDTO> addresses
}

PhoneNumberDTO

package com.saasforge.utils.addressbook.dtos

class PhoneNumberDTO {
    Integer id
    String number
}

Building the RESTful Route

Building the REST routes are almost identical to what we did in the monolithic application. The major difference is that we will need to serialize the data to be sent across the queue. We will also need to tell Camel that we need to wait for a reply before continuing (blocking call). Fortunately, Camel handles that for us very well and requires minimal intervention.

First thing we need to do is build the class to house the route. Add a package com.saasforge.routes and in the package add the class PersonRoute.

Stub out the PersonRoute with the following code

package com.saasforge.routes

import com.saasforge.utils.addressbook.dtos.ListPersonDTO
import com.saasforge.utils.addressbook.dtos.PersonDTO
import org.apache.camel.ExchangePattern
import org.apache.camel.builder.RouteBuilder
import org.apache.camel.model.dataformat.JsonLibrary
import org.apache.camel.model.rest.RestBindingMode
import org.springframework.beans.factory.annotation.Value
import org.springframework.stereotype.Service

@Service('personRouteService')
class PersonRoute extends RouteBuilder {


    private final String ipaddress = "localhost"
    private final Integer port = 5672
    private final String apiVersionPath = '/v1'

    @Value('${rabbitmq.username}')
    private String username

    @Value('${rabbitmq.password}')
    private String password

    @Value('${addressbook.person.findAll}')
    private String findAllTopic

    @Value('${addressbook.person.findOne}')
    private String findOneTopic

    @Override
    void configure() throws Exception { }

}

@Service Annotation

We annotate the PersonRoute class with the @Service('personRouteService'). This tells Spring Boot that this class should be called at runtime, and because of so, Camel will then construct the route. Because we told our configuration to stay alive with Camel routes, this annotation keeps our application alive at runtime. Without it, the class will never be called and the application will never create any routes.

@Value Annotation

We use this annotation to grab the configuration values and inject them into the class.

Setting Up REST configuration

First thing we need to do is define the route variables we will be using. So inside of configure, add the following strings to your method. The findOneRoute and the findAllRoute will be our topics that we eventually publish to.

@Override
void configure() throws Exception { 
    def options = "username=${username}&password=${password}"
    def routeCore = "rabbitmq://${ipaddress}:${port}"
    def findOneRoute = "${routeCore}/${findOneTopic}?${options}"
    def findAllRoute = "${routeCore}/${findAllTopic}?${options}"

}

We need to add the same basic configuration from the last application's route to this one. Inside configure, add the following code

@Override
void configure() throws Exception { 
    ...
    ...
    ...

    restConfiguration()
        .component('netty4-http')
        .scheme('http')
        .host('0.0.0.0').port(7777)
        .contextPath('/api')
        .bindingMode(RestBindingMode.json)
        .apiContextPath("$apiVersionPath/api-doc")
        .apiProperty("api.title", "person api")
        .apiProperty("api.version", "1.0.0")
        .apiProperty("cors", "true")
        .enableCORS(true)
        .dataFormatProperty("prettyPrint", "true")

}

Building the RESTful Route

Now that we have our basics covered for the route, we need to start building out the REST route. Again, its very similar to our previous work but with some minor changes. We will comment the changes and then explain what each does.

@Override
void configure() throws Exception { 
    ...
    ...
    ...
    rest("${apiVersionPath}/person")
        .consumes('application/json')
        .produces('application/json')

        .get()
            .description('get a list of person')
            .type(PersonDTO.class)
            .outType(ListPersonDTO.class)
            .responseMessage()
                .code(200).message("Success")
            .endResponseMessage()
            .route()

                // set the exchange pattern to in-out
                .setExchangePattern(ExchangePattern.InOut)

                .to('log:person-findall?level=INFO&showAll=false&multiline=true')

                // publish to message queue topic
                .to(findAllRoute)

                // transform JSON into ListPersonDTO
                .unmarshal().json(JsonLibrary.Jackson)

        .endRest()

        .get("/{id}")
            .description('get a person via id')
            .type(PersonDTO.class)
            .outType(PersonDTO.class)
            .responseMessage()
                .code(200)
                .message("Success")
            .endResponseMessage()
            .responseMessage()
                .code(404)
                .message("Person not found")
            .endResponseMessage()
            .route()

                // set the exchange pattern to in-ouot
                .setExchangePattern(ExchangePattern.InOut)

                // transform our URL param into a PersonDTO
                .bean(this, 'craftPersonDTO(${header.id})')

                // transform our PersonDTO into JSON
                .marshal().json(JsonLibrary.Jackson)


                .to('log:persons-findall?level=INFO&showAll=false&multiline=true')

                // publish to the message queue topic
                .to(findOneRoute)

                // transform our response from JSON to PersonDTO
                .unmarshal().json(JsonLibrary.Jackson)
        .endRest()
}

Setting the Exchange Pattern

When Camel publishes to an endpoint, RabbitMQ in this case, without telling Camel to wait for a response, it will accept the ACK as a signal to continue. By using .setExchangePattern(ExchangePattern.InOut), Camel will instead wait for the actual data response instead. It still uses the ACK to understand to wait, so if the ACK isn't received, it will treat it as a failure. By default, Camel will wait 20 second for the data response. If not received, it will timeout, throwing an exception. So when we use to(findAllRoute), we are publishing a DTO transformed into JSON to RabbitMQ. On the other end of the queue is at least one subscriber who will act upon the data. Once that subscriber has completed the work it does, it will reply with the data packet as JSON. When this Camel route receives the data, it will update its message body and continue onto the next instruction. So setting the exchange pattern to ExchangePattern.InOut effectively makes the call to another service a blocking call.

Publishing to RabbitMQ

Camel's to DSL method is a service activator pattern. You can think of it as a method call you would use in a class, such that the parameter passed to the method is the body of the message and the return value is the updated message. In our case, we use the to DSL method to activate the RabbitMQ message queue.

RabbitMQ Endpoint

We initially constructed two route topics, findOneRoute and findAllRoute. Each route uses the same core route and same options. This may differ in production, but for our exercise, it suffices. What differs between the two is the topic they publish to. findOneRoute publishes to a topic named addressbook.person.findOne which we injected with the @Value('${addressbook.person.findOne}') annotation and findAllRoute publishes to a topic named addressbook.person.findAll which we injected with the @Value('${addressbook.person.findAll}') annotation.

Marshaling Data

Apache Camel does a wonderful job doing type conversion for us. We will need to occasionally tell Camel when to do conversions though.

GET /person

In our GET /person REST route, we do not have any payload coming in from the client. Calling the route simply activates a request for a list of Person. We decided that the DTO type returned to the client is a ListPersonDTO. Because we set the outtype to that class, as the payload returns from RabbitMQ we can use the unmarshal().json(JsonLibrary.Jackson) DSL to convert JSON to ListPersonDTO quite simply.

GET /person/{id}

In this route, we have a url with Person id the client is fetching. Because of so, we will need to call upon a bean to construct our DTO, which we will cover how that works next, but since our message body is of type PersonDTO, we need to marshal that into JSON before publishing onto the queue. Simply calling .marshal().json(JsonLibrary.Jackson) allows that to happen. Now when we publish to the queue, the message body will be serializable JSON.

Conversely, as the payload is returned from the queue, we will unmarshal it into our outtype by simply calling .unmarshal().json(JsonLibrary.Jackson).

JsonLibrary.Jackson

This parameter passed is just one of many JSON libraries afforded to us in Camel. We could have easily just passed JsonLibrary.Gson if we so chosen. We would however, have to update our dependencies to accommodate for the library.

Modify Our Original MVC Application into a Service

As you should be able to tell, we extrapolated the RESTful features of our MVC application into the AdressBook-BFF. We will need to now remove those publicly facing components from our MVC application and have it subscribe to our new topics, addressbook.person.findOne and addressbook.person.findAll.

Configuration

We will need to configure the application to use RabbitMQ as we did in the BFF. First thing to do, is modify the grails-app/conf/application.yml file. Add the following configuration to the end.

# rabbitmq configs
rabbitmq:
  username: guest
  password: guest

# messaging routes
addressbook:
  person:
    findAll: addressbook.person.findAll
    findOne: addressbook.person.findOne

Then we will need to inject those properties into our service. In PersonRouteService, add the following

class PersonRouteService extends RouteBuilder {

    private final String ipaddress = "localhost"
    private final Integer port = 5672
    private final String apiVersionPath = '/v1'

    @Value('${rabbitmq.username}')
    private String username

    @Value('${rabbitmq.password}')
    private String password

    @Value('${addressbook.person.findAll}')
    private String findAllTopic

    @Value('${addressbook.person.findOne}')
    private String findOneTopic

    ...
}

Updating the Camel Routes

In our original MVC application, the Camel routes were publicly facing RESTful routes. With our update, we will make these routes into subscribers to RabbitMQ topics. The easiest way to handle this is to empty the configure method.

class PersonRouteService extends RouteBuilder {
    ...

    @Override
    public void configure throws Exception {

    }
}

With an empty method, let's add our topic strings in the same manner as we did with the BFF

class PersonRouteService extends RouteBuilder {
    ...

    @Override
    public void configure throws Exception {
       def options = "username=${username}&password=${password}"
       def routeCore = "rabbitmq://${ipaddress}:${port}"
       def findOneRoute = "${routeCore}/${findOneTopic}?${options}"
       def findAllRoute = "${routeCore}/${findAllTopic}?${options}" 
    }
}

This sets up the ability to subscribe. What will will differ greatly now is how we attach our routes as subsribers.

FindOne Subscriber

Our MVC application is no longer a MVC application. While it still has the scaffolded routes through port 8080, we have removed our publicly facing RESTful routes that we expect the clients to use. We have moved them into the BFF and the BFF will now publish the request to RabbitMQ topics. Our first topic is addressbook.person.findOne. For our application to act upon requests coming in on that topic, we need to use the RabbitMQ endpoint as a consumer. This is done with the from DSL method.

class PersonRouteService extends RouteBuilder {
    ...

    @Override
    public void configure throws Exception {
       def options = "username=${username}&password=${password}"
       def routeCore = "rabbitmq://${ipaddress}:${port}"
       def findOneRoute = "${routeCore}/${findOneTopic}?${options}"
       def findAllRoute = "${routeCore}/${findAllTopic}?${options}" 

       from(findOneRoute)
           .unmarshal().json(JsonLibrary.Jackson, PersonDTO.class)
           .to('log:persons-findall?level=INFO&showAll=false&multiline=true')

    }
}

When we call the from DSL method, we have attached our route to listen to the given topic. As the the message comes in, we know that we serialized JSON for the message queue. This means we need to unserialize it and transform the JSON into a PersonDTO. Because we do not have automatic type conversions configured here, we will use the overloaded json method that accepts the JsonLibrary and the coversion type PersonDTO. As the message body makes it to the service activator (logging in this case), the body will have been transformed into the PersonDTO.

Our next step is to use the service activator pattern to call upon a bean that houses our findOne method. This is where we re-attach Camel back into Grails and hence follow the Grails pattern of calling the service, PersonService.

class PersonRouteService extends RouteBuilder {
    ...
    ...
    ...

    @Override
    public void configure throws Exception {
       def options = "username=${username}&password=${password}"
       def routeCore = "rabbitmq://${ipaddress}:${port}"
       def findOneRoute = "${routeCore}/${findOneTopic}?${options}"
       def findAllRoute = "${routeCore}/${findAllTopic}?${options}" 

       from(findOneRoute)
           .unmarshal().json(JsonLibrary.Jackson, PersonDTO.class)
           .to('log:persons-findall?level=INFO&showAll=false&multiline=true')
           .to('bean:personService?method=findOne(${body})')
           .marshal().json(JsonLibrary.Jackson)
    }
}

In our second service activator pattern we use the parameter bean:personService?method=findOne(${body})'). We know from our previous work when first constructing our MVC application, that this service does the heavy lifting, removing the work performed inside the controller. We will simply leverage that to delegate our ESB work, keeping our code clean and concise.

Finally, because our findOne service method returns a PersonDTO fully populated with id, name, addresses, and phoneNumbers, we simply need to now marshal that dto into JSON, which will be automatically returned to the awaiting BFF.

FindAll Subscriber

Very similar to the previous work, we need to use the RabbitMQ endpoint in consumer mode. Again, this means we need to use the from DSL method, passing it findAllRoute.

class PersonRouteService extends RouteBuilder {
    ...
    ...
    ...

    @Override
    public void configure throws Exception {
       def options = "username=${username}&password=${password}"
       def routeCore = "rabbitmq://${ipaddress}:${port}"
       def findOneRoute = "${routeCore}/${findOneTopic}?${options}"
       def findAllRoute = "${routeCore}/${findAllTopic}?${options}" 

       ...
       ...
       ...

       from(findAllRoute)
           .to('log:person-findall?level=INFO&showAll=false&multiline=true')
           .to('bean:personService?method=findAll()')
           .marshal().json(JsonLibrary.Jackson)
    }
}

We do not need to unmarshal this payload since there is not payload, just a request. Again we call upon the service activator pattern to delegate our work to our existing PersonService findAll method. As soon as that bean has completed, it will return the ListPersonDTO, and we simply need to marshal that as JSON.

Running the Distributed Application

We now have two services, our BFF and our core AddressBook. We need to tie them together with RabbitMQ. So first, run RabbitMQ, if not already running, from your terminal

rabbitmq-server

Next, run your BFF. From your addressbook-bff directory, from another terminal, type

mvn clean spring-boot:run

And finally, from a third terminal, inside your addressbook directory, type

grails dev run-app

If all goes well, you should be able to access your application from http://localhost:8080/api/v1/person and http://localhost:8080/api/v1/person/1. Of course, since we have stopped the previous running of our application, if you did not use the BootStrap to add a development user, the in-memory database will be empty, so you will need to use our scaffolded application at http://localhost:8080 to add a Person, Address, and PhoneNumber.

Conclusion

This finishes breaking our monolithic application into 2 distinct parts. While this may seem like a lot of work, realize it affords us the opportunity to scale our BFF or our AddressBook service. Imagine a different BFF needed to access the AddressBook service as well, we could simply create the BFF reusing our core micro-service. Or if we wanted to scale our AddressBook service, we could easily do that without interfering with our BFF. A third option would be to scale our BFF to also access other bits of data not related to the AddressBook but use data found in another micro-service, we could simply add routes that publish to many services and aggregate data how we see fit.

By disconnecting the monolithic MVC into smaller, more manageable parts, we set up ourselves for success, but hopefully you noticed that having to manage all the smaller services can become a nightmare in its own right. That is why we will continue with the next article dockerizing the parts and delegating management to Kubernetes, OpenShift, and Fabric8.