import groovy.transform.Canonical
import groovy.transform.CompileStatic
import org.springframework.http.HttpStatus
import org.springframework.http.ResponseEntity
import org.springframework.web.bind.annotation.GetMapping
import org.springframework.web.bind.annotation.PostMapping
import org.springframework.web.bind.annotation.RequestBody
import org.springframework.web.bind.annotation.RestController
@CompileStatic @RestController
class NotificationController {
private final List<Notification> notifications = []
@PostMapping('/notifications')
ResponseEntity<Void> notify(@RequestBody final Notification notification) {
notifications << notification
new ResponseEntity<Void>(HttpStatus.OK)
}
@GetMapping('/notifications')
List<Notification> list() {
notifications
}
}
@CompileStatic @Canonical
class Notification {
String project
String version
String message
}
Making HTTP Requests from your Build
15 October 2017 ~ groovy gradle
I recently released a Gradle plugin to assist in making HTTP calls from your Gradle build. The HTTP Plugin uses the HttpBuilder-NG library’s clean DSL to configure the calls as Gradle tasks.
|
Note
|
This post assumes some familiarity with the HttpBuilder-NG library. If you have never used the library, I recommend reading through the User Guide or my blog post Take a REST with HttpBuilder-NG and Ersatz to give you a good overview of its functionality. |
As an example, let’s say we want to send release build notifications to some internal notification server. For this example I have created a simple Spring-Boot application with the following controller:
NotificationController.groovy
We see that the controller will accept POST calls to /notifications to submit notifications. It will also provide a list of all notifications at the GET /notifications end point. I am going to omit the Spring-Boot project itself since it’s simply the controller above in a basic generated project. Just assume that the server is running in the background (with gradle bootRun).
To apply the HTTP Plugin we add the following to the top of the build.gradle file:
build.gradle
plugins {
id 'io.github.http-builder-ng.http-plugin' version '0.1.0'
}And then we need to create a task which will post build release notifications to our server:
build.gradle
task notify(type:io.github.httpbuilderng.http.HttpTask){
config {
request.uri = 'http://localhost:8080'
}
post {
request.uri.path = '/notifications'
request.contentType = 'application/json'
request.body = [
project: project.name,
version: project.version,
message: 'Build'
]
response.success {
logger.info 'Notification succeeded.'
}
response.failure { fs, obj->
logger.warn "Notification failed (${fs.statusCode}: ${fs.message})"
}
}
}Notice that the task is of type io.github.httpbuilderng.http.HttpTask which is provided by the plugin. The config block contains the client configuration (analogous to the configure method of the HttpBuilder implementation). The post block configures the actual request to be made. The HttpTask interface supports the GET, HEAD, POST, PUT and PATCH request methods and allows for multiple requests to be called from a single task either synchronously or asynchronously. In our example above we are sending a POST request to http://localhost:8080/notifications with the project name, version and a simple message. If the notification is successful we will see the "Notification succeeded." message in the --info logging. A failure will produce a warning message with status information.
Run the task with gradle notify with the server running. Then you can hit the http://localhost:8080/notifications end point to see a JSON list of the submitted notifications.
|
Tip
|
The HttpTask does not use or collect the responses in any manner. If a response needs to be acted in it must be processed in the response handler methods provided by the HttpBuilder-NG Response interface (as the success and failure handlers in the example). |
Now, maybe we would like to provide a means of listing the notifications from our build. Since we are adding a second task with the same client configuration, we can extract the shared configuration out using the HttpExtension provided by the plugin:
build.gradle
http {
config {
request.uri = 'http://localhost:8080'
}
}This means we can remove the config block from the notify task. Our new task to list the notifications will look like:
build.gradle
task notifications(type:io.github.httpbuilderng.http.HttpTask){
get {
request.uri.path = '/notifications'
response.success { fs, obj->
println "Notifications List"
obj.each { notif->
println " - ${notif.project} (v${notif.version}): ${notif.message}"
}
}
}
}Notice that we omitted the config block here since it we defined it in the http extension. This task makes a GET request to the /notifications end point and prints out the list of notifications (the JSON content is parsed by default). When this task is run with gradle notifications (and assuming we have run the notify task) you will see output similar to the following:
:notifications Notifications List - demo (v0.0.1): Build - demo (v0.0.1): Build BUILD SUCCESSFUL
To this point we have been using the CORE client library provided by HttpBuilder-NG, which is based on the HttpUrlConnection object in the core Java library, but what if we want to use the Apache HttpComponents library as our client? HttpBuilder-NG supports this and so does the plugin. If we add the library = 'apache' line to the http extension block we will start using the Apache client for all of our requests. The OkHttp client library is also supported.
As a last bit of functionality, let’s add the ability to make the notification message more interesting. We can replace the:
message: 'Build'line in the notify task definition with the following line:
message: project.hasProperty('notification-message') ? project.property('notification-message') : 'Build'which allows us to send a notification with a more interesting message, for example:
gradle notify -Pnotification-message="Now with more message!"
That is the HTTP Plugin. It is a new project and making HTTP calls from a build is probably a bit of an edge case, but when you run into it, something like this makes it a lot easier.
Take a REST with HttpBuilder-NG and Ersatz
11 September 2017 ~ groovy
This is a long post and there is a lot of code to look through. If you would rather follow along using the completed code, you can find it in its GitHub project rest-dev.
This blog post is going to be a bit more self-serving and a bit longer than my usual posts. I will be walking through the process of implementing a REST client using HttpBuilder-NG (v0.18.0) and then testing it against an Ersatz Server (v1.5.0) to mock out the endpoints.
Let’s say we work in a big company that is implementing a bunch of microservices. Our team is working on a service that will interface with a service being created by another team doing concurrent development - say they are creating an internal user management service. Our team will need to perform operations against their service before it actually exists. In discussions between the two teams, we have fleshed out a RESTful interface contract which looks something like this:
GET /users - list all users, responds with list of users
GET /users/{id} - get specific user, responds with single user
POST /users <user> - create new user, responds with created user
PUT /users/{id} <user> - update existing user, responds with updated user
DELETE /users/{id} - delete a user, 200 means success
Nothing shocking there, but now while they are developing the actual endpoints, you are developing a client. We need a way to simulate their user API in a realistic manner so we can develop with at least some level of confidence. This is one of the use cases where Ersatz Server comes in handy.
We can quickly define a mock for each of the end points and then write client code against it. First, we will need a User object. Based on our shared contract, the User looks like the following:
@Canonical
class User {
Long id
String username
String email
}Next we need to setup a Spock test which will be used to simulate the API and test our client code. A basic Spock test with an Ersatz server is shown below (if you are not familiar with Spock, I suggest reading through the docs to get a quick feel for it before moving forward):
UserClientSpec.groovy
class UserClientSpec extends Specification {
@AutoCleanup('stop')
private final ErsatzServer server = new ErsatzServer()
}This code creates our ErsatzServer instance for us and registers it to be stopped after each test method.
For our REST endpoints, we will just start from the top and implement the GET /users endpoint first.
UserClientSpec.groovy
def 'retrieveAll'() {
setup:
List<User> users = [
new User(100, 'abe', 'abe@example.com'),
new User(200, 'bob', 'bob@example.com'),
new User(300, 'chuck', 'chuck@example.com')
]
server.expectations {
get('/users').called(1).responder {
code 200
content users, APPLICATION_JSON
}
}
UserClient client = new UserClient(server.httpUrl)
when:
List<User> result = client.retrieveAll()
then:
result.size() == 3
result[0] == users[0]
result[1] == users[1]
result[2] == users[2]
and:
server.verify()
}The client method for this endpoint will be named retrieveAll() so, we will use that as the test name. We setup a few users that will be returned by the call and then configure the Ersatz expectations. The expectations are defined using a DSL to describe each expected request and then to define the response that request will return. In this case we are expecting a GET request with the path /users only once, which will return a status code of 200 and the configured list of users as a string of JSON. We then use the client object (not defined yet) to make the server call and then verify that we got our list of users back and that the server expectation was actually called.
It seems like a significant chunk of code to drop all at once, but if you read though it, it’s actually pretty straightforward.
The first problem we run into when trying to run this code is that the UserClient class does not exist yet, so let’s create that next.
UserClient.groovy
class UserClient {
private final HttpBuilder http
UserClient(final String host) {
http = HttpBuilder.configure {
request.uri = host
}
}
}We are using HttpBuilder-NG (the core client in this case) to make the HTTP calls. It also uses a DSL for configuration. In this case we define the base URI to be a host that we pass in - if you look back at the test we see that it’s the ErsatzServer host in that case. This will be the root of all requests. Now, to make our test happier, we need to implement the retrieveAll() method:
UserClient.groovy
List<User> retrieveAll() {
http.get(List) {
request.uri.path = '/users'
response.parser(JSON) { ChainedHttpConfig config, FromServer fs ->
json(config, fs).collect { x -> x as User }
}
}
}This method will make a GET request to the /users path on the configured host. Note that we also need to configure a parser to handle the incoming response data, which is a list of User objects serialized as JSON.
Now, if we go back and run our test, we get a nasty error about parsing JSON content on the Ersatz Server side:
groovy.json.JsonException: Unable to determine the current character, it is not a string, number, array, or object The current character read is 'r' with an int value of 114 Unable to determine the current character, it is not a string, number, array, or object line number 1 index number 1 [restdev.User(100, abe, abe@example.com), restdev.User(200, bob, bob@example.com), restdev.User(300, chuck, chuck@example.com)]
This means we need to add an encoder to the Ersatz Server configuration so that it knows how to encode the response it is sending back - in this case it will serialize a list of User objects as JSON to be sent as the response. We can configure this on the ErsatzServer constructor as:
UserClientSpec.groovy
@AutoCleanup('stop')
private final ErsatzServer server = new ErsatzServer({
encoder(APPLICATION_JSON, List) { input ->
"[${input.collect { i -> toJson(i) }.join(', ')}]"
}
})I just used the groovy.json.JsonOutput.toJson(Object) method for simplicity. Now, when we run the test it succeeds. At this point we have implemented and tested our client against a real endpoint. I say real because Ersatz creates an instance of an embedded Undertow server and configures the expected endpoints on it. The client code is hitting a real and standard web server with all of the expected server behavior. What you do have to be careful of with this kind of testing is that the contract with the other team does not change. This mocked testing is only as good as the configured expectations and if left unmaintained could drift far from the reality of the production endpoints - something to be aware of.
But we have other endpoints to define and clients to implement. Next, we will handle the single user retrieval case, the retrieve(long) method (GET /users/{id}). Our test for this method looks very similar to the first test:
UserClientSpec.groovy
def 'retrieve'() {
setup:
User user = new User(42, 'somebody', 'somebody@example.com')
server.expectations {
get('/users/42').called(1).responder {
code 200
content user, APPLICATION_JSON
}
}
UserClient client = new UserClient(server.httpUrl)
when:
User result = client.retrieve(42)
then:
result == user
and:
server.verify()
}Notice that in this case, we are configuring only a single user in the response. Learning from our last test, we know that we will also need to configure an encoder to handle single User objects. This one is even simpler and makes our constructor look like:
UserClientSpec.groovy
@AutoCleanup('stop')
private final ErsatzServer server = new ErsatzServer({
encoder APPLICATION_JSON, User, Encoders.json
encoder(APPLICATION_JSON, List) { input ->
"[${input.collect { i -> toJson(i) }.join(', ')}]"
}
})For the single object case we just define the default JSON encoder. Ersatz takes the stance that if you need/want encoders and decoders you need to configure them rather than having them provided out of the box. It keeps the configuration less surprising and more explicit.
The client code for the GET /users/{id} endpoint is as follows:
UserClient.groovy
User retrieve(final long userId) {
http.get(User) {
request.uri.path = "/users/${userId}"
}
}which along the same lines as our first client method, we will need to add a response parser for deserializing the incoming JSON response. We can configure shared response parsers in the main HttpBuilder.configure() method that we have in our constructor, so that they will be available to all HTTP method calls. The client constructor now looks like:
UserClient.groovy
UserClient(final String host) {
http = HttpBuilder.configure {
request.uri = host
response.parser JSON, { ChainedHttpConfig config, FromServer fs ->
json(config, fs) as User
}
}
}This uses the NativeHandlers.Parsers.json method and casts it as a User object to satisfy our object typing.
When we run our tests again, we see that they are both successful. That’s enough for the GET requests, let’s move on to something different. The POST /users <user> endpoint is tests as the others are:
UserClientSpec.groovy
def 'create'() {
setup:
User inputUser = new User(null, 'somebody', 'somebody@example.com')
User createdUser = new User(42, inputUser.username, inputUser.email)
server.expectations {
post('/users') {
called 1
body inputUser, APPLICATION_JSON
responder {
code 200
content createdUser, APPLICATION_JSON
}
}
}
UserClient client = new UserClient(server.httpUrl)
when:
User result = client.create(inputUser)
then:
result == createdUser
and:
server.verify()
}In this case we are expecting a POST method with a User as the body content, serialized as JSON. When the request is successful we respond with the user data which also includes the id. To decode the incoming request content we need to add a decoder to the ErsatzServer constructor:
UserClientSpec.groovy
@AutoCleanup('stop')
private final ErsatzServer server = new ErsatzServer({
encoder APPLICATION_JSON, User, Encoders.json
encoder(APPLICATION_JSON, List) { input ->
"[${input.collect { i -> toJson(i) }.join(', ')}]"
}
decoder(APPLICATION_JSON) { byte[] bytes, DecodingContext dc ->
Decoders.parseJson.apply(bytes, dc) as User
}
})For the most part it is just the provided JSON decoder with the result cast as a User object. Now, for our client implementation
UserClient.groovy
User create(final User user) {
http.post(User) {
request.uri.path = '/users'
request.body = user
request.contentType = JSON[0]
}
}We just use the post() method and configure the request body content, which we will need a means of encoding into the outbound JSON format. Our client constructor now becomes:
UserClient.groovy
UserClient(final String host) {
http = HttpBuilder.configure {
request.uri = host
request.encoder JSON, NativeHandlers.Encoders.&json
response.parser JSON, { ChainedHttpConfig config, FromServer fs ->
json(config, fs) as User
}
}
}For the encoder, we can use the one provided with the library. Run the tests again and we see that everything is green.
I am going to skip the description of the user update method and its test. They are basically the same as those for the create functionality. The DELETE /users/{id} endpoint provides a few different concepts, at least on the client side. We will flip the order with this one and show the client implementation first:
UserClient.groovy
boolean delete(final long userId) {
http.delete {
request.uri.path = "/users/$userId"
response.success {
true
}
response.failure {
throw new IllegalArgumentException()
}
}
}Notice the success and failure handlers used here. If you get a successful response (e.g. 200), the success handler is called, otherwise the failure handler is called. For our implementation, we want to return true if the delete is successful` and throw an IllegalArgumentException if the user was not deleted - yes, it’s a bit odd, but it shows a bit more functionality.
In order to test this method, we need to test cases:
UserClientSpec.groovy
def 'delete: successful'() {
setup:
server.expectations {
delete('/users/42').called(1).responds().code(200)
}
UserClient client = new UserClient(server.httpUrl)
when:
boolean result = client.delete(42)
then:
result
and:
server.verify()
}
def 'delete: failed'() {
setup:
server.expectations {
delete('/users/42').called(1).responds().code(500)
}
UserClient client = new UserClient(server.httpUrl)
when:
boolean result = client.delete(42)
then:
thrown(IllegalArgumentException)
!result
and:
server.verify()
}One test case tests the successful path and the other the failure case. While there is still a lot of functionality left to implement and test (e.g. more failure cases, bad input data, etc), we’ve got a good starting point and a framework for future testing.
Yes, this is a very code-rich discussion, but hopefully it was all pretty transparent about what was going on. You can find the code for both the client and the test in the rest-dev project on GitHub.
HttpBuilder-NG and Erstaz make a great team, and that’s actually somewhat by design. Ersatz is what HttpBuilder-NG uses to test its own functionality. Also, while the examples here are written in Groovy, both libraries work just as well with standard Java 8.
This post has only scratched the surface of the functionality provided by both libraries. Poke around their documentation and see what else you can do, and feature requests are always welcome.
Update: I have added a pure Java 8 implementation of the code for this post (source and tests). Yes, both libraries really do work well with Java too!
Fixed Minute Timers
03 March 2017 ~ groovy
A co-worker of mine recently complained about how our metrics reporting component wasn’t smart enough to start the reporting timer on discrete minutes, but rather it just starts reporting at the start time, for example if you start the application at 10:42:34 and report every five minutes, you will get events recorded at the following times:
10:47:34 10:52:34 10:57:34
which can be annoying for a user when you want to determine metric changes over time, especially when using a dashboard tool. While this is a critical defect, it can be a data quality annoyance and it turns out, its not all that hard to fix.
What you want to do is determine the delay between now and the next desired minute-mark, the five-minute mark in our case, and you can do this with a little math:
def delay(final int mark, final LocalTime now = LocalTime.now()){
int minute = now.minute + 1
int minuteMark = minute % mark
minuteMark = minuteMark == 0 ? minute : minute - minuteMark + mark
int hour = now.hour
if( minuteMark == 60 ){
hour++
minuteMark = 0
}
now.until(LocalTime.of(hour, minuteMark, 0, 0), ChronoUnit.MILLIS)
}where mark is your minute-mark (5), and now is either a time you pass in (mostly for testing) or the current time by default. We need to roll the minute hand forward one to account for how far we are already into that minute and then we figure out what the next minute-mark is (accounting for hour-boundary rollover). Then you determine the difference between that next interval time and the current time, returning that value.
This does not account for the time spent in the method itself, but we are working on the scale of minutes and seconds here so our calculation time should not matter.
A full example of using this to schedule a timer on the five-minute marks follows:
import java.time.*
import java.time.temporal.*
import java.util.concurrent.*
def delay(final int mark, final LocalTime now = LocalTime.now()){
int minute = now.minute + 1
int minuteMark = minute % mark
minuteMark = minuteMark == 0 ? minute : minute - minuteMark + mark
int hour = now.hour
if( minuteMark == 60 ){
hour++
minuteMark = 0
}
now.until(LocalTime.of(hour, minuteMark, 0, 0), ChronoUnit.MILLIS)
}
def now = LocalTime.now()
println "Now: $now"
int mark = 1
long delay = delay(mark,now)
println "Delay: $delay ms"
println "Start: ${now.plus(delay, ChronoUnit.MILLIS)}"
ScheduledExecutorService ses = Executors.newScheduledThreadPool(1)
ses.scheduleAtFixedRate({
println LocalTime.now()
}, delay, mark*60000, TimeUnit.MILLISECONDS)which will print out something like:
Now: 08:33:22.611 Delay: 97389 ms Start: 08:35 08:35:00.088 08:40:00.088 08:45:00.088
These nice clean time bounaries lead to cleaner reporting visualizations. You can use any minute as the mark points, but generally you would use: 1, 5, 10, 15, 20, 30, or 60.
Ignoring SSL Issues
SSL is great, but it can be a real pain to deal with in testing or when you use self-signed certificates; browsers, generally handle it gracefully after manually accepting the certificate, but APIs can be tricky. In working on HttpBuilder-NG to add the “ignore SSL issues” feature back in from the original version, I got the grand tour of how to ignore certificate issues in some modern HTTP clients.
All you need are a couple custom components, which are thankfully shared across the client implementations I will discuss. You need an all-trusting javax.net.ssl.TrustManager:
X509TrustManager allTrusting = new X509TrustManager() {
public X509Certificate[] getAcceptedIssuers() {
return new X509Certificate[]{};
}
public void checkClientTrusted(X509Certificate[] certs, String authType) {
}
public void checkServerTrusted(X509Certificate[] certs, String authType) {
}
}and an all-accepting javax.net.ssl.HostnameVerifier:
HostnameVerifier ANY_HOSTNAME = (s, sslSession) -> true;With these, we can create an javax.net.ssl.SSLContext:
SSLContext sslContext = SSLContext.getInstance("SSL");
sslContext.init(null, new TrustManager[]{allTrusting}, new SecureRandom());Now you are ready to configure your clients. For the Java core HttpsURLConnection you can inject these using:
HttpsURLConnection https = // created elsewhere
https.setHostnameVerifier(ANY_HOSTNAME);
https.setSSLSocketFactory(sslContext.getSocketFactory());If you are using the Apache HttpComponents library, it is also quite simple when using the HttpClientBuilder:
HttpClientBuilder builder = // created elsewhere
builder.setSSLContext(sslContext);
builder.setSSLSocketFactory(new SSLConnectionSocketFactory(sslContext, ANY_HOSTNAME));Lastly, if you are using the OkHttp client, you can:
OkHttpClient.Builder builder = // created elsewhere
builder.sslSocketFactory(sslContext.getSocketFactory(), allTrusting);
builder.hostnameVerifier(ANY_HOSTNAME);Ok, now that the gritty details have been discussed, what about a simpler approach - how can this be done in HttpBuilder-NG? All you need to do is apply the ignoreSslIssues() helper method to your configuration as:
def http = JavaHttpBuilder.configure {
ignoreSslIssues execution
// other config...
}Which will apply the configurations discussed above and you are ready to go. A means of doing this via system property is also provided (see the User Guide for more details).
Now you are off and running to ignore SSL certificate issues. As a quick disclaimer and reminder, SSL is an important security measure for web connections and should not be disabled/ignored ligthly and never in an internet-facing production environment - these procedures for ignoring errors are really meant for testing purposes.
Fuzzy Text Matching
12 January 2017 ~ groovy
From time to time I have needed to find matching text data based on some user input data, for example given a list of known company names, return a list of potential matches to a company name entered by the user - or for our example here: given a list of people’s first names, find the best matches to the user-provided name. Yes, there are tools, libraries and frameworks to do this in really efficient ways and in large volume, but if you are not already using them and if this is the only such problem you have to solve, it’s better to have a simple solution that works well-enough without adding bulk to your application.
As I mentioned above, let’s say we have a list of first names - I collected 100 from a random name generator site and put them in a text file (a sample is shown below):
names.txt
Sona
Terisa
Shasta
Jerold
Joetta
Harrison
Earle
Isaiah
Torrie
Valarie
Lynell
Mignon
Sharla
Kiesha
ArtWhen given a name, such as "Harry", how can we filter the list of names to provide the best matches? I have found the getJaroWinklerDistance in the Apache Commons Lang StringUtils class to be quite useful. It’s a string similarity algorithm that returns a double similarity result given two strings - the larger the number, the better the match.
With that you can load the names into a collection and process each of them against your given name to find the best N results (let’s say 10). The script is as follows:
find_name.groovy
@Grapes(
@Grab('org.apache.commons:commons-lang3:3.5')
)
import static org.apache.commons.lang3.StringUtils.getJaroWinklerDistance
def query = args[0].toLowerCase()
def names = new File('./names.txt').readLines().unique()*.toLowerCase()
println "Searching for: ${query}\n"
println 'Name Score'
println '--------------------'
names.collect { n->
new Tuple(n, getJaroWinklerDistance(n, query))
}.sort { -it.get(1) }[0..10].each { r->
println "${r.get(0).padRight(15)}${r.get(1)}"
}
println ''If I run this against the set of 100 names I used, I get the following result:
> groovy find_name.groovy Harry Searching for: harry Name Score -------------------- harrison 0.86 gary 0.78 sharla 0.7 darrin 0.7 art 0.69 margart 0.68 maryellen 0.64 sari 0.63 hana 0.63 earle 0.6 shana 0.6
We see that there is actually a pretty good match, "Harrison", and even "Gary" for that matter. I have used this method to provide a list of suggestions back to the user so that they can make the final selection from them (being the list of items actually available in your system).
It’s an interesting technique and I am sure that there are better ways - feel free to suggest them.
Introducing Ersatz
10 December 2016 ~ blog groovy testing
While working on tests for the HttpBuilder-NG project, I tried out a couple different mock server libraries, my old go-to Mock Server and then the OkHttp Mock Server, but both had their own issues and just didn’t really fit the bill for what I wanted to be able to do with mock server testing. So, I decided to do some prototyping over a long weekend and I was able to come up with the Ersatz Server.
My goal was to use an standardized embedded HTTP server and then provide a rich DSL to configure expectations on it with all the bells and whistles of any other mocking library. I used the Undertow web server with both a Java 8 chained builder and a Groovy DSL approach to configuration to allow very simple and expressive expectation configuration.
With the Groovy DSL you can define expectations such as:
ErsatzServer ersatz = new ErsatzServer()
server.expectations {
get('/say/hello'){
verifier once()
query 'name','Ersatz'
responder {
content 'Hello Ersatz','text/plain'
}
}
}
ersatz.start()
URL url = "${ersatz.serverUrl}/say/hello?name=Ersatz".toURL()
assert url.text == 'Hello Ersatz'
assert ersatz.verify()
ersatz.stop()which will respond to a GET request to /say/hello?name=Ersatz with the text content Hello Ersatz and it will be expected that this request is called exactly once, or the verify() call will fail. This could also be written in standard Java:
server.expectations( expect -> {
expect.get("/say/hello").verifier(once()).query("name","Ersatz").responds().content("Hello Ersatz","text/plain");
});The builder form and the DSL form are equivalent and may be used together when developing in Groovy.
Expectations can be configured across the major HTTP request methods and can be matched by path as well as headers, cookies, body contents and other custom conditions. Multiple responses may be configured on a request so, for example, the first call would respond with some value, but all subsequent calls would respond with a 500 error status, such as:
server.expectations {
post('/save'){
body data, 'application/json'
responder {
content outdata, 'application/json'
}
responder {
code 500
}
}
}This allows for some interesting and flexible configuration options.
It’s a new library but I have replaced the mock server code in HttpBuilder-NG with it and it makes the tests a bit cleaner and adds some nice features that we can utilize going forward that were not present in the other mock server libraries. Give it a try.
Writing Gradle Plugins
07 December 2016 ~ blog groovy gradle
In my last post, Gradle: A Gentle Introduction, I discussed the basics of Gradle and how to get up and running quickly. Now, I am going to dive into the deeper part of the pool and talk about how to write your own Gradle plugins.
First, we need a project to work with. Let’s say that we want to add a custom banner to our build output - who doesn’t love banners? Something like:
_______ _ ____ _ _ _
|__ __| | | _ \ (_) | | |
| | | |__ ___ | |_) |_ _ _| | __| |
| | | '_ \ / _ \ | _ <| | | | | |/ _` |
| | | | | | __/ | |_) | |_| | | | (_| |_ _ _
|_| |_| |_|\___| |____/ \__,_|_|_|\__,_(_|_|_)
We need to create a directory named banner-build and then create a build.gradle file in it with the following starting content:
build.gradle
plugins {
id 'groovy'
}We just need a basic starting point. Run ./gradle wrapper --gradle-version=3.2 to generate the wrapper and we are ready to start (we can run /gradlew from here on out).
Now, in order to write out a banner we need to create a custom task that will render it for us:
task banner {
doFirst {
if( !project.hasProperty('noBanner') ){
file('banner.txt').eachLine { line->
logger.lifecycle line
}
}
}
}
gradle.startParameter.taskNames = [':banner'] + gradle.startParameter.taskNamesThis task will add our action to the top of the execution list (the startPrameter modification makes it always run) so that if the noBanner property is not specified, our banner will be loaded from the specified file and displayed to the output log.
We will read our banner from a file, banner.txt in the root of the project - so we will need to create that with the banner content from above. Then, when you run ./gradlew build you will see something like the following:
> ./gradlew build
:banner
_______ _ ____ _ _ _
|__ __| | | _ \ (_) | | |
| | | |__ ___ | |_) |_ _ _| | __| |
| | | '_ \ / _ \ | _ <| | | | | |/ _` |
| | | | | | __/ | |_) | |_| | | | (_| |_ _ _
|_| |_| |_|\___| |____/ \__,_|_|_|\__,_(_|_|_)
:compileJava UP-TO-DATE
:compileGroovy UP-TO-DATE
:processResources UP-TO-DATE
:classes UP-TO-DATE
:jar
:assemble
:compileTestJava UP-TO-DATE
:compileTestGroovy UP-TO-DATE
:processTestResources UP-TO-DATE
:testClasses UP-TO-DATE
:test UP-TO-DATE
:check UP-TO-DATE
:build
BUILD SUCCESSFUL
Total time: 0.559 secs
Notice also, that we can turn off the banner, passing the -PnoBanner option on the command line or as a property in your gradle.properties file, if you have one - if you run under one of those conditions, the banner will not be printed.
At this point, we have accomplished our original goal and we can go on with our lives… until the next project comes along and you need the same sort of functionality. You could just copy and paste this code into your project, but you don’t do that… right? That’s where plugins come into play; they allow us to share functionality across different project builds.
To create the plugin, first we need a separate Gradle project for it; create a directory (outside of the one for our demo project), called banner-plugin and add a build.gradle file to it with:
banner-plugin/build.gradle
plugins {
id 'groovy'
id 'java-gradle-plugin'
}
version = "0.1.0"
group = "com.stehno.gradle"
sourceCompatibility = 8
targetCompatibility = 8
repositories {
jcenter()
}
dependencies {
compile gradleApi()
compile localGroovy()
testCompile('org.spockframework:spock-core:1.0-groovy-2.4') {
exclude module: 'groovy-all'
}
}and run gradle wrapper --gradle-version=3.2 in it to generate our wrapper. The build file for a plugin project is a standard Gradle build file, but with the java-gradle-plugin plugin to provide extra tools needed for plugins, as well as dependencies for the Gradle API and it’s associated Groovy distribution. With plugins, the project name is used as part of the unique plugin ID, so it’s generally a good practice to be explicit about the project name using a settings.gradle file:
banner-plugin/settings.gradle
rootProject.name = 'banner-plugin'The last piece of plugin-specific configuration is the plugin properties file, which is a file in the resources/META-INF/gradle-plugins directory named <group>.<name>.properties, for this example:
banner-plugin/src/main/resources/META-INF/gradle-plugins/com.stehno.gradle.banner-plugin.properties
implementation-class=com.stehno.gradle.banner.BannerPluginNow we can create the basic skeleton for our plugin, which is an implementation of the Gradle Plugin<Project> interface:
banner-plugin/src/main/groovy/com/stehno/gradle/banner/BannerPlugin.groovy
package com.stehno.gradle.banner
import org.gradle.api.Plugin
import org.gradle.api.Project
class BannerPlugin implements Plugin<Project> {
@Override void apply(final Project project) {
// your config here...
}
}This is the main entry point for our plugin. When it is "applied" to the project, the apply(Project) method will be called. If we do a clean build of the project at this point, it will pass, but it does nothing. We need to transfer our functionality (the banner task) from our original build.gradle file to the plugin. Let’s create the plugin task skeleton:
banner-plugin/src/main/groovy/com/stehno/gradle/banner/BannerTask.groovy
package com.stehno.gradle.banner
import org.gradle.api.DefaultTask
import org.gradle.api.tasks.TaskAction
class BannerTask extends DefaultTask {
}and give it something to do:
@TaskAction
void displayBanner(){
logger.lifecycle 'Doing something!'
}Once we have a task, we need to wire it into the plugin so that it is applied to the project. Change the apply(Project) method of our BannerPlugin class to the following:
@Override void apply(final Project project) {
project.task 'banner', type:BannerTask
project.gradle.startParameter.taskNames = [':banner'] + project.gradle.startParameter.taskNames
}This will apply our new task and then cause it to be called whenever the build is run. Now, how do we check our progress? We could build the plugin and deploy it to our original project but that would be quite a lot of round-trip time every time we wanted to test a change, but there is no need for that, Gradle provides a rich test framework which works well with Spock. Let’s create a Spock test for our task:
banner-plugin/src/test/groovy/com/stehno/gradle/banner/BannerTaskSpec.groovy
package com.stehno.gradle.banner
import spock.lang.Specification
import org.junit.Rule
import org.junit.rules.TemporaryFolder
import org.gradle.testkit.runner.BuildResult
import org.gradle.testkit.runner.BuildTask
import org.gradle.testkit.runner.GradleRunner
import org.gradle.testkit.runner.TaskOutcome
class BannerTaskSpec extends Specification {
@Rule TemporaryFolder projectRoot = new TemporaryFolder()
def 'simple run'(){
given:
File buildFile = projectRoot.newFile('build.gradle')
buildFile.text = '''
plugins {
id 'groovy'
id 'com.stehno.gradle.banner-plugin'
}
'''.stripIndent()
projectRoot.newFile('banner.txt').text = 'Awesome Banner!'
when:
BuildResult result = GradleRunner.create()
.withPluginClasspath()
.withProjectDir(projectRoot.root)
.withArguments('clean build'.split(' '))
.build()
then:
println result.output
}
}It’s a bit of code, but it’s not too bad once you dig in. We have a standard Spock test, with a TemporaryFolder rule - this will be our test project directory. Then, we create a build.gradle file for our test with our plugin and the groovy plugin, similar to what our original Gradle file looked like. Next, we use the GradleRunner to create and configure a Gradle environment using our file, which is then executed as a build. The results are then printed out to the command line. If you run ./gradlew test on the project now and view the test output (in the report standard out), you will see:
:banner Doing something! :clean UP-TO-DATE :compileJava UP-TO-DATE :compileGroovy UP-TO-DATE :processResources UP-TO-DATE :classes UP-TO-DATE :jar :assemble :compileTestJava UP-TO-DATE :compileTestGroovy UP-TO-DATE :processTestResources UP-TO-DATE :testClasses UP-TO-DATE :test UP-TO-DATE :check UP-TO-DATE :build BUILD SUCCESSFUL Total time: 2.019 secs
where we can see our output and we have a way to quickly test our new task. So, moving onward, we need to add the real functionality to our task. Update the displayBanner() method to:
@TaskAction
void displayBanner(){
if( !project.hasProperty('noBanner') ){
project.file('banner.txt').eachLine { line->
logger.lifecycle line
}
}
}Notice that we prefixed project. before the file() call since we are no longer directly in the "project" scope, but other than that this code was copied right from our original build file. If you run the test, you see our message in the test output:
:banner Awesome banner! :clean UP-TO-DATE :compileJava UP-TO-DATE :compileGroovy UP-TO-DATE :processResources UP-TO-DATE :classes UP-TO-DATE :jar :assemble :compileTestJava UP-TO-DATE :compileTestGroovy UP-TO-DATE :processTestResources UP-TO-DATE :testClasses UP-TO-DATE :test UP-TO-DATE :check UP-TO-DATE :build BUILD SUCCESSFUL Total time: 2.019 secs
Our test is good, but it doesn’t really verify anything, it just prints out the build output. Let’s make it verify that the build passed and that our expected message is in the output - the then: block becomes:
then:
result.tasks.every { BuildTask task ->
task.outcome == TaskOutcome.SUCCESS || task.outcome == TaskOutcome.UP_TO_DATE
}
result.output.contains('Awesome Banner!')The test will no longer generate the build output to the command line, but we are actually verifying the expected behavior.
We can test the noBanner property support as well, but we should also refactor the test a bit so that shared code is reused - now our test looks like:
banner-plugin/src/test/groovy/com/stehno/gradle/banner/BannerTaskSpec.groovy
package com.stehno.gradle.banner
import spock.lang.Specification
import org.junit.Rule
import org.junit.rules.TemporaryFolder
import org.gradle.testkit.runner.BuildResult
import org.gradle.testkit.runner.BuildTask
import org.gradle.testkit.runner.GradleRunner
import org.gradle.testkit.runner.TaskOutcome
class BannerTaskSpec extends Specification {
@Rule TemporaryFolder projectRoot = new TemporaryFolder()
private File buildFile
def setup(){
buildFile = projectRoot.newFile('build.gradle')
buildFile.text = '''
plugins {
id 'groovy'
id 'com.stehno.gradle.banner-plugin'
}
'''.stripIndent()
projectRoot.newFile('banner.txt').text = 'Awesome Banner!'
}
def 'simple run'(){
when:
BuildResult result = GradleRunner.create()
.withPluginClasspath()
.withProjectDir(projectRoot.root)
.withArguments('clean build'.split(' '))
.build()
then:
println result.output
buildPassed result
result.output.contains('Awesome Banner!')
}
def 'simple run with status hidden'(){
when:
BuildResult result = GradleRunner.create()
.withPluginClasspath()
.withProjectDir(projectRoot.root)
.withArguments('clean build -PnoBanner'.split(' '))
.build()
then:
buildPassed result
!result.output.contains('Awesome Banner!')
}
private boolean buildPassed(final BuildResult result){
result.tasks.every { BuildTask task ->
task.outcome == TaskOutcome.SUCCESS || task.outcome == TaskOutcome.UP_TO_DATE
}
}
}Mostly I just extracted the setup code and the buildPassed check, then added a test for the noBanner property support.
Wouldn’t it be nice to make the banner file location configurable? Gradle plugins have a "extension" construct that allows for rich configuration of plugins by adding functionality to the Gradle DSL. For our plugin, we would like to support something like the following:
banner {
enabled = true
location = file('banner.txt')
}which would be used to toggle the banner display on and off and also provide a means of configuring the banner file location. This structure and both of its properties are optional, but allow additional configuration. Adding them to the plugin is fairly simple. The extension itself is just a POGO class, which for our case would be:
banner-plugin/src/main/groovy/com/stehno/gradle/banner/BannerExtension.groovy
package com.stehno.gradle.banner
class BannerExtension {
boolean enabled = true
File location
}To register the extension with the plugin, you add the following to the first line of the BannerPlugin apply(Project) method:
project.extensions.create('banner', BannerExtension)The last part of adding the extension support is to have the task actually make use of it. The displayBanner method of the task will look like the following when we are done:
@TaskAction
void displayBanner(){
BannerExtension extension = project.extensions.getByType(BannerExtension)
boolean enabled = project.hasProperty('bannerEnabled') ? project.property('bannerEnabled').equalsIgnoreCase('true') : extension.enabled
File bannerFile = project.hasProperty('bannerFile') ? new File(project.property('bannerFile')) : (extension.location ?: project.file('banner.txt'))
if( enabled ){
bannerFile.eachLine { line->
logger.lifecycle line
}
}
}I modified the noBanner property and converted it to a flag so that now you would pass in -PbannerEnabled=false to disable it. I also added a means of configuring the banner file from the command line or via the extension, with the default still being banner.txt. The CLI and settings properties will override the extension values if they are present. We need to modify the 'simple run with status hidden' test to handle the new parameter:
def 'simple run with status hidden'(){
when:
BuildResult result = GradleRunner.create()
.withPluginClasspath()
.withProjectDir(projectRoot.root)
.withArguments('clean build -PbannerEnabled=false'.split(' '))
.build()
then:
buildPassed result
!result.output.contains('Awesome Banner!')
}Now, if you run the tests, everything still passes - so the defaults work as expected. Let’s add some tests using the extension to override the file location.
def 'extension run'(){
setup:
buildFile.text = '''
plugins {
id 'groovy'
id 'com.stehno.gradle.banner-plugin'
}
banner {
location = file('other-banner.txt')
}
'''.stripIndent()
when:
BuildResult result = GradleRunner.create()
.withPluginClasspath()
.withProjectDir(projectRoot.root)
.withArguments('clean build'.split(' '))
.build()
then:
buildPassed result
result.output.contains('Awesome-er Banner!')
}In this test we have to override the default build file we created in setup. I also added the creation of the other banner file in the setup method:
projectRoot.newFile('other-banner.txt').text = 'Awesome-er Banner!'Now, run the tests again and see that our extension works as expected.
With our newly minted Gradle plugin we should be able to use it in our original project as a local test before deployment. An easy way to do this is to publish it to your local maven repository and then configure the other project to use it. In the plugin project, add id 'maven-publish' to the plugins block, which will allow us to publish to the local maven repo. Then run ./gradlew publishToMavenLocal, which does what it says.
In the original external build.gradle file we need to add bootstrapping code to bring in the local plugin and also remove the old banner task. The updated build.gradle file will look like this:
build-banner/build.gradle
buildscript {
repositories {
mavenLocal()
}
dependencies {
classpath "com.stehno.gradle:banner-plugin:0.1.0"
}
}
plugins {
id 'groovy'
}
apply plugin: "com.stehno.gradle.banner-plugin"Notice that we are pulling the plugin from the local maven repository. If you run the build now, you get your expected banner:
> ./gradlew build
:banner
_______ _ ____ _ _ _
|__ __| | | _ \ (_) | | |
| | | |__ ___ | |_) |_ _ _| | __| |
| | | '_ \ / _ \ | _ <| | | | | |/ _` |
| | | | | | __/ | |_) | |_| | | | (_| |_ _ _
|_| |_| |_|\___| |____/ \__,_|_|_|\__,_(_|_|_)
:build
BUILD SUCCESSFUL
Total time: 0.543 secs
However, we should be able to use a different banner file. Create another banner file as flag.txt (with whatever you want in it) and configure the build to use it by adding:
banner {
location = file('flag.txt')
}to the bottom of the build file. Now, with my new version, I get:
> ./gradlew build :banner This is GRADLE!!! :build BUILD SUCCESSFUL Total time: 0.474 secs
We can also disable the banner via config, set enabled = false in the extension code, and it will not appear. But, you can force it on the command line by adding -PbannerEnabled=true.
From here, you can distribute your plugin to friends and coworkers as long as you have some shared repository that you can point them to, but what if you came up with something cool enough to share to a larger audience? For that you want to publish to the http://plugins.gradle.com repo, which is what is used by the plugins block of the build.gradle file. I won’t go too far down that path in this post, but basically you will need to add the id 'com.gradle.plugin-publish' version '0.9.4' plugin to the plugin project, which will handle the actual publishing for you once you configure it for your project. In our case this would be something like:
pluginBundle {
website = 'http://yourdomain.com/banner-plugin'
vcsUrl = 'https://github.com/cjstehno/banner-plugin'
description = 'Gradle plugin to add a fancy banner to your build log.'
tags = ['gradle', 'groovy']
plugins {
webpreviewPlugin {
id = 'com.stehno.gradle.banner-plugin'
displayName = 'Gradle Build Banner Plugin'
}
}
}Once you have that in place and have signed up with the plugins portal (free) you run ./gradlew publishPlugins and if all goes well, you have a publicly available plugin.
This tutorial has really only scratched the surface of plugin development, there is a lot more there to work with and most of it is well documented in the Gradle User Guide or through some Google searches. It’s a powerful framework and well worth spending the time to learn if you are working in Gradle.
Gradle: A Gentle Introduction
02 November 2016 ~ blog groovy gradle
The Gradle build tool has become widely-used over the past few years, but there are still a lot of developers who are unfamiliar with it, and like any new framework or technology it is easier to get started with some guidance. Hopefully this will provide a nice jump start into doing some actual work with Gradle. With that being said, let’s dig in!
I will forgo installing Gradle - you can read about how to do that for your platform in the Gradle documentation. Let’s assume you have Gradle installed, preferably a current version. If you run gradle --version you should see something like:
------------------------------------------------------------ Gradle 3.1 ------------------------------------------------------------ Build time: 2016-09-19 10:53:53 UTC Revision: 13f38ba699afd86d7cdc4ed8fd7dd3960c0b1f97 Groovy: 2.4.7 Ant: Apache Ant(TM) version 1.9.6 compiled on June 29 2015 JVM: 1.8.0_102 (Oracle Corporation 25.102-b14) OS: Linux 4.8.0-26-generic amd64
First, create a directory for your project and cd into it:
mkdir hellogradle cd hellogradle
Every Gradle project needs at least a build.gradle file so we will start with the minimal requirement. Create the following file in your project directory:
build.gradle
plugins {
id 'groovy'
}
repositories {
jcenter()
}
dependencies {
compile 'org.codehaus.groovy:groovy-all:2.4.6'
}The plugins block is used to specify the Gradle plugins used in the build; in this case we just need groovy since we are making a Groovy project. The groovy plugin extends the java plugin so we get its functionality as well.
The repositories block specifies which repositories are available for resolving dependency artifacts - in most cases jcenter() (the Bintray repository) is enough to start with.
Lastly, the dependencies block is where the project dependencies are defined as <configuration> '<group>:<artifact>:<version>'.
At this point you have a working Gradle project. You can build it:
gradle clean build
You should see something like the following:
> hellogradle gradle clean build :clean :compileJava UP-TO-DATE :compileGroovy UP-TO-DATE :processResources UP-TO-DATE :classes UP-TO-DATE :jar :assemble :compileTestJava UP-TO-DATE :compileTestGroovy UP-TO-DATE :processTestResources UP-TO-DATE :testClasses UP-TO-DATE :test UP-TO-DATE :check UP-TO-DATE :build BUILD SUCCESSFUL Total time: 0.492 secs
You can see that it is already doing quite a bit for so few lines of build code. You can run gradle tasks to see a list of the tasks available to your project.
So far, we have been running against my local version of Gradle. One of the nice features of Gradle is the wrapper functionality. The wrapper allows the project to specify the version of Gradle it should be built under; this code is then checked into your source control system so that a new developer can checkout the project and build it with the correct version of Gradle without it being installed on their system.
Add the following to the bottom of the build.gradle file:
task wrapper(type: Wrapper) {
gradleVersion = '3.1'
}Then, whenever you add the wrapper or change the supported version, you need to execute the gradle wrapper task to regenerate the configuration. Once the wrapper is in place, you will want to execute all your Gradle tasks using it rather than your local installation. You do this using the gradlew or gradlew.bat scripts provided in the root of your project. Now let’s do a clean build again with the wrapper:
> hellogradle ./gradlew clean build :clean :compileJava UP-TO-DATE :compileGroovy UP-TO-DATE :processResources UP-TO-DATE :classes UP-TO-DATE :jar :assemble :compileTestJava UP-TO-DATE :compileTestGroovy UP-TO-DATE :processTestResources UP-TO-DATE :testClasses UP-TO-DATE :test UP-TO-DATE :check UP-TO-DATE :build BUILD SUCCESSFUL Total time: 0.678 secs
We end up with the same result as before.
This project doesn’t do anything at this point - there is no code. Let’s add some Groovy code; create the directories for src/main/groovy/demo and then add the file:
HelloGradle.groovy
package demo
class HelloGradle {
String greet(final String name){
"Hello, ${name ?: 'Gradle'}!"
}
}This code simply says hello to the name passed in as an argument, or Gradle by default. Now we will need to unit test our work, so let’s add support for the Spock testing framework. Add the following to your dependencies closure:
testCompile 'org.spockframework:spock-core:1.0-groovy-2.4'We now have Spock available, so let’s write a unit test for our code. Create the test directories: src/test/groovy/demo and then create the file:
HelloGradleSpec.groovy
package demo
import spock.lang.Specification
import spock.lang.Unroll
class HelloGradleSpec extends Specification {
private final HelloGradle greeter = new HelloGradle()
@Unroll def 'say hello #name'(){
expect:
greeter.greet(name) == result
where:
name | result
null | 'Hello, Gradle!'
'' | 'Hello, Gradle!'
'Chris' | 'Hello, Chris!'
}
}This test will verify that our greeter returns the expected values for null-ish inputs as well as when a name is provided. I won’t go into the details of the Spock test at this point. Now, when you build the project, you will also run the tests by default:
> hellogradle ./gradlew clean build :clean :compileJava UP-TO-DATE :compileGroovy :processResources UP-TO-DATE :classes :jar :assemble :compileTestJava UP-TO-DATE :compileTestGroovy :processTestResources UP-TO-DATE :testClasses :test :check :build BUILD SUCCESSFUL Total time: 1.789 secs
Notice :test near the bottom. Gradle also provides an HTML report of your test results. The report will be generated in the build/reports directory and will look something like the following image:
Now that we have a test, it might be nice to have some idea of our test coverage. Gradle provides a plugin for the jacoco code coverage library. You can add the plugin by adding id 'jacoco' to the plugins block of your build.gradle file, which allows you to run:
./gradlew clean build jacocoTestReport
to build the project with tests and a generated test coverage report. Again, the report is generated in the build/reports directory - it will look something like:
The coverage report allows you to drill down into the source code and see what is and is not covered by your tests.
Testing your code is nice, but you need a way to run your application outside of testing. Let’s first add a main method to our Groovy code:
static void main(args){
println new HelloGradle().greet(args ? args[0] : null)
}Nothing fancy, just instantiate the HelloGradle class and call the greet(String) method with the first argument, if there is one. To make the project runnable, we need to add the application plugin and specify a "main class". To do this:
-
Add
id 'application'to thepluginsblock -
Add
group = 'demo'to give the project an artifact group -
Add
version = '0.0.1'to give your project a version -
Add
mainClassName = 'demo.HelloGradle'to thebuild.gradlefile outside of other configuration blocks.
With that, you now have a new task run which will run the application for you:
> hellogradle ./gradlew run :compileJava UP-TO-DATE :compileGroovy :processResources UP-TO-DATE :classes :run Hello, Gradle! BUILD SUCCESSFUL Total time: 1.526 secs
It also generates .tar and .zip distributions of the project which contain starter scripts and all required dependencies to deploy and run your application outside of the project itself.
Code quality analysis tools are also available as Gradle plugins. A common one for Groovy development is CodeNarc which runs quality rules against your code to generate a report of possible issues. We can add this to the project by adding id 'codenarc' to the plugins block and adding some additional config to build.gradle:
codenarcMain {
ignoreFailures false
configFile file('config/codenarc-main.rules')
maxPriority1Violations 0
maxPriority2Violations 5
maxPriority3Violations 10
}
codenarcTest {
ignoreFailures true
configFile file('config/codenarc-test.rules')
}Which configures a different rules and criteria for main source code vs test source code. The main and test rule sets are based on their suggested configurations, personal preference and experience - I generally use the files from my Vanilla project (main, test) for simplicity. This configuration will fail the build when the violation thresholds are exceeded for the main classes, but will simply report on the violations for test classes. The build will now run the codenarc checks when a build is executed.
The build will let you know if violations were found, and in any case will generate a report in the build/reports/codenarc directory. The report will look something like the following:
At this point, we have a Gradle-based Groovy project with portable support for building, testing, coverage, code quality and application run/deployment, all with a few dozen lines of understandable code. While there is a lot more you can and should do with Gradle, this is a good starting point. From here, you should read through their documentation in general or touch on topics as you need them to figure out how to do something. Also Google is your best reference for finding how-tos or 3rd-party plugins; however, there is an official plugin repository that is starting to catch on.
Once you get the hang of it Gradle is hard to let go of due to its compact code, expressiveness and flexibility without the pains and rigor of older tools, like Maven and Ant.
Gradle Natives Plugin Update
19 September 2016 ~ blog java groovy gradle
A few years ago I wrote a post about my Gradle-Natives plugin, called "Going Native with Gradle". The plugin was the result of some failed attempts at game programming and it pretty much stopped there; however, it seems there are some users who found it useful. In the years since it was written, it sat and got buggy and then recently just became useless due to external library changes and the rigidity of the plugin functionality. Tryign to be a good open source citizen, I figured it would be a good time to do some updates that will hopefully keep the plugin viable for a while.
With the new release, I figured it would be good to go back to the original post and attempt a similar example. Using the example code from the LWJGL Getting Started Guide we have:
import org.lwjgl.*;
import org.lwjgl.glfw.*;
import org.lwjgl.opengl.*;
import static org.lwjgl.glfw.Callbacks.*;
import static org.lwjgl.glfw.GLFW.*;
import static org.lwjgl.opengl.GL11.*;
import static org.lwjgl.system.MemoryUtil.*;
public class HelloWorld {
// The window handle
private long window;
public void run() {
System.out.println("Hello LWJGL " + Version.getVersion() + "!");
try {
init();
loop();
// Free the window callbacks and destroy the window
glfwFreeCallbacks(window);
glfwDestroyWindow(window);
} finally {
// Terminate GLFW and free the error callback
glfwTerminate();
glfwSetErrorCallback(null).free();
}
}
private void init() {
// Setup an error callback. The default implementation
// will print the error message in System.err.
GLFWErrorCallback.createPrint(System.err).set();
// Initialize GLFW. Most GLFW functions will not work before doing this.
if ( !glfwInit() )
throw new IllegalStateException("Unable to initialize GLFW");
// Configure our window
glfwDefaultWindowHints(); // optional, the current window hints are already the default
glfwWindowHint(GLFW_VISIBLE, GLFW_FALSE); // the window will stay hidden after creation
glfwWindowHint(GLFW_RESIZABLE, GLFW_TRUE); // the window will be resizable
int WIDTH = 300;
int HEIGHT = 300;
// Create the window
window = glfwCreateWindow(WIDTH, HEIGHT, "Hello World!", NULL, NULL);
if ( window == NULL )
throw new RuntimeException("Failed to create the GLFW window");
// Setup a key callback. It will be called every time a key is pressed, repeated or released.
glfwSetKeyCallback(window, (window, key, scancode, action, mods) -> {
if ( key == GLFW_KEY_ESCAPE && action == GLFW_RELEASE )
glfwSetWindowShouldClose(window, true); // We will detect this in our rendering loop
});
// Get the resolution of the primary monitor
GLFWVidMode vidmode = glfwGetVideoMode(glfwGetPrimaryMonitor());
// Center our window
glfwSetWindowPos(
window,
(vidmode.width() - WIDTH) / 2,
(vidmode.height() - HEIGHT) / 2
);
// Make the OpenGL context current
glfwMakeContextCurrent(window);
// Enable v-sync
glfwSwapInterval(1);
// Make the window visible
glfwShowWindow(window);
}
private void loop() {
// This line is critical for LWJGL's interoperation with GLFW's
// OpenGL context, or any context that is managed externally.
// LWJGL detects the context that is current in the current thread,
// creates the GLCapabilities instance and makes the OpenGL
// bindings available for use.
GL.createCapabilities();
// Set the clear color
glClearColor(1.0f, 0.0f, 0.0f, 0.0f);
// Run the rendering loop until the user has attempted to close
// the window or has pressed the ESCAPE key.
while ( !glfwWindowShouldClose(window) ) {
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT); // clear the framebuffer
glfwSwapBuffers(window); // swap the color buffers
// Poll for window events. The key callback above will only be
// invoked during this call.
glfwPollEvents();
}
}
public static void main(String[] args) {
new HelloWorld().run();
}
}If we put this in a simple Gradle project with build.gradle as:
plugins {
id 'com.stehno.natives' version '0.2.4'
id 'java'
id 'application'
}
version = "0.0.1"
group = "com.stehno"
mainClassName = 'hello.HelloWorld'
sourceCompatibility = 8
targetCompatibility = 8
repositories {
jcenter()
}
dependencies {
compile 'org.lwjgl:lwjgl:3.0.0'
compile 'org.lwjgl:lwjgl-platform:3.0.0:natives-windows'
compile 'org.lwjgl:lwjgl-platform:3.0.0:natives-linux'
compile 'org.lwjgl:lwjgl-platform:3.0.0:natives-osx'
}
task wrapper(type: Wrapper) {
gradleVersion = "2.14"
}We can view the native libraries for all platforms using ./gradlew listNatives:
:listNatives
Native libraries found for configurations (compile, runtime)...
- lwjgl-platform-3.0.0-natives-linux.jar:
[LINUX] libjemalloc.so
[LINUX] liblwjgl.so
[LINUX] libglfw.so
[LINUX] libopenal.so
- lwjgl-platform-3.0.0-natives-osx.jar:
[MAC] liblwjgl.dylib
[MAC] libjemalloc.dylib
[MAC] libglfw.dylib
[MAC] libopenal.dylib
- lwjgl-platform-3.0.0-natives-windows.jar:
[WINDOWS] lwjgl.dll
[WINDOWS] lwjgl32.dll
[WINDOWS] OpenAL.dll
[WINDOWS] jemalloc.dll
[WINDOWS] glfw.dll
[WINDOWS] glfw32.dll
[WINDOWS] jemalloc32.dll
[WINDOWS] OpenAL32.dll
and we can build and run the HelloWorld application with ./gradlew clean build run, which begs the question of whether or not this plugin is needed, since at this point the application works and we have not used the plugin at all. I will leave that to developers who actually work with this stuff and may use the plugin - I am just updating the existing functionality.
You can inlude the native libraries in the build using ./gradlew clean build includeNatives which will unpack the native libraries into the project build directory.
There are still a number of configuration options available through the natives DSL extension, such as including and excluding libraries, as well as limiting the scan to certain configurations and platforms, but I will leave those for the official documentation. Without any additional configuration you get all of the native libraries from the compile and runtime configurations for all platforms unpacked into the build/natives directory.
This plugin is still pretty raw, but hopefully it is useful enough to make some developers lives easier.
HTTP Builder NG for Groovy and Java
18 September 2016 ~ blog groovy
The HttpBuilder project has been a go-to library for the simplification of HTTP requests for years; however, development on the project has stalled and seemingly died. A friend of mine (Dave Clark) decided to pick up where the project left off and to bring it up-to-date with modern Groovy and Java 8 support. The HTTP Builder NG project is a major update and refactor of the original project. I joined on to help with development, documentation and testing. In my opinion, this effort has brought the library back from the dead, better than ever. In this post, I will walk through accessing a simple REST service using the HttpBuilder with both Groovy and Java examples - yes, the new version of the library supports standard Java 8 coding.
First, we need a REST service to work with. I have thrown together a simple set of endpoints using the Spark Web Framework to make a "message of the day" service. There are three endpoints:
-
GET /message- retrieves the current stored message -
POST /message- saves thetextfield of the JSON body content as the new message -
DELETE /message- deletes the current message
There is not much to it, but it should be enough to play with. You can find the code in the [repo for this post](https://github.com/cjstehno/httpb-demo). Startup the server by running:
./gradlew run
In the root of the project. The server will be running on http://localhost:4567.
Let’s start off by retrieving the current message from the server. We need a base configured HttpBuilder object to make requests from:
HttpBuilder http = HttpBuilder.configure {
request.uri = 'http://localhost:4567'
}Then, we need to make a GET request to the /message path:
def result = http.get {
request.uri.path = '/message'
}When you run this code, you will get the following:
[text:n/a, timestamp:2016-09-16T12:47:55+0000]
which is a Map of the parsed JSON data coming back from the server - the HttpBuilder recognizes the application/json response content and parses it for you. In this case all we really want is the text, so let’s transform the response data a bit:
String text = http.get(String){
request.uri.path = '/message'
response.success { FromServer from, Object body->
body.text
}
}We have added an expected result type of String and a response.success() handler. This handler will be called when a successful response code is received (code < 400). When it is called it will pull the text field out of our body object, which in this case, is the already-parsed JSON data. The return value from the success() method is returned as the result - the text of the current message. When you run this version of the code you get the current message text:
n/a
This is the default "empty" message content. How do we update the message to something more interesting? The service exposes POST /message which will take the text field of the request body content and use it as the new message. We can write a POST request just as easily as our GET request:
String updated = http.post(String) {
request.uri.path = '/message'
request.contentType = 'application/json'
request.body = { text 'HttpBuilder is alive!' }
response.success { FromServer from, Object body ->
body.text
}
}Again, we will expect the text of the new message back from the server, but this time we are calling the post() method with a JSON content type. Note that our body content is using the Groovy JsonBuilder closure format, it could have just as easily been a Map of the data to be encoded. Similar to the response decoding, the request body is automatically encoded based on the content type.
If you run the code now, you will get:
HttpBuilder is alive!
You could also call the get() method again and verify that it is the current message.
As a final example with our service, let’s call the DELETE /message endpoint to reset the message back to it’s "empty" state. A DELETE request is just as simple:
String deleted = http.delete(String){
request.uri.path = '/message'
response.success { FromServer from, Object body ->
body.text
}
}The result will be the new message after deletion:
n/a
which is the "empty" state.
One thing we notice now that we have written all of the verb calls is that there are a lot of similarities between them. They all call the same path and they all handle the successful response content in the same manner. I am not a fan of duplication, so we can move the common configuration up into the main configure method:
HttpBuilder http = HttpBuilder.configure {
request.uri = 'http://localhost:4567/message'
response.success { FromServer from, Object body ->
body.text
}
}and our verb methods, now contain only what they need to do their work:
String message = http.get(String) {}
String updated = http.post(String) {
request.contentType = 'application/json'
request.body = { text 'HttpBuilder is alive!' }
}
String deleted = http.delete(String) {}Nice and clean. Now wait, I know, I promised something similar in plain old Java, well Java 8 anyway… ok, you can do the same operations in Java with a fairly similar expressiveness:
HttpBuilder http = HttpBuilder.configure(config -> {
config.getRequest().setUri("http://localhost:4567/message");
config.getResponse().success(new BiFunction<FromServer, Object, String>() {
@Override public String apply(FromServer fromServer, Object body) {
return ((Map<String, Object>) body).get("text").toString();
}
});
});
String message = http.get(String.class, config -> {});
System.out.println("Starting content: " + message);
// update the content
String updated = http.post(String.class, config -> {
config.getRequest().setContentType("application/json");
config.getRequest().setBody(singletonMap("text", "HttpBuilder works from Java too!"));
});
System.out.println("Updated content: " + updated);
// delete the content
String deleted = http.delete(String.class, config -> {});
System.out.println("Post-delete content: " + deleted);Notice that the Java 8 lambdas make the syntax about as simple as the Groovy DSL. When you run this version of the client you get:
Starting content: n/a Updated content: HttpBuilder works from Java too! Post-delete content: n/a
In Java or Groovy, the library makes HTTP interactions much easier to work with. Check out the project and feel free to submit bug reports and feature requests, or even suggested details to be documented.
ND4J Matrix Math
In my last post (Commons Math - RealMatrix), I discussed the matrix operations support provided by the Apache Commons Math API. In doing my research I also stumbled on a library that is much closer in functionality to the Python NumPy library (commonly used in Machine Learning examples). The ND4J library is a scientific computing library for the JVM, meant to be used in production environments, which means routines are designed to run fast with minimum RAM requirements.
The main draw I had at this point was that their support for array-style element-by-element operations was much deeper than the matrix operations provided by the Apache Commons Math API and much closer to what I was seeing in the Python code I was working with, which makes conversion simpler.
With NumPy in Python you can multiply two arrays such that the result is the multiplication of each value of the array by the corresponding value in the second array. This is not so simple with matrices (as shown in my last post). With ND4J, it becomes much simpler:
def arrA = Nd4j.create([1.0, 2.0, 3.0] as double[])
def arrB = Nd4j.create([2.0, 4.0, 6.0] as double[])
def arrC = arrA.mul(arrB)
println "$arrA + $arrB = $arrC"will result in:
[1.00, 2.00, 3.00] * [2.00, 4.00, 6.00] = [ 2.00, 8.00, 18.00]
which is as we would expect from the Python case. ND4J also has the ability to do two-dimensional (matrix-style) arrays:
def matA = Nd4j.create([
[1.0, 2.0, 3.0] as double[],
[4.0, 5.0, 6.0] as double[]
] as double[][])
println "Matrix: $matA\n"which will produce:
Matrix: [[1.00, 2.00, 3.00], [4.00, 5.00, 6.00]]
All of the other mathematical operations I mentioned in the previous post are available and with data structures that feel a lot more rich and refined for general use. This is barely scratching the surface of the available functionality. Also, the underlying code is native-based and has support for running on CUDA cores for higher performance. This library is definitely one to keep in mind for cases when you have a lot of array and matrix-based operations.
Older posts are available in the archives.
CoffeaElectronica.com content is copyright © 2016 Christopher J. Stehno and available under a Creative Commons Attribution-ShareAlike 4.0 International License.