Has your logging output suddenly disappeared when using Solr 4.3?

I’m currently working on a Java-based component which is utilising Solr heavily. We bundle the Solr core library dependency within a fat JAR (which we deploy standalone), and this dependency is managed via the de facto Maven approach.

The Problem

After a series of new features were added to this component by a member of the team we suddenly noticed that the usual Solr logging to the console has stopped. At first we thought Solr had stopped working (which caused a little panic 🙂 ), but even though no logging was being displayed we could still access the web console, and everything appeared to be functioning correctly.

What most of the team didn’t realise is that during the addition of new features one of the developers also bumped the version of the Solr dependency from 4.2.0 to 4.3.0. The intentions were good – get the latest and greatest version, and the expectation is that a minor version number increase usually fixes bugs and adds a few pieces of new functionality.

However, this time around it was probably worth reading the release notes, as the Solr team have fundamentally altered their approach to logging in the 4.3.0 release  http://wiki.apache.org/solr/SolrLogging

The Solution

The fix for us was to include an appropriate log4j.properties config file within our component’s Maven resources directory. We were already including the slf4j and log4j dependencies within our component Maven POM, and so we didn’t need to perform any additional steps to incorporate these into the deployment artifact (our fat JAR) as mentioned by the Solr team at http://wiki.apache.org/solr/SolrLogging

[Book Review] Instant Chef Starter

Instant Chef Starter – by John Ewart

TL;DR If you are looking for a rapid introduction to Chef, and haven’t got the time (or inclination) to look around the web then this book will meet your needs. However (and it’s a fairly big however), you can get pretty much the same information free from a combination of the excellent Opscode (Chef creators) website and a few Blogs resulting from a web search.

Anyone active within the so-called DevOps space will have heard of tools like Chef, Puppet, CFEngine and Saltstack. These provisioning tools are arguably the backbone on which the modern DevOps movement is based, and in combination with Cloud-based programmable infrastructure (like AWS) they are the driving force in automating configuration and application deployment. Understandably everyone is keen to write books on what is rapidly becoming the next big thing, and in my opinion this does lead to rushed output. This output is often useful to a limited degree, but the same information can often be gleaned (freely) from the interweb, and (perhaps more importantly) tools and techniques on the bleeding edge often evolve so fast that any printed media can become outdated quite quickly. I believe ‘Instant Chef Starter’ book fits into this category.

If you are looking for a book to read on the commute home, which details the key concepts and motivations behind Chef, then you can’t go wrong with getting the Kindle version of this book. However, you will definitely need to supplement your reading by checking the Opscode website for latest developments, and you will also need to start experimenting with the tool to fully understand the benefits it offers. Trust me, when you start spinning up Chef-driven servers and other resource with just a couple of commands via the CLI you are going to be blown away at how easy this stuff is. Your mind will be further blown when you realise that what you’ve just done is massively repeatable and scales very easily (in comparison to older techniques).

In summary, the Kindle version is worth a look if you’re looking for a fast-paced and concise introduction to Chef, and you don’t want to invest time in looking around the Opscode website or other provisioning websites just yet. I can’t recommend the printed book, because at £12 this doesn’t offer good value.

Click Here to buy ‘Instant Chef Starter’ on Amazon (This is a sponsored link. Please click through and help a fellow developer to buy some more books!  )

Hosted Chef, Vagrant and ‘There may already be a client named “vagrant.vm”‘…

I’ve been playing around with Chef again this afternoon, and ran into a problem after following the (very useful) Opscode tutorials and then experimenting on my own

The Problem

localhost ==============================================================
localhost
localhost Chef encountered an error attempting to create the client "vagrant.vm"
localhost
localhost ==============================================================
localhost
localhost
localhost
localhost
localhost Authorization Error:
localhost
localhost --------------------
localhost
localhost Your validation client is not authorized to create the client for this node (HTTP 403).
localhost
localhost
localhost
localhost
localhost
localhost
localhost Possible Causes:
localhost
localhost ----------------
localhost
localhost * There may already be a client named "vagrant.vm"
localhost
localhost * Your validation client (xxxxxx-validator) may have misconfigured authorization permissions.
localhost

It’s quite obvious that my earlier tutorial-based activities had registered the ‘vagrant.vm’ node name with my hosted Chef. Accordingly I visited my Hosted Chef portal and removed the node, but after receiving confirmation of the node being deleted I was still getting the same error when attempting to provision my local vm box.

Solution

Give the second Vagrant node a new name when bootstrapping e.g.

$ knife bootstrap localhost \
 --ssh-user vagrant \
 --ssh-password vagrant \
 --ssh-port 2222 \
 --run-list "recipe[apt]" \
 --sudo \
 --node-name "vagrant.vm2"

Alternatively you can delete the first node you created via knife on the CLI (rather than attempting to delete the node via the web-based Hosted Chef interface):

$ knife node delete "vagrant.vm"

DevOps and the Cloud. Not just localised in the Bay Area…

I’ve just read this interesting blog post about DevOps and the Cloud, and felt compelled to leave a comment, which is now turned into a blog post. The original article can be found here:

DevOps and cloud: A view from outside the Bay Area bubble

The post is USA-centric, talking about the Bay Area in particular, but it does make some very good (and thought-provoking) comments about how cutting-edge practices such as the DevOps philosophy tend to gravitate around the well established tech hubs.

I wanted to add my 2 cents (2 pence?) to the discussion to make sure people don’t just assume this practice only occurs around the West coast of America. Obviously this geographical area is highly influential in the global IT landscape, but people all over the world experience similiar trends and practices, albeit on a more micro scale.

I work as a freelance development consultant in the London areas, and many small companies are investing heavily in the new DevOps philosophy, particularly around the “Silicon Roundabout” area which is a haven for tech-focused start-ups. Although there are many other tech-hubs around the UK (as I’m sure there are all over the US; NYC for example?) it’s all too easy to see the pattern mentioned in the above article, especially in the more established ‘traditional’ IT sectors. When it comes to the Cloud, people in these sectors often talk a good game, but play very badly. This trend is regardless of geographical location.

It’s not difficult to see why. If I’m riding my ‘startup’ pedal bike then I can change direction at any time. If I see something I like, or something that looks new and shiny I can hop onto the sidewalk or even ride down a one-way street if I really want to. If I’m driving my SME articulated truck then I have to think ahead, but if something interesting pops up on the sat nav then I can usually change direction within a reasonable distance. If I’m driving (piloting?) my Enterprise-grade freight train then I have to start planning things months in advance and talk to a thousand other people before I even consider travelling on another track.

Having said this, I am generally very encouraged to see these new ‘DevOps’ approaches emerging, regardless of where this is happening. Anyone who is a true practitioner of this philosophy knows and can demonstrate the benefits it will bring to a business. My personal favourites are how DevOps methodologies can enable the implementation of Continuous Delivery, thus allowing more iterative product/feature releases, and also how DevOps can facilitate the automation of provisioning and deployments, thus allowing rapid auto-scaling to meet demand and also lowering the cost of experimentation. Anyone who is a fan of the “Lean Startup” methodologies should be jumping up and down with joy at the thought of this, but notice how I mentioned the words ‘Lean Startup’ There isn’t a ‘Lean Enterprise’ philosophy that I know of

In my opinion it’s only a matter of time before these early DevOps adopters spread the good word and this practice becomes mainstream. It doesn’t really matter where they are located or in what size company they work – if the methodology is based on sound and proven principles then it will eventually become adopted by almost everyone in the industry. It’s just a matter of how flexible the organisation is, and this largely relates to size and willingness to take (what are perceived as) risks.

[Flashback to the early 2000’s] TDD and Agile development anyone? Surely only the cool Bay Area kids do that? 😉

[Book Review] Scala in Action

Scala In Action – by Nilanjan Raychaudhuri

TL;DR This is a great book if you are looking for a rapid introduction to Scala (and its idiomatic usage) with a very practical focus. Where else would you be building a Scala-based RESTful API and MongoDB driver within the first few chapters, and more importantly be having fun doing it!

‘Scala in Action’ states that the target audience is open to all levels, but I believe that programmers with some previous exposure to an OO language (especially Java) will get the most benefit. This is not to say novice developers won’t enjoy the book, but they may find it moving at quite a fast pace.

‘Scala in Action’ will offer seasoned developers much to think about, as this book is very focused in teaching you pragmatic (and idiomatic) usage of Scala. The fact that the Scala language manages to blend both Object Oriented and Functional paradigms means that learning this language will provide a great transition into the differing approaches offered by these styles. (Could Scala be considered a ‘gateway’ language? 🙂 ). This makes Scala a great choice for your next language, and the fact that the book encourages idiomatic usage of the language is a great help in fully understanding the power and appropriate application of the language.

In addition to the topics mentioned above, this book also covers concurrency using the Actor model with Akka (to which I am becoming more and more of an enthusiast!), testing and TDD (which I believe is a mandatory topic in any new programming book) and the interoperability between Java and Scala.

I haven’t fully digested all of the book yet, and even when I have I’m confident that I will be reading the content several more times (and more imporantly, experimenting even more with the code samples).

In summary, ‘Scala in Action’ has been a perfect companion in my first serious voyage into learning this language, and it comes highly recommended for anyone else attempting the same task!

Click Here to buy Scala in Action on Amazon (This is a sponsored link. Please click through and help a fellow developer to buy some more books! 🙂 ) 

Welcome to the new Tai-Dev Blog!

Thanks for stopping by the new home of the Tai-Dev Blog! I’ve moved to WordPress because the technical limitations of my old blogging platform were starting to restrict what I could to post.

blog.co.uk was great for sharing text, but as a developer I want to share code and diagrams easily, and this platform had severe limitations. For example, blog.co.uk doesn’t support GitHub gists, or allow links through to books I wanted to recommend on Amazon. These are two activities I wanted to undertake on a near-weekly basis, as my primary motivations for blogging are to share my thoughts and recommendations to help other developers.

Accordingly, I’m marking the old blog http://tai-dev.blog.co.uk/  as officially ‘deprecated’ from today. I’ll leave it up and running for a few months, and I may even port across some of the more popular articles when I get chance, but all new content will appear here.

Thanks for reading, and I look forward to hearing everyone’s thoughts!