Archive

Tag Archives: DevOps

On Wednesday I had the pleasure of presenting a new talk at the London Microservices User Group, which was entitled “The Business Behind Microservices: Organisational, Architectural and Operational Challenges”.

The key theme of the talk was exploring the often under-appreciated organisational and people impact that introducing (or moving to) a microservice architecture will have on a business. As mentioned in the talk, I’ve lead the implementation of microservices architectures in several organisation as part of my work with OpenCredo and Container Solutions, and so I was keen to share my lessons from the trenches.

You can find a recording of the talk over on the Skillsmatter website, who were our generous hosts for the evening https://skillsmatter.com/skillscasts/6450-the-business-behind-microservices-organisational-architectural-and-operational-challenges 

Daniel Bryant - Business Behind Microservices video recording

 

You can also find the slides on slideshare:

 

The original talk abstract was as follows:

The technology changes required when implementing a microservice-based application are only one part of the equation. The business and organisation will also most likely have to fundamentally change. In an ideal world, this shouldn’t be a problem – what with the rise of agile, lean and DevOps – but this is not always the situation I encounter in my consulting travels. I would like to share some stories of successful (and not so successful) strategies and tactics I have used over the past four years when introducing service-oriented architecture into organisations.

Join me for a whistle-stop tour of the business and people challenges that I have experienced first hand when implementing a greenfield microservice project, and also breaking down a monolith. We’ll look at ‘divided companies’ vs ‘connected companies’, determine the actual impact of conway’s law, briefly touch on the lean startup/enterprise mindset, dive into change management without the management double-speak, and look at the lightweight processes needed to ensure the technical success of a microservices implementation.

As usual, please do let me know if you have any questions!

On Thursday I once again had the pleasure of presenting at the London Java Community (LJC) meetup. This time I presented a new talk “The Craft Consultant’s Guide to… DevOps”. This builds on a DevOps parody talk I gave at last year’s LJC Unconference, and also combines some of my thoughts from my earlier “Chuck Norris doesn’t do DevOps, but Java developers might benefit” talk.

The goals of this talk was to provide a high-level overview of some of the key themes of DevOps, specifically from a tooling perspective. It’s worth stating at this point that in my opinion DevOps is more about methodology, process and culture more than it is tooling. However, for the intended audience of this presentation the focus on tooling provides a framework on which to build understanding for the non-technical issues. The slides can be found below:

The original abstract for the talk can be found in the next paragraph, and as usual, if you have any comments, thoughts or (constructive) criticism then please do get in touch!

“Come along and learn how the Crafty Consultant makes his money by consulting craftily in DevOps. We’ll see how silos can be broken down by introducing more independent and isolated team, how only idiots automate everything, and why monitoring only provides actionable insight that simply confuses your clients…

…and then we’ll look at the real world implementation of DevOps 🙂 The primary aims of this talk are to introduce the concepts behind the DevOps movement, and we’ll do this by debunking all of the Crafty Consultant’s advice. We’ll cover the drivers of breaking down silos (in business and in tech), the benefits of automation (especially with provisioning and configuring infrastructure), and the power that monitoring provides (particularly when deploying to the cloud, or implementing a microservice architecture).”

Details of the event can be found on meetup.com

Building Microservices – by Sam Newman

5_star

TLDR: If you are looking for an introduction to and overview of the current ‘microservice’ landscape and the concepts and thinking behind it, then look no further. I would recommend this book for architect, developer, QA and operations (DevOps)

Microservices are obviously a very hot topic at the moment, and everyone and their dog appears to have an opinion about this concept. Accordingly, writing a book about microservices and keeping everyone happy is going to be a challenge (and probably impossible).

In my opinion Sam has done a great job here and manages to provide key information on a range of relevant microservice issues, such as the motivations for microservices, how to model services, integration with other systems, deployment, testing, monitoring, security and architecting/implementing microservices at scale.

I’ve seen negative comments towards this book on other sites stating that more code examples should be provided, and although I appreciate their motivations, I don’t believe this will be the book to address these issues. In my experience, code contained within a book can quickly become stale, and the diversity of language (and the associated rate of change) that is currently being used to create microservice implementations would make pleasing everyone an impossible challenge. Instead I recommend people interested in code samples have a Google, or visit InfoQ, DZone or Voxxed where there is plenty of microservice implementation code.

In my opinion I will use this book much like I did the original ‘Continuous Delivery’ book by Jez Humble and Dave Farley – the book will provide an excellent high-level overview of the issues (and potential solutions), help me to understand core concepts and how key components and methodologies relate to each other, and also provide inspiration and pointers to further reading.

Much like I didn’t expect to create a full build pipeline implementation simply from reading the ‘Continuous Delivery’ book, I wouldn’t expect to build a complete microservice ecosystem from Sam’s book. However, after reading both books I began the respective tasks with a lot more insight then I originally had, and I made much smarter decisions (and knew what to look for when searching for more knowledge) once I understood the big picture.

As stated above, I would recommend this book for any software delivery role. I’ve personally seen the benefits of a microservice architecture when deployed for the correct use case, but the nature of this architecture creates new challenges whether you are a developer, QA or operations specialist.

(In the interest of full disclosure I did provide feedback on this book as it was being published via the O’Reilly Early Access program. However, I have endevoured to write a review that takes into account only the final published book.)

I’ve received some great feedback after posting my proposal for a microservice maturity/classification model last week, some positive, and some negative.

Some private communications suggested that I may be getting caught up in the marketing hype, and several emails suggested that the microservice architecture really is just classical SOA re-invented. Other emails balanced out these comments by suggesting that microservices present an opportunity to learn and iterate on the mistakes made in the original implementation of SOA, especially now that we are embracing concepts such as domain-driven design, and are applying more consideration to well-defined software architectures.

The only public response I’ve seen so far is by my fellow London-based microservice and Spring framework expert Russ Miles – you can read it on the Simplicity Itself blog. In the interest of full disclosure I do know Russ personally (and have sank a few beers in his company), but that’s not going to influence what I think could be a great discussion about my proposal.

Maturity – not all it’s cracked up to be…

The first comment made by Russ is that the approach of creating a maturity model could be dangerous. I think this is totally fair, and it crossed my mind several times when writing the initial post. So much so, that I added the word ‘classification’ to the title as an alternative to maturity.

If you look at other maturity models, such as the Richardson model for APIs or the Continuous Delivery model, then there is a clear sense of scale, from negative to positive. As Russ quite rightly points out, there is room for interpretation in my model that smaller is better, and I probably should have taken more care to make it clear that I don’t think this is necessarily the case.

In some cases a monolithic, but well-structured, architecture may be the best solution. Russ has also conjectured in a recent talk a Skillsmatter that starting out with building a monolith and then moving to a microservice architecture may be the fastest way to build software. It’s definitely difficult to prove this beyond anecdotal evidence, but my instincts (and experience) tell me this is probably true in certain cases, especially at the current point in time where we have little in the way of modelling or tooling support for building microservices.

In my opinion Russ is quite right to think about how this model could be used negatively, and although my initial intention was to give people a model that they could look at and point to where they think their software is, it could easily be abused. I would be keen to get more feedback on how the model could be shaped or evolved to make my intentions clearer.

Size – it’s what you do with it that counts (but size still matters)

Russ also mentioned that size is a dangerous metric, and I agree. Although lines of code (KLOC) is potentially an arbitrary metric, especially with the variety of languages and frameworks currently available, I still do believe that size is important. Not in the “if you’re application is over a 100 lines, then it’s not a microservice” kind of way (which, in fairness, could have been read from my model), but in the perspective of encapsulation, responsibility and comprehension.

After a bit more thought, a measure of architectural/code cohesion is probably a better metric for this concept. I definitely believe the microservice architecture is rooted in the principle of high cohesion (and loose coupling), but it has been argued by the likes of Simon Brown and Bob Martin that this can be achieved in a monolithic codebase without the need for the creation of separate ‘services’. The key here is modularisation or componentisation.

Accordingly, I do believe that microservices should be ‘small’ in size. For me, this was one of the failings in the original approach with SOA. The lack of skilled modelling and architectural guidance allowed services to morph into ‘all singing and dancing’ applications that offered low cohesion. Hard system boundaries (and potentially expensive coordination and communication) provided by microservices in combination with the notions of ‘bounded contexts’ from domain-driven design (DDD) should make it more obvious to developers when we are straying outside the remit of a component.

Bloated vendor tools that emerged from traditional SOA also allowed developers to ‘cheat’ by circumventing the loose coupling that patterns such as the service bus initially proposed, and monstrosities such as the heavyweight ESB were born. These days we are seeing tooling emerge that encourage reactive systems and event-driven architectures based on small components (such as the very interesting AWS Lambda). With services such as AWS Lambda a codebase size limit is enforced due to the nature of execution. It will be interesting to see what applications emerge from these frameworks.

Putting a size limit on a codebase may not be an exact science, but I believe an upper bound can at least be used as a trigger to discuss if a service is growing beyond it’s original remit (or if the architectural quality is degrading). A lot of agile and architectural techniques I teach clients are not hard-and-fast rules to determine which decisions should be made, but often act as a cue for the team (or organisation) to engage in conversation or a whiteboard session to check that we are still designing high-quality software that models the business correctly.

Dogma or dogfooding – you decide…

The final section of Russ’ post contains the strongest argument, in that dogma over thinking will only lead us down the wrong path “A maturity model can be used in place of thinking; I’d like to avoid that if we can”. Yeah, this sucks, but I agree.

The problem is that my academic background drives me towards the sharing of ideas and proposals among my peers, and I enjoy the ensuing discussion. We should always take care to make sure we are being pragmatic in these discussions (and not “solving the world’s problems at the dinner table”), but I’m still a supporter of pushing stuff out to the public for comment.

Russ also makes a great reference to Greg Young’s talk at muCon last year, which should be essential watching for anyone building microservices. Paraphrasing Greg massively, he suggested that a lot of the concepts behind microservices have already been done before, and that if we aren’t careful then we will re-invent the wheel (albeit more ‘micro’ than before 🙂 ).

Greg’s observations about the negative impact of dogmatic standardisation and overly-opinionated vendor tooling were also especially damning, and I couldn’t help but nod in agreement through a lot of his talk (on a side note, the whole muCon conference was awesome, and I would highly recommend attending the next iteration later this year! Massive kudos to Russ for kickstarting this conference).

I’m definitely going to take care to avoid being dogmatic (or inspiring dogma), but I’m still keen to share my thoughts on things like the maturity/classification model. It might turn out that this approach isn’t useful, but I’ve already been using a slightly less polished version of this model with tech friends over the last few months to help them understand where their software stack currently sits in relation to the ‘unicorn’ organisations such as Netflix and Amazon (what else are techies going to discuss over a few beers! :- ).

Something that I believe could emerge from this type of proposal (which may be more valuable) is some kind of model that shows organisations where their software sits on the big picture scale of innovation, architecture and delivery. Each level of the model should also clearly show the benefits and drawbacks, and provide guidance on why (if at all) organisations should move some of their software to the next level. We would also need to show how organisations should go about doing this, both from a pragmatic organisational and cultural perspective (Conway’s law in action), and also from a technical tooling and process perspective.

I’m currently reading Jez Humble’s new book ‘Lean Enterprise’, and this is providing some superb inspiration for approaching these tasks. I’m also dogfooding some of my new models and processes, and as soon as I have some useful insights I’ll make sure I share them.

In summary…

I really appreciate Russ taking the time to reply to my original post, and I’m definitely going to think more of several of the great points he’s made (and I’m sure I’ll also catch up with him for a beer after an upcoming London Microservices User Group meetup).

I will also take more care to make my intentions clearer, but I’m still keen to share my thoughts and inspire debate. I’m also keen to avoid dogma, and focus more on dogfooding the model, and here I’ll use some of Russ’ comments to attempt to refine the model.

As usual, if anyone has any comments or feedback then please do get in touch!

I’m currently at JavaOne and have just finished presenting the latest iteration of my “Cloud Developer’s DHARMA” talk, which was great fun. As promised, here are the slides:

 

 

The abstract for this talk is included here (just for the search engine’s benefit 🙂 )

“Building Java applications for the IaaS cloud is easy, right? “Sure, no problem. Just lift and shift,” all the cloud vendors shout in unison. However, the reality of building and deploying cloud applications can often be different. This session introduces lessons learned from the trenches during several years of designing and implementing cloud-based Java applications, which we have codified into our Cloud Developer’s “DHARMA” rules: Documented (just enough); Highly cohesive/loosely coupled (all the way down); Automated from code commit to cloud; Resource-aware; Monitored thoroughly; and Antifragile. “
 

If you have any questions then please do get in touch!

Ok, so the review of this excellent conference is probably about a month overdue, but time just seems to have flown by! Anyway here it goes…

Life on the Program Committee

I was fortunate enough this year to be invited to sit on the program committee and review submissions (thanks Mark et al!). I’ll create a separate post about this experience later, but I wanted to start this review by mentioning that the number of submissions for Devoxx UK was very high, and the quality was outstanding.

According to my fellow Program Committee members, the job of picking which talks to include in the conference gets more difficult each year! As many committee members mentioned, we could have easily picked talks to fill an entire week of conference, but the difficult thing was that Devoxx UK was only running for 2 days!

The Conference Itself

The conference began in style with a live beatbox session from the UK Champion ‘Reeps One‘. Check out some of his stuff on youtube, but this guy needs to be seen live to get the full experience!

After a great introduction to the conference from the steering committee (Mark Hazell, Ellie May, Stephan Janssen, Dan Hardiker, James McGivern) Dan North opened up the with the keynote “Deliberate advice from an accidental career”. I enjoy most of Dan’s stuff, due largely to his story-telling skills and the fact that I can relate to most of the experiences he talks about, and this presentation was no exception. The key takeaways can be summarised in the photo below, but if you get chance to watch the complete presentation then I strongly recommend you take the opportunity 🙂
2014-06-12 10.17.57

Notes on good sessions

The first official session of the day for me was Andrew Harmel-Law’s “The 5 whys: Counter Intuitive Solutions to (all too common) Problems”, which was a great discussion of key problems in relation to the DevOps movement. The central theme to the talk related to a book on Japanese farming, and how the concepts could be mapped onto continuous delivery and DevOps, and as a fan of Eastern philosophy I enjoyed this metaphor very much 🙂

During the next two sessions I was dropping in and out of talks, and also catching up with other presenters and friends who I hadn’t seen in a while. Talks that looked interesting were Graham Allan’s “How switching to Scala made me less productive, and why that matters less than I thought” and Dick Wall’s “What have Monads ever done for us?”

The final conference session of the day that I attended was John Smart’s “It’s testing Jim, but not as we know it”. This session was a personal highlight of the conference for me, and the discussion of BDD really hit home to the projects I have recently been working on. John also has a MEAP book ‘BDD In Action’ which I can highly recommend.

In the evening Mani Sarkar, Richard Warburton, Martijn Verburg and myself presented an OpenJDK BOF session, which was well received, and generated plenty of great questions. If you want more information (or want to get hacking yourself) then please pop along to Mani’s excellent ‘Getting Started with OpenJDK document’

Roll on to Friday…

Friday began with Simon Brown’s “Software Architecture and the Balance with Agility”, which was another personal conference highlight. As part of my program committee role I had invited Simon to Devoxx UK, as not only am I a big fan of his work, but I also think the topic of architecture is often neglected in developer conferences. Simon didn’t disappoint, and key takeways for me included; developing software in an agile way doesn’t guarantee that the resulting software architecture is agile; a good architecture enables agility; there could be a sweet spot (architecturally speaking) somewhere between a monolithic architecture and SOA (but no-one want to hear this?); sketches can be maps – the key thing is creating a shared vision. Simon has a great website www.codingthearchitecture.com and also a book entitled “Software architecture for Developers”, both of which I can recommend. I definitely am a strong believer that all developers should be aware of architecture (and also that all architects should code!)

Next was an interesting talk ‘Groovy DevOps and the Cloud’, which touched on many of my favourite DevOps topics, but with a Groovy theme. I also couldn’t help noticing that a lot of the tools developed already existed in a non-groovy form, and so I wondered in some of this was vulnerable to the ‘not invented here’ argument, but the presenter did seem aware of this fact, and gave some great recommendations for resources and books at the end of the session

The third session of the day was my turn to present with Steve Poole “Moving to a DevOps mode: Easy, hard or just plain terrifying”. Here is an action-shot of Steve and I on stage…

14577280683_1bd3e6b977_z14370589518_8ac0759092_z

It’s safe to say that both Steve and I had a great time presenting, and the feedback and questions after the session were great. You can watch the recording of the session here (subscription required, but free to all who attended Devoxx UK). You can also find the slides in another blog post here

After our talk both Steve and I stayed in the same room to listen to the panel session “What does the Oracle/Google shenanigans mean to the Java Developer”. This talk was not only very funny (almost guaranteed when you create a panel containing Ted Neward with a lawyer…), but also quite pertinent, what with all of the interesting IP/patent issues constantly flying around.

The next session of the day “Modern web architecture” by Ted Neward was great. Ted has his own brand of ‘edu-tainment’, and this session lived-up to his usual billing. I’m not going to spoil his punchline, but I’ll simply say that watching the recording will be well worth your time!

I spent the remainder of the day in the Hackergarten, chatting to many people about OpenJDK and various JSRs (and also answering a few DevOps questions). I also managed to catch up with old friends in the hackergarten, and Anatole Tresch and I also planned our upcoming JSR-354 hackday which we ran at OpenCredo.

The conference was drawn to a close with an ensemble keynote, which was awesome. Martijn did a great job reviewing the conference content, Dick Wall shone an interesting light onto the technological present (including lots of details about wi-fi enabled dog Dotty!), and Arun and company covered (a hopefully very exciting) future, especially relating to the work everyone is doign in Devoxx for kids

The Saturday after-party

A few of us managed to continue the party into Saturday, with John Stevenson from Salesforce offering to run a Devoxx UK themed ‘Hack the Tower’ event. I had great fun here as well, and managed to get a few more people hacking on OpenJDK 🙂

In Conclusion

The conference was awesome, and built upon last years success. Devoxx UK is still one of my favourite conferences in terms of atmosphere, and the ‘by developers, for developers’ mantra really does ring true. Everywhere I went I overheard interesting conversations, saw impromptu hacking, and people taking the time to connect with one another. The quality of speakers was also excellent, and the range of topics nicely balanced (although I may be biased here… 🙂 )

I would also like to say a big thanks to all of the organisers, the volunteers, the speakers and the attendees for making the conference so great. Thanks also to all the friends (both old and new) I managed to catch up with, and I’ll see you all at another conference soon (maybe JavaOne?)

On a final note, Devoxx UK has already been confirmed for 2015, and so please book space in your diaries for a 3 day extravaganza (yes 3 days, not 2) between 17th-20th June!

14553793591_ff1f8f8d43_z

 

 

I once again had the pleasure of presenting a lightning talk at the London Software Craftsmanship Community #LSCCtalks series. This time I presented a new talk I’m working on – “Crafting DevOps: Applying software craftsmanship to DevOps”.

The presentation (hopefully) does what it says on the tin, in that I talk about the ideas of applying software craftsmanship principles to DevOps . If the DevOps mantra of “Infrastructure as Code” is true (which I think it is), then I believe it makes sense to learn from the craftsmanship community – in particular TDD, BDD and continuous integration.

This is very much a work-in-progress talk, and I hope to develop it more over the coming weeks and months (please do send me your feedback!).

 

Thanks again to Sandro and Samir from the LSCC for inviting me to talk, and as usual, if you have any questions, then please do get in touch. In particular please do let me know if you would like to see more on this topic.