Archive

Tag Archives: Cloud

On Wednesday I had the pleasure of presenting a new talk at the London Microservices User Group, which was entitled “The Business Behind Microservices: Organisational, Architectural and Operational Challenges”.

The key theme of the talk was exploring the often under-appreciated organisational and people impact that introducing (or moving to) a microservice architecture will have on a business. As mentioned in the talk, I’ve lead the implementation of microservices architectures in several organisation as part of my work with OpenCredo and Container Solutions, and so I was keen to share my lessons from the trenches.

You can find a recording of the talk over on the Skillsmatter website, who were our generous hosts for the evening https://skillsmatter.com/skillscasts/6450-the-business-behind-microservices-organisational-architectural-and-operational-challenges 

Daniel Bryant - Business Behind Microservices video recording

 

You can also find the slides on slideshare:

 

The original talk abstract was as follows:

The technology changes required when implementing a microservice-based application are only one part of the equation. The business and organisation will also most likely have to fundamentally change. In an ideal world, this shouldn’t be a problem – what with the rise of agile, lean and DevOps – but this is not always the situation I encounter in my consulting travels. I would like to share some stories of successful (and not so successful) strategies and tactics I have used over the past four years when introducing service-oriented architecture into organisations.

Join me for a whistle-stop tour of the business and people challenges that I have experienced first hand when implementing a greenfield microservice project, and also breaking down a monolith. We’ll look at ‘divided companies’ vs ‘connected companies’, determine the actual impact of conway’s law, briefly touch on the lean startup/enterprise mindset, dive into change management without the management double-speak, and look at the lightweight processes needed to ensure the technical success of a microservices implementation.

As usual, please do let me know if you have any questions!

So, I’m just heading back home after an awesome few days in Budapest, where I was attending the second edition of Craft Conference. I didn’t realise the conference was happening last year, but when my twitter stream was flooded was interesting looking #CraftConf tweets I had a look at the UStream live stream, and then I realised I was missing out! I managed to watch many of last year’s talks via UStream after the conference, and there was some great speakers and great content. As soon as this year’s Craft Conf was announced I bought a ticket, flights and hotel! Having completed the roller-coaster ride that was CraftConf 2015, I’m happy to say that I wasn’t disappointed…

The good…

A quick look down the list of speakers will give you an insight to the amount of tech rockstars that were in the building: Mary Poppendieck, Dan North, Jessica Kerr, Randy Shoup, Trisha Gee, Michael Nygard, Amber Case, Michael Feathers, Sandro Mancuso….need I go on? If you look back at the list, something may strike you as different compared to the majority of other tech conferences – many of the rockstars present are female.

Having first-hand experience of sitting on tech conference program committees myself, I know that it can be very difficult to organise and encourage talk submissions from a diverse group of speakers, of which gender diversity is just one part. I have to tip my hat to the CraftConf organisers for running a tech conference where 25% of the speakers were female, and the fact that they stated in the keynote that this isn’t enough should be applauded (kudos also to the organisers for offering ‘diversity tickets’ to attendees)

The vast majority of talks lived up the billing, and all four keynote sessions were phenomenal (and I don’t use this word lightly!). I left every keynote buzzing, and judging by the amount of chatter and questions from my fellow attendees, so did most other people. Don’t get me wrong, the keynotes weren’t all ‘ra ra – new tech’ or ‘developers, developers, developers’, they were genuinely thought-provoking and some of them boarder-line offensive (in a good ‘call to action’ kinda way). If you go to as many conferences as I do (including some of the bigger names), you’ll again realise how much effort must have gone into organising and planning this content – a big thanks once again to the organisers.

The ‘standard’ sessions running throughout the day were superb (with only a couple of misses detailed below), and the fact that talks were running from 9am to 7pm allowed me to definitely think I was getting value for my money. My general rule for tech conferences these days, is that if I enjoy over 50% of the talks I attend then I’m happy – CraftConf was in the 80-90% range, which is very rare (only the QCon series manages this usually).

During breaks, or when I did skip a talk session, there was always someone to talk to out in the sponsors area or one of the lounges. A tip of the hat again to many of the speakers, who constantly made themselves available and approachable when milling around the conference (and thanks in particular to Adrian Trenaman, Randy Shoup, Sandro Mancuso, Werner Schuster, Trisha Gee, Isra Kaos, Sven Peters, Simon Brown and James Lewis for some great chats).

I’ve listed some of my favourite talks below (in no particular order), and have included links to InfoQ summaries I have written where available (I will try and write more over the coming weeks).

There were also several talks I missed, due to InfoQ commitments or otherwise chatting to interesting people, and I’ll try to watch these back when they are released on UStream:

  • The Ethical Developer – Grady Booch (heard great things about this talk throughout the conference!)
  • DevOps for Everyone – Katherine Daniels (I follow Katherine on twitter, and she talks about some cool stuff)
  • Automating the Modern Datacenter – Mitchel Hashimoto (who doesn’t know what Hashicorp do?)
  • Consensus Systems for the Skeptical Architect – Camille Fournier (I again heard great things about this talk via twitter)

The logistics of the conference was also superb. I found the venue easily enough, registration was a breeze, and moving around the venue was straight-forward (after some initial hunting for the ‘tent’ room, which was actually a tent outside! 🙂 ) Lunches and breaks were well spaced apart and nicely timed, and the food and drink was also excellent (including getting fed for the entire day on the Thursday). The queues for the lunch sessions did get a bit disorganised, but after completing the Paris Marathon and nearly getting shoved to the ground several times when stopping for food and water breaks, I’m starting the think that queueing may be a distinctly-British thing… 😉

The (not so) bad…

I had a few minor grumbles, and to be honest these really were minor. A couple of the talks were too high-level or a bit disorganised, but I only left one early. I’m not going to name and shame people here, but if the conference organisers do ask for feedback then I’m happy be honest. What could help for next year is to indicate the intended audience ‘level’ for a talk, but even then I would expect quite deep content for each session at the conference that I believe CraftConf is trying to be.

I did have a couple of interesting chats with people who mentioned that CraftConf isn’t really a conference about software craftsmanship, and I do agree here – at least in terms of how we define mainstream ‘software craftsmanship’ in London and the USA i.e. focusing on clean code, TDD and outside-in development.

However, I strongly believe that CraftConf is a conference about the software development and delivery craftsmanship, and this is exactly what a generalising specialist like me wants. I care deeply about software development from idea inception through to development, delivery and ultimately validation (thinking about project management, leadership and stakeholder management along the way), and CraftConf hit all of these points nicely for me. The only reason I mention this fact, is that if you are a hardcore London or US-based software craftsman, then don’t come to this conference expecting the type of content you may be used to in typical craftsman events.

I’ve seen a couple of tweets and another blog (including the ever-informative Burning Monk) saying that there was a lack of deep-dive language and functional talks, but I think the conference was all the better for it. This is obviously a subjective opinion, and I enjoy learning about languages and functional as much as the next developer, but I tend to target specialist conferences around these subjects when I want to know more (for example, Scala eXchange or SpringOne 2GX).

For me, the balance of talks at CraftConf was superb, and there could be a danger of diluting the content (and the audience) if the organisers try to make the conference ‘all things to all people’. Just my two pence, but the organisers have run two amazing conferences already, and so I say listen to your heart, not necessarily the advice… (and I appreciate the irony of my advice in this context 🙂 )

And yeah, there was no ugly…

If you’ve read the above content, then you’ve probably figured that I think quite highly of CraftConf, and you would be right. I thoroughly enjoyed my time at the conference, and the combination of speakers, topics and attendees was fantastic. I’ll definitely attempt to head back next year, and this time I’ll keep an eye out for the CFP, as I would love to speak here as well.

The primary meme I took away from this year’s CraftConf is that software development is on the verge of breaking out of the traditional IT shackles, in much the same way computing infrastructure has broken-free and is rapidly evolving with the mainstream adoption of cloud. For anyone familiar with the diffusion of innovation graph, lean principled and business-focused software development is getting dangerously close to the late majority, and this really will shake up the industry (especially the traditional software development shops and developers). Alf Rehn’s and Marty Cagan’s keynotes were great example of this

I’m starting to think that the ‘full-stack’ developer of 2018 will not only have the skills that we expect now, but they will also be business savvy, and know enough statistics and data science to appropriately challenge slow moving executives as to what the expected return on investment will be from the latest (HiPPO) instructions from above… People may say this kind of thing happens now, and it does (and it famously has been in companies like Amazon for quite some time), but I believe this will soon happen in the majority of companies. Don’t forget, if you attended CraftConf or are reading this blog you are already part of a self-selecting audience who aspires to learn more and become better – and you’re in the minority within our industry…

The second meme that I picked up on was is that we are working in a golden age of tooling and process. With the emergence of technologies like cloud and containers, and practices like lean, BDD and TDD, there really is no excuse to be building bad software applications. It may take a bit more thought and design, as mentioned by Mary Poppendieck, Dan North and Jessica Kerr, but it is most definitely possible to build large complex systems that can be validated, tested and deployed with relative ease.

In summary, if you are looking for premium-level conference content at a frankly bargain price (with a fantastic location thrown into the mix), you can’t go wrong with CraftConf. I guarantee you’ll leave with more knowledge than when you arrived, and a whole-lot more questions…

Once again, many thanks to all of the organisers, volunteers and speakers – a great job by all concerned!

Building Microservices – by Sam Newman

5_star

TLDR: If you are looking for an introduction to and overview of the current ‘microservice’ landscape and the concepts and thinking behind it, then look no further. I would recommend this book for architect, developer, QA and operations (DevOps)

Microservices are obviously a very hot topic at the moment, and everyone and their dog appears to have an opinion about this concept. Accordingly, writing a book about microservices and keeping everyone happy is going to be a challenge (and probably impossible).

In my opinion Sam has done a great job here and manages to provide key information on a range of relevant microservice issues, such as the motivations for microservices, how to model services, integration with other systems, deployment, testing, monitoring, security and architecting/implementing microservices at scale.

I’ve seen negative comments towards this book on other sites stating that more code examples should be provided, and although I appreciate their motivations, I don’t believe this will be the book to address these issues. In my experience, code contained within a book can quickly become stale, and the diversity of language (and the associated rate of change) that is currently being used to create microservice implementations would make pleasing everyone an impossible challenge. Instead I recommend people interested in code samples have a Google, or visit InfoQ, DZone or Voxxed where there is plenty of microservice implementation code.

In my opinion I will use this book much like I did the original ‘Continuous Delivery’ book by Jez Humble and Dave Farley – the book will provide an excellent high-level overview of the issues (and potential solutions), help me to understand core concepts and how key components and methodologies relate to each other, and also provide inspiration and pointers to further reading.

Much like I didn’t expect to create a full build pipeline implementation simply from reading the ‘Continuous Delivery’ book, I wouldn’t expect to build a complete microservice ecosystem from Sam’s book. However, after reading both books I began the respective tasks with a lot more insight then I originally had, and I made much smarter decisions (and knew what to look for when searching for more knowledge) once I understood the big picture.

As stated above, I would recommend this book for any software delivery role. I’ve personally seen the benefits of a microservice architecture when deployed for the correct use case, but the nature of this architecture creates new challenges whether you are a developer, QA or operations specialist.

(In the interest of full disclosure I did provide feedback on this book as it was being published via the O’Reilly Early Access program. However, I have endevoured to write a review that takes into account only the final published book.)

A micro approach to a macro problem?

The microservice hype is everywhere, and although the industry can’t seem to agree on an exact definition, we are repeatedly told that moving away from a monolithic application to a Service-Oriented Architecture (SOA) consisting of small services is the correct way to build and evolve software systems. However, there is currently an absence of traditional ‘Enterprise’ organisations talking about their adoption of microservices. This blog post is a preview to a larger article, which explores the use of microservices in the Enterprise.

Interfaces – Good contracts make for good neighbours

Whether you are starting a greenfield microservice project or are tasked with deconstructing an existing monolith into services, the first task is to define the boundaries and corresponding Application Programming Interfaces (APIs) of your new components.

The suggested granularity of a service in a microservice architecture is finer in comparison with what is typically implemented when using a classical Enterprise Service Oriented Architecture (SOA) approach, but arguably the original intention of SOA was to create cohesive units of reusable business functionality, even if the implementation history tells a different story.

A greenfield microservice project often has more flexibility, and the initial design stage can define Domain Driven Design (DDD) inspired bounded contexts with explicit responsibilities and contracts between service provider and consumer (for example, using Consumer Driven Contracts).

However, a typical brownfield project must look to create “seams” within the existing applications and implement new (or extracted) services that integrate with the seam interface. The goal is for each service to have high cohesion and loose coupling; the design of the service interface is where the seeds for these principles are sowed.

Communication – Synchronous vs asynchronous

In practice, we find that many Enterprises will need to offer both synchronous and asynchronous communication in their services. It is worth noting that there is a considerable drive within the industry to move away from the perceived ‘heavyweight’ WS-* communication standards (e.g. WSDL, SOAP, UDDI), even though many of the challenges addressed by these frameworks still exist, such as service discovery, service description and contract negotiation (as articulated very succinctly by Greg Young in a recent presentation at the muCon microservices conference).

Middleware – What about the traditional enterprise stalwarts?

Although many heavyweight Enterprise Service Bus ESBs can perform some very clever routing, they are frequently deployed as a black box. Jim Webber once joked that ESB should stand for “Egregious Spaghetti Box,” because the operations performed within proprietary ESBs are not transparent, and are often complex.

If requirements dictate the use of an ESB (for example, message splitting or policy-based routing), then open source lightweight ESB implementations such as Mule ESB or Fuse ESB should be among the first options you consider.

I usually find that a lightweight MQ platform, such as RabbitMQ or ActiveMQ is more suitable because we believe the current trend in SOA communication is towards “dumb pipes and smart endpoints” In addition to removing potential vendor fees and lock-in, other benefits of using lightweight MQ technologies include easier deployment, management, and simplified testing.

Deploying microservices – How hard can it be?

However you choose to build microservices, it is essential that a continuous integration-style build pipeline be used which includes rigorous automated testing for functional requirements, fault-tolerance, security and performance. The classical SOA approach of manual QA and staged evaluation is arguably no longer appropriate in an economy where ‘speed wins’ and the ability to rapidly innovate and experiment is a competitive advantage (as captured within the Lean Startup movement).

Behaviour of your application can become emergent in a microservice-based platform, and although nothing can replace thorough and pervasive monitoring in your production stack, a build pipeline that exercises (or tortures) your components before they are exposed to your customers would appear to be highly beneficial. As I’ve argued in several conference presentations, a good build pipeline should exercise services in the target deployment environment as early in the pipeline as possible.

Summary – APIs, lightweight comms, and correct deployment

Regardless of whether you subscribe to the microservice hype, it would appear that this style of architecture is gaining traction within practically all software development domains. This article has attempted to provide a primer for understanding key concepts within this growing space, and hopefully reminds readers that many of these problems and solutions have been seen before with classical Enterprise SOA. We would be wise to take care not to reinvent the proverbial ‘service-oriented’ wheel.

Please click here for the complete original article, which provides additional information on microservice implementation options on the JVM platform, and also discusses the requirement for Continuous Delivery. A version of this article was originally published in the DZone 2014 Guide to Enterprise Integration.

References

A full list of references and recommended reading can also be found in the original article and a recent article discussing the business implications of microservices.

I’m currently at JavaOne and have just finished presenting the latest iteration of my “Cloud Developer’s DHARMA” talk, which was great fun. As promised, here are the slides:

 

 

The abstract for this talk is included here (just for the search engine’s benefit 🙂 )

“Building Java applications for the IaaS cloud is easy, right? “Sure, no problem. Just lift and shift,” all the cloud vendors shout in unison. However, the reality of building and deploying cloud applications can often be different. This session introduces lessons learned from the trenches during several years of designing and implementing cloud-based Java applications, which we have codified into our Cloud Developer’s “DHARMA” rules: Documented (just enough); Highly cohesive/loosely coupled (all the way down); Automated from code commit to cloud; Resource-aware; Monitored thoroughly; and Antifragile. “
 

If you have any questions then please do get in touch!

I’m very excited to be part of Oracle’s JavaOne event again this year, and I can’t wait to head over to San Francisco at the end of September!

This year I’ll be presenting a solo session on “Cloud Develop’s DHARMA: Redefining done for Cloud applications” (which is an improved version of the talk at gave at Skillsmatter earlier this year), and three other joint Birds-of-a-feather (BOF) sessions on OpenJDK Adoption, the JCP Process, and “How to make your Java User Group and Java More Awesome”

You can find details of all of the talks on my JavaOne conference profile page here.

 

I'm speaking at JavaOne

I’m speaking at JavaOne – Join me!

 

Last year at JavaOne I was fortunate to be invited to talk to theserverside.com’s Cameron McKenzie about a range of topics, and you can find a link to the recordings below [please note that some of the editing in the videos is a bit wonky, as it appears that I am not always answering the question asked – I can assure that you I was answering appropriately during the actual interview itself 🙂 ]

I look forward to catching up with old friends (and making new ones) at this year’s JavaOne, and so if you are there please come and say hello!

I once again had the pleasure of talking at Skillsmatter in early May, and this time I presented “Cloud Developer’s DHARMA: Redefining ‘done’ for Cloud applications”. I wrote about this on my company’s blog the night after I delivered the talk, but I’ve just realised I didn’t post anything here – therefore here we are. The synopsis for the talk can be found below.

As is always the case with giving a presentation at Skillsmatter, I very much enjoyed the experience, and there were some great questions and chat in the pub afterwards. Many thanks to all who attended – your comments and feedback are very much appreciated!

Skillsmatter have very kindly recorded the session, and you can watch my full talk here. You can also find a link to the slides on slideshare below.

Cloud Dharma Talk at Skillsmatter - Daniel Bryant

 

Talk synopsis:

Building applications for the IaaS Cloud is easy, right? “Sure, no problem – just lift and shift!” all the Cloud vendors shout in unison. However, the reality of building and deploying Cloud applications can often be different. This talk will introduce lessons learnt from the trenches during two years of designing and implementing cloud-based Java applications, which we have codified into our Cloud developer’s ‘DHARMA’ rules; Documented (just enough); Highly cohesive/loosely coupled (all the way down); Automated from code commit to cloud; Resource aware; Monitored thoroughly; and Antifragile.

We will look at these lessons from both a theoretic and practical perspective using a real-world case study from Instant Access Technologies (IAT) Ltd. IAT recently evolved their epoints.com(http://epoints.com/) customer loyalty platform from a monolithic Java application deployed into a data centre on a ‘big bang’ schedule, to a platform of loosely-coupled JVM-based components, all being continuously deployed into the AWS IaaS Cloud

If you have any questions then please do get in touch via the usual methods!