My colleague Luke Punnett recently recommended Dave Farley’s book ‘Modern Software Engineering’. While it’s not quite a classic, it’s a superb summary of the state of the art in software development. Anyone writing enterprise software should read this, and ideally follow the book’s advice.
Farley attempts to build a foundation for software engineering as a discipline using the scientific method. Writing code is formalised as a series of experiments, set within the process of ‘characterise, hypothesise, predict and experiment’: “Software engineering is the application of an empirical, scientific approach to finding efficient, economic solutions to practical problems in software.”
What was most valuable about this book for me was getting a glimpse of how a very good and experienced software engineer approaches his work (Farley mentions a couple of times that he was involved in the LMAX exchange project).
There are several topics on which the book is particularly strong. Farley approaches agile from a fresh angle, renewing my faith in it – a faith that had been ground down by SaFE and ritualised agile processes. Farley is also excellent on test-driven development, arguing that TDD is not about producing code coverage, but a method of working that produces “a pressure … to write code that is more testable“. Farley then argues that testable code has the same attributes as code that is easy to maintain. There is also some excellent discussion of the pros and cons of microservices, arguing that their main strength is in allowing smaller, more focussed teams.
Robert C Martin’s Clean Code feels like it has reached the end of its life. People are less comfortable with what some of the guidelines that it proposes. Farley’s book is, I think, an excellent replacement. It is short and well-argued, and sets out a clear case for its recommendations. Some of the things Farley proposes – TDD, microservices, test automation etc – are still controversial in some companies. Hopefully this book will help towards their wider adoption.
Category: books
Categories
Effective Remote Work: A Review
We’re now deep into the second year of what’s been referred to as “The Great Remote Work Experiment”. March 2020’s lockdowns came in suddenly, with many companies treating them as a temporary inconvenience. The long-term effects of new working practises are still not obvious.
James Stanier’s second book, Effective Remote Work is an important book, providing the sort of clear ground-rules for remote working which would have been incredibly useful at the pandemic’s start. The book is humane and thoughtful, looking at how to work productively while protecting one’s mental and physical health.
My personal belief is that a properly run remote team will always be more effective than a co-located team. Too many companies rely on having people in the same room to deal with complexity rather than looking at managing things properly. Remote work also gives employees control over their lives, allowing them to live anywhere, and to have a schedule that fits in with their life. Remote employees don’t lose hours each week in commuting to open-plan vanity offices. Most importantly, people who cannot manage a 9–5 in a physical office due to childcare, caring responsibilities or poor health are welcomed back into work by flexible remote companies.
While I’ve read a few books about remote working, Stanier’s is an obvious leader since it has such clear practical guidelines. These recommendations are all based on hard data where possible (such as a comparison of reseach on reading vs listening speeds). Stanier’s book is aimed at any remote worker, regardless of seniority — while management staff have more leverage to introduce new practises, Stanier also shows how even junior staff can produce changes, addressing the small areas of the company that they can influence.
The two most useful ideas for me were the spectrum of synchronous/asynchronous communication, and the ‘golden rule’. Stanier looks at the continuum of media available, from in-person meetings to formal documentation, and how best to use asynchronous media that don’t rely on people being around at the same time. Stanier’s golden rule is that if some employees are remote, everyone should be treated as such. No more meetings where the remote staff stare at a diagram on a whiteboard that they can barely see. This should not need stating — but too often I’ve worked with offshore teams who were treated as if they were second-class. Since the pandemic, most people hopefully have a better understanding of the challenges from remote working.
Another topic the book discusses in detail is onboarding, which Stanier describes as “a lens that we can continually apply to everything we do, all the time”. People always need to do new things, and the more effective the training is, the more easily anyone can switch to new work. (One worrying effect of pandemic remote working is a reduction in roles for more junior staff, which some have thought might be down to concerns about training staff with less work experience).
Effective Remote Work also has an entertaining writing style, and contains the best simile I’ve read in years: “sneaky like a small mouse in a trenchcoat with sunglasses on”.
I think the remote working revolution has only just started. While there are companies treating remote working as a temporary inconvenience, others have made the transition successfully, and who will begin overtaking less flexible competitors — in output, hiring and growth. Stanier’s book provides a clear framework for how companies and employees can prosper over the next few years.
Tldr; things looked bleak for Java in the mid-noughties, but the language and eco-system have undergone massive changes, allowing Java to thrive and survive.
My new job has a daily commute, which means time to read. I’m focussing on tech books and recently picked Bruce A Tate’s Beyond Java. I originally read this in July 2006, the year after it was published. The book described why Java was doomed, with Tate saying in the introduction, “Java’s reign will end. It’s not a question of if but when.” (pviii). While the predictions have turned out to be wrong it is interesting to re-read the book and see how Java escaped its fate.
Revisiting the mid-noughties was fascinating in itself: a time when AJAX was just emerging, Web 2.0 was the next big thing and folksonomies would change everything. It was an environment that caused obvious problems for Java. Its flaky text-processing was just no good for large XML files, and Java was forced to deal with a lot of XML. In addition, Java tended to be used for the middleware on top of databases, which it was not particularly suited for (p52).
Even with early versions of Spring and hibernate, making Java applications was a chore, with overwhelming amounts of boilerplate and XML config. Even with ORMs, Java required you to work with both entities and database code, and to keep them consistent.
Tate talked about this difficulty in getting software running, and all the tools that were needed. Compared with PHP, Java felt slow to develop in, and its advantages weren’t sufficient compensation. It did seem that Java was on the ropes. I remember dropping out of Java to do simple text tasks. Ruby-on-rails and similar frameworks promised the chance to get applications running quicker.
The book is a pessimistic one. Tate saw the problems as unsolvable and, given Java’s history and trajectory, this was a fair assumption to make. Tate looked at Java critically and suggested other platforms developers should investigate. Reading the book with hindsight, one can see Tate’s predictions have failed, but the discussion is still interesting.
In the 12-or-so years since Tate was writing, Java has managed a comeback. Decent dependency management from maven and gradle has eradicated the nightmare of setting classpaths. The language has been made more expressive with new features, such as the functional constructs in Java 8. And the growing fashion for opinionated configuration has also made things easier – getting Spring Boot running is incredibly easy. The culture of the language has changed for the better too. In the past, checked exceptions proliferated everywhere, and now people tend more towards throwing runtime exceptions and handling everything at a single sensible point.
One particularly strong improvement is the Spring framework. At the time Tate was writing, Spring was a mess of XML configuration and cryptic errors. With annotations, it has become easy to use and unobtrusive. New platforms such as Spring Boot allow developers to release production-ready features out quickly. At the same time, Java/Spring offers a mature platform, where code can be taken to production and supported at massive scale. Ruby-on-rail’s promises have faltered, with Twitter famously moving to Java because of scaling issues. The XML problem has disappeared as JSON takes over as a less robust, easier to use standard. I don’t miss the days of choosing between SAX and DOM parsers for XML.
Around the time Tate was writing, a debate raged about variable typing: strong vs weak, static vs dynamic. Java’s static and strong typing felt too rigid for many people. This debate seems to have reached an interesting conclusion: Javascript has been restricted by typescript while Java 10 has introduced the var keyword.
(I’ve always seen the inflexibility of Java as a positive. While it’s less expressive than languages such as python or, um, perl, the code it produces tends to look very similar. It’s a good language for non-rock stars – who massively outnumber the sort of people capable of producing cryptic code given the slightest chance).
At the time of writing, Tate saw Java’s fate as connected with that of Sun. Tate wondered what might happen if Sun disappeared, wondering who might take over Java. The resulting situation, with Oracle running Java (and MySQL!) was hard to predict but has worked out well – even with the occasional fake news scare that Oracle would start charging for Java.
Tate discussed a number of contenders for Java’s throne. Perl has lost popularity over the last decade, with Python offering a more structured (and readable) alternative. Perl, like PHP, was a great tool for its time but ultimately a ladder to better things. The book makes a quick mention of functional languages: Clojure seems to have come and gone, but Scala and Kotlin (first released in 2011) seem to be gaining traction. Ruby itself is much less popular. Java seems to have been saved by the flexibility of the JVM, which offers more powerful options for those that need them.
For me, the most interesting discussion was around state in web-servers, and the problems of that. Tate talks about continuation servers as an upcoming opportunity. This is another debate that has been settled, with REST winning out over stateful alternatives. I worked with Seam for a time – it was an interesting idea, but so alien to the way most developers worked, that its subtleties tended to be fought with rather being made use of.
One problem that Tate raised which is still present is the approachability of Java. Even in the innocent days of 2005, the simplest useful Java application required use of a large number of tools. This issue is present in a lot of fields, including front-end dev work, where increasing sophistication has made things much more complicated. There are solutions to this, with JHipster being a promising one, but these work to hide the complexity. Just because the tools are set up for you doesn’t mean they aren’t there and can’t go wrong in strange and unpredictable ways.
EDIT: Tom Hume pointed out that the discussion above doesn’t include Android, which was originally released in 2008 and has become hugely popular, with over 2 billion users. As someone who focuses on backend dev, I didn’t engage with this, but Android has been contributed significantly to Java’s survival. Having said that, Kotlin is becoming more popular in Android, threatening Java’s dominance on that platform. Interestingly, Kotlin is failing to see significant uptake in backend dev, despite recently becoming compatible with Spring.
If I had more time, I’d also look at the JVM platform as distinct from Java. While Groovy and Clojure have faltered, Scala (launched 2004) and Kotlin (launched 2011) are both popular JVM languages
Categories
Some notes on remote working
- Maybe there’s an irony to reading the 37 Signals book Remote while on a business trip. I’ve been remote working for clients since finishing at Intel. It seems a more humane way to work – and I’m convinced it’s also better for the companies. But, obviously, there is still some contact time required, which means business travel.
- For me, the biggest argument for remote work is removing the link between geography and employment. That means people can live where they want, with no need to commute. They don’t need to be near an office, removing pressure on housing.
- We should definitely be using technology to end commuting. All that travel is bad for the environment and produces stress. And not everyone can manage a daily commute – carers or the disabled cannot make long journeys every day. And, while I’ve had good days commuting, no commute beats quiet time with family and friends.
- I particularly like remote working as I have a nice flat and that flat is better for me than any office I’ve worked in. Sure, dotcoms sometimes offer laundry services – but I have a washing machine right here. The coffee is excellent, and there’s a chaise long when I need to read long reports. I also have flexibility – to work from friends’ houses, or a library, or a co-working centre like the Skiff. Research has shown that the length of the daily commute is inversely proportional to how happy people are with their lives. I’ve never been happier than I have since getting more control over my working schedule and conditions.
- Remote work is also better for companies. Many start-ups I’ve worked for have ended up as oral cultures, with documentation and processes distributed by word of mouth. Remote working forces companies to make things explicit. This makes their actions more efficient, repeatable, and reduces the risk of knowledge disappearing when employees leave.
- (Related to this are some interesting related questions about transparency within companies. Why should my email account not be visible as documentation for all my colleagues? If another employee seeing what I write is a problem, then I shouldn’t be writing it.)
- Remote working requires much more written communication. Many companies already worry about the amount of email, with some even instituting email-free Friday. I’m not convinced that email is the evil that some people are making it – rather, like meetings and managers, these are good things that are misused.
- (One of the best email strategies I saw was in the early days at Brandwatch. That company had the only functional corporate wiki I’ve ever seen. One reason for this is that they used mediawiki, which is a much more fluent tool than rivals like Confluence. Brandwatch also had a rule that questions should not be answered by email – instead you had to write it in the wiki and send a link. And, after a while, the wiki answered a lot of questions, cutting down on repetitive emails.)
- Another cargo cult of management is open plan offices. These environments are toxic for knowledge work, but companies still go for them, despite decades of research. Many times it’s because managers want to stop employees slacking off. This leads to a culture where people are rewarded for turning up, not for the work that is done. Remote working forces companies to be aware of what is actually happening. 37 signals talk about the vigilance needed for remote work – that small problems must to be addressed rather than allowed to grow. And this is not just in terms of work done, but also employee satisfaction and engagement. If employees are slacking off, it could be because they are not engaged. Forcing companies to be pro-active about these things can only be positive.
- There are excellent tools to support remote working, but these take some getting used to. Conference calls are tricky (as this site demonstrates). And slack is taking some getting used to: am I posting in the right channel? what should be a DM and what should be public? I’m also finding it difficult to know how best to work with remote reports.
- But Remote working is a skill and I am still learning. For example, I’ve discovered how much I rely on quick, informal chats rather than explicit briefings. But, as I get better at remote working, I become a better employee all round.
Categories
Book Review: Release It!
Michael T. Nygard’s Release It! was referred to several times at MuCon and keeps coming up when people discuss microservices. Despite being released in 2007 the book feels up-to-date. There was a lot of useful advice in here and, even when material was familiar, it was still entertaining. I wish I’d read this years ago – it would have saved a few mistakes.
The book is a good mix of instruction and case studies. Seeing how Nygard coped with real-life outages is both instructive and fascinating. The advice ranges from high-level (an excellent overview of networking) to low-level (when doing load tests, make sure some connections don’t log out).
Some of the sections I found particularly useful were the one on third party SLA’s and how to deal with them; QA vs production (one of those issues that keeps returning for me); the need to consider data purging from the start of a platform. There is also the discussion of the circuit breaker pattern, which seems to have been one of the most influential parts of the book.
As my responsibility has increased, I’ve needed to think more about the terrible things that might happen to a system. A lot of junior developers are perhaps too optimistic and books like this one are good for giving an idea of how subtly brittle systems can be and the need to develop a certain cynicism. For example, in a section on testing, Nygaard writes:
“A good test harness should be devious. It should be as nasty and vicious as real-world systems will be. The test harness should leave scars on the system under test. Its job is to make the system under test cynical”
Parts of Release It! feel like a horror novel for software developers. It will open your eyes to places where your software is vulnerable and how bad things might get. The book is also funny – laugh out loud funny in a few places. The best endorsement I can give is that this is one of those books you want to force everyone around you to read.
Last Saturday, while waiting to be picked up by my friends Joh and Simon, I was reading the original paper on Conway’s Law. It is one of the reference points of microservices and I wondered exactly what it said. How had Conway proved his Law? Talking about this in the car, Joh and Simon said I should read Laurent Bossavit‘s Leprechauns of Software Engineering.
In this small book (which is a good thing – too many software books are much larger than they should be) Bossavit investigates the common-sense things that ‘everybody knows’ about software development and finds that many of them are based on hearsay and poor citation. The book is a sort of Bad Science for computer programmers, what Bossavit refers to as “epistemic hygiene”. Some of the myths examined are the idea of the 10x programmer, the idea that bugs cost more the later that they are found, and the problems of the waterfall method.
I love this sort of research. I spent an hour during my MA investigating the often-quoted story that Tristan Tzara caused a riot by reading random poetry. Since my essay was on William Burroughs and detournement, I barely managed to fit the research into a footnote, but the work was fascinating.
Bossavit wants to train software developers to be more sceptical, and outlines his method. Of course, the big flaw with this book like this is that it demands a lot from the reader. Without going back and reading the original sources for myself, I can’t be sure that Bossavit’s claims aren’t hearsay themselves. But, even without doing that work, there is value in the critical doubt that it stirs up.
For me, the most interesting part of the book was the discussion of waterfall, and the suggestion that the attacks on waterfall are based on a straw-man. While I love agile, and feel that it produces more humane projects, a well-run traditional project can be more effective than a poorly run agile one. Indeed, one of the problems with agile is that failed projects are often dismissed as ‘not being properly agile’. As Bossavit writes: “Software engineering is a social process, not a naturally occurring one- it therefore has the property that what we believe about software engineering has causal impacts on what is real about software engineering.”