Dev meeting – Data intensive applications; and Rust

At our dev meeting this week, Rich talked about ‘Designing Data Intensive Applications‘, that he’d been reading, and Inigo talked about ‘Talking to your TV using Rust‘.

‘Designing Data Intensive Applications’, from O’Reilly, describes various tools and approaches for storing and processing data. It’s split into four sections. Section 1 discusses the basics of databases and how they store content, for SQL databases, NoSQL databases, flat file stores. It covers in-memory stands, b-trees, indexing structures, and the best approaches to use in various situations. Section 2 talks about distributed databases – clustering, transactions, serializing commits at a point in time so shared logs can work, and so on. Section 3 discusses distributed batch processing, using tools like Spark. The first three sections were all very interesting and informative. The fourth section was not as good – it covered the future of data, and was a lot woollier than the previous sections. Rich found the book on the whole good.

To talk to his TV using Rust, Inigo used a Google Home Mini, an RM 3 Mini IR blaster, an Open Source library to interface with the IR blaster, and some Rust on a Raspberry Pi. The Google Home part was setting up a Google Home App via https://developers.google.com/actions/building-your-apps – you configure a set of actions and intents online, and then assign a ‘fulfillment’ to it, which is a web service you make available via HTTPS. This part is fairly straightforwards – it requires filling in various web forms, and it’s easy to publish a test version of an app that’s only available via your own Google account.

To actually use this for a home application, you need to have SSL, so you need a domain name and a certificate. Inigo used Amazon Route 53 for the domain name, and Lets Encrypt for the certificate. There are various other services that will provide a domain name for a dynamic IP address for free, although the original dyndns is no longer free.

THe ‘fulfillment’ part is a web service that Inigo wrote that accepts JSON from Google, and then calls out to a Python library that interacts with the IR blaster. He wrote the first version in Scala, to get something simple working fast, and then rewrote it in Rust. He learned from O’Reilly’s Programming Rust book, and used IDEA’s Rust plugin. The experience was positive, although it was a bit tricky to get some of the memory management working correctly as a first-timer using Rust. The code is on Github at https://github.com/inigo/googlehome-remotecontrol.

How to read more interesting books

In our dev meeting, we talked about interesting development related books we had read, and how to read more interesting books. Various people each talked about a book that they had read recently or recommended.

Chris talked about “The Nature of Software Development“, by Ron Jeffries.

It’s by Ron Jeffries, one of the signers of the Agile Manifesto signer. It’s a good book to give to people who don’t know about Agile, and who might be scared of Agile jargon – it’s entirely jargon free. It explains the fundamentals, and the reasons why software development is better done Agile. It cuts through the cargo culting that is common with Agile implementations.

Tim talked about “What to look for in a code review“, by Trisha Gee.

The author is from Jetbrains, and there is a little undercover marketing for Jetbrains tools, but it’s not excessive. It’s written pragmatically (although not from the Pragmatic Press!). It’s very short, which is a plus, particularly if you have a young child and limited reading time. It is split into chapters based on things to look for in code reviews, like tests, security, performance, good principles. Although it is fairly Java focussed, it is still useful for any language. The tips are fairly obvious to most experienced developers, but there is some good stuff in there.

Pasquale talked about “Clean Coder” by Bob Martin.

This is about “saying no” and “saying yes” – the language used when committing to a task – you should say what you’re going to do and then do it. It recommends and explains Test Driven Development. It recommends coding practice, using katas. It discusses time management and managing your own time; and doing estimation. It is good for intermediate developers.

Loic talked about “Designing the user interface“.

This is a classic of Human-Computer Interaction. It’s primarily aimed at students. It covers a little of everything you need to know about HCI; usability, mice, collaboration, etc. It is useful for all developers to learn a bit about usability. It’s 20 years old, and a bit dated – but still relevant.

Reece talked about “The Design and Evaluation of C++“.

This book describes how the early C++ compiler was created, and how various features were designed, and the compromises made to overcome problems. It goes up to C++ 98. It is good for anyone interested in the history of programming languages and how they are created and evolve.

Simon talked about “My job went to India” (now renamed to “The Passionate Programmer“).

Simon read this about 10 years ago, and still recommends it. It is organized into 6 sections, with 52 tips across that. It is useful to think about how to do more than just what’s needed right now in the short-term. Some particular tips are: “Be a generalist” – don’t get bogged down in particular technologies; “Be the worst” – have better people around you; “Be a mind reader” – think about the software being developed and spike out future requirements that haven’t yet been considered, and think about what a customer might need, but they aren’t saying; “Be where you’re at” – do your job well; “How much are you worth” – was the work I did today worth what I am paid? The book could be read by anyone working in tech, but it’s aimed more at programming than other tech jobs. The ‘outsourcing’ part of the title is just a marketing vehicle.

Stephen talked about the “Mythical Man Month

It was written in 1975, but is still true, although a bit dated. Its core message is that a job that takes two months for two people, does not take one month for four people – because of communication, training, etc. It famously coined “Brooks’ Law” – adding more people to a late project makes it later. That’s still true, although perhaps less so nowadays since people it’s easier to split up tasks and make them independent. Sometimes you think “I could have done that in an hour” – but it actually takes a day: because it takes nine times longer to write good quality deployable shareable code, than it does to write something for yourself only. A modern insight on this is that reuse of code, better tools, and TDD can all help.

We had technology problems talking to Bart, but he passed on his notes on “Algorithms To Live By“.

  • It is a book on algorithms that either technical and non-technical people can get an insight on most important algorithms in computer science. Algorithms are presented in form of brief history and application in daily life. For people with computing background this could be a great supplement to already gained knowledge , i.e in the university. For non-computing people this would explain that computer science is not only about installing/fixing Windows. Some of the chapters most interesting to Bart are:
  • Optimal stopping – you will learn the most efficient methods to employ secretary, find a wife or parking space, or sell the house
  • Explore and exploit – you will learn how medical trials with unknown drugs are conducted, when to stop looking for a good restaurant, how the founder of Amazon explains Regret Optimism
  • Caching: How to store data, clean up the house, and human memory issues
  • Scheduling – importance of time, various methods to resolve problem with tasks dependency, how Pathfinder stop responding and was fixed while on Mars
  • Overfitting: how not to over-learn (self and machines)
  • Randomness – how to calculate PI by dropping a pin, how playing cards helped in building the atomic bomb, how to find gigantic prime numbers
  • Games theory – recursion in humans decisions, the “Prisoners’ dilemma” explained

Rhys also wasn’t present, but passed on his recommendation for “Functional Programming in Scala

This discusses how to solve problems in a functional way, how to think about combining functions in algebraic way. It includes exercises too. Particularly when coming from a Java / procedural background, it’s quite a mind shift.

We also briefly discussed why those particular books:

  • Chris had seen it on PragProg.com, and read about it on the mailing list, and liked Ron Jeffries as an author.
  • Tim has a young child and wanted to read a short book.
  • Pasquale realized that human processes around coding are important, and had already read and liked Clean Code.
  • Loic studied HCI at university, and it’s a heavily recommended book from ACM curriculum.
  • Reece is interested in language design, and this was written by the creator of C++. It was interesting to see thought processes of language designers.
  • Simon was doing a lot of Ruby development at the time, and reading blog posts by the author of the book, and found his attitude of interest – and the book was unexpectedly good.
  • Stephen read Mythical Man Month because it was famous.