Notes from The Lead Developer conference 2017, day 2

Continuing my notes from The Lead Developer 2017. You can also read the notes from day 1.

Kevin Goldsmith - Fail fast, fail smart, succeed

Invention requires failure. It's hard to something new perfectly on the first try.

If you don't accept failure how do you deal when it inevitably happens?

Create a fail safe environment. Not a "fail safe" as in "avoid fail at all costs".

Fail smartly.

Small controlled explosions propelling us towards the right direction. Not a nuclear bomb.

Failure: opportunity to learn.

Find the cheapest path to learning.

The Lesson of Clippy: it's not that they didn't know what they were doing. They were very clever people. Yet they got it wrong. But they couldn't figure out what was wrong. Against them:

  • Inflexible delivery times: trying to get it done in time for "golden master day" to send master CD copy.
  • Little and no advanced user research.
But today? You can ship multiple times a day. There's no real excuse to wait until you get the big fail.

Thomas J. Watson, IBM: "to increase the success rate, double your failure rate". IBM failed in parallel.

What is the culture you build and how do you punish (or not) failure?

Don't punish failure. Punish not learning from failure.

If there's no accountability, there's not learning from failure. This is wasting your time.

Each failure teaches you about...

  • Your process
  • Your team
  • Your perception of your customer
  • Your understanding of your product yourself
You make a "myth" of your product and customers... the only way to prove/disprove it is to ship and find the truth.

Even the customers don't always remain. They come and go. You might need to change your product in response.

Extract lessons. Do retrospectives for everything!


  1. Not only what happened
  2. but also how to make it better in the future (so you make better projects)
Catalog lessons. Create a repository of lessons learned. It can be a simple google doc. This is hard, but it creates a culture of sharing and not feeling bad when things fail.

Idea: the FAIL WALL. Show the mistakes you've made (in post-its, in a visible space).

Bob Sutton: "most creatives learn more from setbacks than from competitors."

Fail faster and cheaper. Minimise the cost of failure. Reduce the time you spent on building and shipping an MVP. So you can spend time tweaking and iterating and shipping again in response to clients' feedback.

Use things like paper prototypes. Don't build the whole future.

Ship it → Prove it → Tweak it... and back to Ship it again!

Roll to a small % of customers. Test and measure. If it's OK, roll to more. If not, adjust and roll again.

Concept: the MVP LOOP, while you finish the idea.

The investment won't go down to 0, because there's maintenance costs, etc.

The lesson of Spotify now: shipping in a rush, without the right data, wrong assumptions, a lot of investment that resulted in losing customers, and not knowing why.

H. Ford: "A failure: an opportunity to begin again, more intelligently."

Birgitta Boeckeler - "We're Agile, we don't do documentation"

Documentation seems to be a stigma in Agile:
  • "we're agile, we don't do documentation"
  • "our code is self documented"
But teams need to share a common understanding:
  • ... that third time Alice explained the same thing to someone
  • ... does anyone remember why did we chose X tech again?
  • describe software
  • how to use, install
  • for developers, are communication with each other
A documentation scale:
  • no docs
  • self documented
  • tests and readable specs
  • ...
  • UML
Think: what is the purpose of these documents? It shouldn't be for the sake of process.

Why should you document?

  • To create a common understanding on the team. Concept: the wall of common understanding. Everyone in the team should know about this. E.g. containers and tech stack, components, other essentials. What you explain to new timers and stake holders and doesn't change often.
  • To surface and understand complexity === things that aren't trivial, but are critical. More like text + graphics than a wall of text. Creating docs like this help you check you're not creating unnecessary complexity.
  • To build "widget kits", to explain things later. It's very useful for things with lots of history, it can be used to have conversations about the software.
  • To create empathy
    • between decision makers and developers. Brings tech requirements to discussion. Empathy driven development. Working without docs is anxiety producing.
    • between product people and developers. Explain the logic behind product decisions, and surface rules behind decisions.
    • between your developers, and other developers. Explain past architecture decisions for the future. It's otherwise hard to explain past decisions. You don't want to blindly follow a path that was set before by someone without explaining why. An example of how to do this: keep simple markdown files with the status of the decision, and the consequences.
      • This is where lead developers bring value: look up for those moments: will this be hard to change in the future? But describe the problem, not the solution. E.g. "we want to improve repeatability" vs "we installed a test runner".
  • To encourage creative problem solving. Two sides of the brain - different ways of thinking about problems. Concept: solving problems on the "back of the napkin" (Dan Roan). "The hand is mightier than the mouse". Once you solve the problem, throw the picture away.
    • A reluctance to document prevents you from "napkin viz", and this prevents you from solving your problem.
Question: how do you keep documentation up to date?

You can generate diagrams automatically but they aren't normally useful

Anything not necessary to keep the software running will be eventually out of date.

  • Document as little as possible.
  • Make it visible. A wall, a readme for newcomers.
  • Include it in "rituals" (weekly refresher of docs, etc)
  • Create ownership through collaboration: not just one person writing docs.
Do something your future self will thank you for.

Mathias Meyer - Building and scaling a distributed and inclusive team

How they're building a diverse distributed team at Travis CI

A push for remote helped increasing diversity, by hiring from different backgrounds. But there are two challenges: communication and culture.

Communication challenge

They use

  • github
  • slack "good morning!", plus lots of channels for cats etc
  • google docs
  • little email, not by design--it just happened organically
In slack, they have 'Chat Ops'. Channels for deploying: you can deploy software by talking to a bot. This:
  • increases visibility
  • reduces barrier for junior developers: no need to learn new tooling, if they already know slack
In GitHub
  • they track discussions in open issues.
  • hand off between timezones ("when Europe goes to sleep hand off to the US")
  • issues are used for onboarding: an issue with tick list for the first week. Each new joiner has an outline of what is going to happen.
  • used as "procurement" e.g. conference requests etc
This creates visibility, rather than emails. The problem is they have lots of repositories, and can be hard to follow. They try to keep things in issues, rather than emails.

They have a "how-we-work" repository  to discuss salaries, diversity, etc. Concept: it would be "continuous integration for company culture".

Concept: Inclusive decision making.

Book: "the open organisation".  Hear as many perspectives as possible. You benefit from diverse participants. It takes longer, and some discussions are never fully done (... like software).

Concept: distributed teams incur a tax on communication.

Put offline discussions into writing.

Book: "Turn the ship around" by David Marquet. Don't move information to authority, move authority to the information. People need to be able to make quick decisions without waiting for approval.

Less oral communication means more accidental documentation.

Concept: distributed facetime.

Monthly all hands (it's a Hangout)

Continuous improvement talks every ~2 weeks (instead of brown bag lunches). People attend if they are interested in the topic.

Biggest challenge: inclusive communication. Culture challenge: bridging cultures.

Rotate on-calls. Use distributed timezones.

This fosters documentation - run books to update/install services, etc.

Blameless culture = learning culture

In a complex system with humans interacting there's always more than one root cause. Don't blame people for mistakes.

Engineers work in customer support 1 week per quarter. Engineers feel customers' pain... and successes too! Break walls between developer and customer.

Culture differences

  • German: "This is what is wrong"
  • US: "Amazing!"
German says: "Needs improvement", US person understands "you're going to be fired". You need to find somewhere in the middle.

German language is "on point", very specific (e.g. long compound words). English is very ambiguous. Add to it when speakers...

  • are not native speakers
  • do not know native colloquialisms
it's hard to respond/parse lots of realtime input (includes videoconference, slack, irc)

The tone and nuance can get lost in writing, or accidentally added

Don't attribute to stupidity or malice what is explained by a simple perspective (i.e. you may be missing context)


Sabrina Leandro - Big rewrite strikes again

Lessons from big rewrites: big learning is "how not to need a big rewrite again"

Why does code need rewriting? The only "done" code is the code that is not in production anymore

  • over engineered
  • slow tests
  • not enough tests
  • not structured
Sarah Mei's code that is like "the hoarder's house" - no space to do things

"If only I could throw it all away"

Is this the right time to rewrite?

Invest in maintainability, but find the right time. Does the product have business value today?

Find the business case in rewriting. If you can't, you just want to experiment and learn, so maybe find something else to learn on. Look for business value with stakeholders, product owners, clients...

A good reason can be "easier to onboard"

Sell the rewrite. Have a one clear business case:

  • how will the world be after this is finished
  • find allies outside team
  • make pain visible, with graphs or metaphors
  • do repeat yourself (clear message) each time you talk to different people
How to run the rewrite?
  • Have a clear tech vision for the future
  • Guiding principles
  • Decide what is right now
  • It is a process
  • Prototype if unsure. Timebox prototypes. Agree with the team.
Technical approaches
  1. rewrite from scratch (value is only provided when 'done')
  2. progressively replace parts (the "strangler vine" concept from Fowler), this delivers value continuous and immediately
Incentivise picking features by priority order. Adapting + changing direction + pausing is possible.

Deleting is faster than rewriting. Reduce scope. Work only on what is needed. Finish faster! Maybe users don't value some of the features. You will not rebuild everything in this rewrite.

What to do (a plan)

  1. Find first blocker
  2. remove
  3. repeat
Finish things.

Migrate clients using discontinued features.

Cut the features out.

Show internal progress

Get support

This is not an exercise in technology

Celebrate successes!

Repeatedly go back to your goals and remind yourself why are you doing this


There is no "finish".

Pick changes with higher impact first.

Rewrite = opportunity to focus

Adopt good habits.

Software entropy is a reality. Try to get ahead of that.

Make cost visible. Create a product inventory and determine the level of maintenance. Who owns it? Review this each quarter.

Do you have more products than teams?

If code has no value, delete it.

Review products too. You can deprecate stuff. It doesn't say anything about code quality. You learnt something.

If we were starting over, what would we build today knowing what we know now?

Encourage day to day cleanups. Housekeeping.

"A refactor a day keeps the rewrite away" —don't ask for permission.

A culture of collaboration: users / devs / PMs.

You were not hired to deliver lines of code, but to deliver value.

Patrik Karisch - continuous accessibility

Automation for test complements manual tests.

allym is a module to generate accessibility reports automatically, maybe you could create bugs automatically on new low scores (if run on a continuous integration system)

Carly Robinson - Mentoring Junior Engineers at Slack

Went from acting into software engineering

What are software engineering practices that foster mentorship?

Places where people learn coding: codecadamy, hackbright.

To her surprise she liked coding! her ballet background taught her the importance of a strong foundation.

Junior engineers are listening to The Horror Stories. Mentors should know these exist and are in juniors' heads.

When she was in theatre, she used to get rejected-had to develop a 'thick skin' (similar to tech...). But many people with potential are scared of entering tech.

Companies often promise mentorship to juniors but in practice it often falls down, e.g. in startups.

The engineering culture plays a big role on how fast engineers

  • ramp up
  • can make (big) mistakes
What made Slack supportive?

A culture of empathy, craftsmanship, solidarity

Mentorship is a relationship.

Set expectations. Am I a good fit? You should be able to "leave" the relationship.

Decide on communication style.

Mentee: what are your goals? where do you see yourself in a year? what do you expect from the mentor? If the mentee is shy, they should tell the mentor, so they don't think something's wrong with them

Mentor: What is your learning style? What about pair programming? Do you tend to throw someone to the fire? Agree on this with the mentee

Juniors might not have thoughts about any of this. They might not even know how to ask good questions. Junior: what does success look like to you?

A mentor should share, develop respect and ownership in collaborator.


Build a list of high level goals, and the associated progress.

Goals should be accountable to success. It's easier to track progress and acknowledge success - so you feel less like an impostor

Code review culture

Root comments in a "lesson" so it shows you're sharing knowledge and making them a better engineer

Internal "idiot" reaction = "I don't understand why they made their choice. Maybe they know something I didn't?"

Trajectory for junior

Six month stretch = when impostor syndrome can kick in

Give them a stretch project. A bit uncomfortable, with relatively high visibility, and potential for failure.

Holistic approach

Growth infrastructure -- a culture to make better citizens of the world -- a positive culture

  • Personal and professional development fund
  • Brown bags
  • non tech workshops
  • office hours with top engineers
Slack = a culture of learning vs a culture of academia

Arrogance can be mistaken for intelligence

"Battles of the brightest" lower individual morale

When hiring, have zero tolerance for arrogance

Prefer: humble, tolerant, knowledge sharing. Craftmanship, courtesy, humanity, empathy.

A growth mindset: thrive by not feeling stupid. Accelerate junior engineers progress.

Randal Koutnik - Implementers, solvers and finders: rethinking the development career path

Going into management shouldn't be the only way to move people up. This often turns excellent software developers into mediocre managers.

But what does 'senior' mean? Expensive? Bored? 'Opinionated'? Years of experience?

What should it mean?

  • something quantifiable
  • gives a path forward
  • describe the work they do
A career path:
  1. solution implementer: problems are broken down already
  2. problem solver: break problems into solutions + implement
  3. problem finder: given a context, prioritise and find issues

Crystal Huff - Tech hiring is sometimes an F- experience

"Frivolous" could be understood as
  • I don't have anything to offer you, or...
  • I have a different perspective from you
Think: what would it be like to hire someone who's very different from you in this company? Hire a consultant. Talk to colleagues about it.
  1. Cultural upbringing
  2. Unconscious bias
  3. Response (internal) - reinforcing kyriarchy
  4. Considered external responses - conscious control
  5. More consideration - to control 1 to 3
You can't control feelings, but you can change behaviours... and thoughts
  • Plan hiring process before you're in the middle of hiring
  • Think of questions to ask
  • Learn about candidates. Get past the initial impression. Know them.
If you don't have people in leadership that don't look like you, it's very hard to recruit. It says a lot about your company.

Also: non diverse companies perform worse in every way.

Sally Jenkinson - From execution to strategy

Senior people:
  • experienced
  • responsibility
  • speak to people
  • detective work or executing?
  • people that are respected and trusted
Why move into this?
  • an specific opportunity
  • lure of different types of focus
  • want more power to make things better
  • better benefits
Difficult things
  • new responsibilities and challenges
  • meetings!
  • maybe you're not good at that yet
  • maybe you're bad a those and estimating
  • embracing a new identity: tech or not technical?
Your own prejudice

"developer snobbery" === feeling less technical was making her "feel stupid"

But her new functions made her learn lots of new things, now she can facilitate things for other people

Cut yourself some slack

Give yourself credit for learning to be a better manager

Able to talk to senior people and follow through

Burning out

Staying in touch with tech is tiring

Be careful. It can be lonely.

Find support in your org. Or outside.

You're not alone.

Be mindful of the above

Find your sweet spot. The speaker loves helping people, help them understand their options; she's confident in her strengths. So that's a good starting point for her.

The day to day developing helps getting better at strategic work. E.g. setting budgets, componentising, identifying dependencies, refactor, optimise...

You can be a better supporter of developers because you've been one, and developers can trust talking to you, and you'll be the bridge between them and other teams.

Also you can understand when something should be a tech or not a tech solution.

Set strategy -- to build more robust architecture.

Share your perspective with your reports.

When speaking to people:

  • Assess impact
  • Considerations
  • What else should they know?
  • Don't use jargon, people come from different backgrounds
Bag of advice
  • gradually expand responsibilities, spread out
  • have friends (or make friends) in other circles
  • learn to communicate with different people with different views
  • stupid side projects are good
  • don't beat yourself up
  • teach others: mentoring, kids...
  • smash the "tech/non tech" boxes
  • watch for burn out!

Jill Wetzler - The inclusive leader: tips for developing diverse teams

(opens with a terrible graph depicting the lack of diversity in tech)
  • Earn trust
  • Retain people different from you
  • Be considerate with underrepresented groups
  • Grant trust, but don't assume you're trusted
  • If people don't feel safe, they're going to leave
  • Offer authenticity, get authenticity
    • Forms a connection with reports
  • Cultural awareness is part of your job: pronouns, religion, etc...
  • You can't deny ignorance of their concerns
  • Diversify media, blogs, your networks
  • Join support groups to hear what's affecting them (you don't need to talk-just listen)
For tough personal conversations: read Managering in terrible times from Lara Hogan

What to do when safety is violated? Say: "I'm sorry!" and STOP, and read, and do better next time

If it's other people's safety that's violated: Allow the person who suffered harm to determine response. And put people in a safe position.


It is your job (specially when it sucks).

Learning how to receive feedback is good.

"Fear of the tears" -- ask "are you okay?"

Add structure: same for all and helps avoid bias

  • Observation: facts! indisputable
  • Impact: outcome as related to the expectations
  • Expectations: should be fair for their level
  • Assistance: take an active role in development
Check yourself for bias:
  • Would you give the same feedback to someone of a different race or gender?
  • Are you asking someone to be something they're not?
  • Have you made a statement about who they are or tend to be? ("personality penalties")
  • What feedback should you give to others too?
What if you get biased feedback from others? Filter, contextualise, eventually ignore if need be


The 70/20/10 rule: how you advance

  • 70% stretch assignments
  • 20% interactions, mentorships, sponsorship
  • 10% classroom learning
The problem: women do not have access to stretch assignments--they are getting penalised when they ask for them!

Solution: 20%: leveling up others is your job. Actively sponsor people unlike you. Talk about them. Bring up their name.

  • volunteer them for new roles or projects
  • share their interests with influential people
  • publicly endorse them for things they're good at
  • hold senior team members accountable for sponsoring others
Watch Janice Fraser talk: Female career advancement summed up in one usable diagram from Calibrate Conf 2016

Tom Booth - centralising the right things

Phase 1: all stuff centralised
  • Are there competing teams in your organisation?
  • Protectionism built through competition
  • The chain of blame and mistrust when things go wrong
  • Poor central tools
  • When centralised, getting features to the user is slow
  • One team owning the whole service
Phase 2: decentralising all the things
  • continuous delivery is good for users, business, and not being blocked on others is great...
  • ... but communication between teams went bad
  • solving same problems same way, but with different implementations
  • Operating at scale requires knowledge
  • No one has complete practice
  • Not focusing on users...
Phase 3: recentralise
  • New cross-team team
  • Still team ownership
  • Identify common patterns in infrastructure
  • Build tooling
  • Teams to support each other
  • Teams: room to experiment and do what's best for them
  • Help other parts of the company
  • Focus on value again
  • Work together, not apart
    • specially when architecture issues or bugs show up in production at 3 AM

Lara Hogan - Leading by speaking

Tells her experience speaking in public for the first time on a big event - they got confused with another speaker, so she was literally an impostor 😂

The spotlight as a developer comes in many forms:

  • stand ups
  • reviews
  • pitching
  • managing up
yet public speaking is the #1 fear
  • What do you want to say
    • Write a structure. Back it up!
      • Landscape
      • Analysis
      • Problem
      • Options
      • Solutions
      • Reason to believe you
    • Set expectations
    • Hold their attention
      • Jokes might not be relevant, you don't need to be funny
      • Be authentic
      • Keep them engaged
      • Delight yourself
        • A post it, something that makes you feel good
      • Don't distract (GIFs, jokes)
  • Practice + Feedback
    • Find a small group of 3-4 people (not just people who speak highly of you) to practice delivering your message
      • Humans are bad at giving feedback
      • Decide on a feedback medium
    • Maybe record the talk and send it to friends
    • If you practice in the room, it gives you a chance to use non verbal feedback
    • Ask them:
      • is this too complex?
      • does my argument hold?
      • am I speaking too fast?
      • does it make sense?
    • Practice surviving a fumble - don't restart again. build confidence in case it happens on the stage
    • Prepare your brain to hear feedback. You should have decided when, the medium, how...
    • Good feedback: specific and actionable - how does it add value to get better?
      General Specific
      Positive ♥︎
      Negative ♣︎
      ask for ◆ and ♠
    • Practice answering follow up questions. What are you afraid of?
      • Not knowing the answer
      • Etc
      • You can say "I don't know"---it's better than bullshitting answer
      • Remember you're still in control during Q&A
      • To practice, ask your feedback group to get weird and ask weird questions: "imagine ways people can get things or points wrong and ask me questions"
  • Prepare the environment
    • Wear what makes you feel like a superhero
    • Back up everything: offline, screencasts of demos, etc
    • Power poses for 1 minute
  • Once done... celebrate!
    • A big sigh of relief!
    • Eat a donut 🍩
    • Celebrate your achievement
We need a spectrum of new voices (diverse) - so either you speak or help others speak!