Bias to the server-side

Problem statement

I was recently working on a support issue, which had client- and server-side aspects. Complicating the issue, we only had partial visibility into server-side health and no visilibity into client-side health. It was hard to even tell where to start investigating. We were also working closely with a partner, who could give us some visilbility, but with high coordination cost.

One approach was to create client visiblity for us and the partner, but this would take time to roll out, didn’t immediately reduce the coordination cost and risked fatiguing the partner.

An alternative approach was to increase the server-side visiblity. We took this approach because we could start investigating immediately (no coordination cost or roll out latency). We might even be able to resolve the issue without requiring the partner to do any work. Also, having more visibility and confidence in the server-side would help if/when we do need to make client-side changes.

Solution

So, my takeaway is simple: when faced with a choice between server- and client-side options, bias toward server-side.

From a product perspective, any work required of customers is friction to adoption. From a technical perspective, it’s much easier to change servers than clients.

The value of a dashboard

Problem statement

A project I’m familiar with recently had a series of issues. Each issue was investigated somewhat independently. It was hard to share common code, share data across roles and track progress over time.

Solution

  1. Capture canonical queries in version control
  2. Periodically run queries, persist and visualize output (aka ETL)
  3. At a higher level, invest in tooling to facilitate such dashboard creation

The end result is much more awareness of the underlying data. Folks in different roles can see the data and ask questions, which often improves the quality of analysis. For example, we now review the dashboard weekly and look for changes as we roll out fixes. Because we now have a pipeline, we can also run different data sources through it to check the analysis.

Resume guidance

I’ve recently been reviewing student resumes as part of a university recruiting program and a few best-practices stand out.

Google has a good video on resume formatting that describes most of these practices. In these cases, I mention a time in the video.

Ok. Here are the best-practices:

  • Recommended resume format at ~2:30
  • Resumes are read in the context of a job description (example). Companies hiring for a job read a resume to see if you have the training and/or experience to do that job, so
    • You can briefly describe what the project was, but invest most space describing what you did and how you did it. Google recommends a phrasing at ~6:00: “Accomplished [x] as measured by [y] doing [z]”
    • Bold key words to highlight your toolkit (~5:25), eg “backend engineer”, “python”, etc, so the reader can match the resume to the role at a glance.
  • It takes a lot of effort to tune a resume for all applications, but may be worth it for the 1-2 jobs you really want
  • Ideally, your resume tells a story, eg some Python in year 1, more Python in year 2, created a service using Python in year 3 –> this person has experience in Python and is using it to start exploring service eng
  • Submitting a resume may feel like an impersonal process, but it’s just people on the other side looking for new teammates. That’ll be you after you’re hired 🙂 The points above try to make it easy for that person to see you’re a great fit for the job.

Squads

This is an organizational pattern I like:

  • 2-5 ppl
  • Cross-functional
  • Focused on a specific goal
  • Weekly demo to squad

I’ve heard this refered to as a “squad”, “swarm”, “e team” and “feature team”.

One of the nice things is the sense of comraderie from working closely with a small group on a specific goal. Another nice thing is the group dissolves after the goal is accomplished, giving the members closure and chances to try new things without switching teams. Another benefit is broad awareness of how a system works.

This pattern works well in a larger context:

  • Shared code ownership
  • 10-50 ppl
  • Focused on a specific, but larger goal
  • Fortnightly demos from all squads
  • Shared calendar for all squad coordination meetings

When a squad accomplishes its goal, the members dissolve into the larger group. Individuals can learn about work in the larger group by attending the fortnightly demos and/or sitting in on other squads’ coordination meetings.

For example, a product may be supported by several teams. To avoid exposing the org chart in the product, make a large team who’s goal is to make that product excellent. Define a squad for each significant work item. All members of the large group are free to contribute code to the product.

An underlying principle is alignment around customer value over specific products or features. Rather than a group of people owning a code base in perpetuity, regardless of the amount of work required, squads form in response to need.

A counter-example would be several teams supporting a product and that product having disjoint features. Another counter-example would be a large team trying to maintain the interest of all its members in weekly coordination meetings. Yet another counter-example would be lots of individuals working in isolation, esp if they’re doing so to avoid coordination co

Binary prioritization

Prioritizing unplanned work is hard, especially when we’re overwhelmed.

In this context, I’ve found it helpful to split things into two groups: important or not. My team calls this “binary prioritization”. We work on the important stuff and forget everything else; anything misidentified as unimportant will bubble back up.

Brandon Chu’s essay “Ruthless Prioritization” provides a helpful distinction: work within a project vs work between projects. The former benefits from binary prioritization; the latter from a more rigorous process, like OKRs.

This also reminds me of Amazon’s “Bias for action” principle, and the Agile principles. For example, Agile embraces change (“Agile processes harness change for the customer’s competitive advantage”). Binary prioritization enables us to embrace a high volume of change.

Bias for action

This is straight from Amazon’s leadership principles. It also seems related to binary/ruthless prioritization, the Agile principle of “Simplicity — the art of maximizing the amount of work not done”, and customer focus.

An ambiguous problem may have many possible solutions. We risk slowing to a halt investigating all of them. It’s helpful to regularly question if a given task is important. “Analysis paralysis” might describe this, but I prefer Amazon’s phrasing: “Bias for action”.

As a concrete example, in an interview, there may be many possible ways to solve a problem, but time is limited and solving the problem by any means is preferable to identifying all possible solutions without implementing any.

A pattern for reducing ambiguity

Here’s the pattern:

  1. Identify the owner
  2. Have the owner describe the problem and solution in a doc
  3. Invite stakeholders to comment on the doc
  4. If discussions on the doc linger, schedule a meeting to finalize
  5. Document conclusions in the doc, to facilitate communication and closure, and for future reference

This may seem obvious, but I often forget it, especially in cases where a task starts small and grows in complexity. A problem may seem too small to formally “own”. A solution may seem too trivial to document. Stakeholders may attend a meeting without context. A meeting may conclude with stakeholders feeling like they’ve expressed themselves, but there’s no actionable plan to resolve the problem.

Identifying one person from a group of stakeholders to own the project and be responsible for leading work to completion reduces organizational ambiguity.

Documenting the problem and proposed solution in writing reduces ambiguity by capturing ideas from a variety of mediums in a single, relatively objective place that stakeholders can comment on.

Documentation alone may achieve closure, but it may also spawn extensive commentary. Meetings are relatively expensive, but scheduling a meeting to drive closure reduces ambiguity by distilling commentary on the document into conclusions.

Documenting conclusions reduces ambiguity by rendering them in an objective form all stakeholders can agree on.

A few symptoms that indicate when this pattern might be useful:

  1. Endless back-and-forth in chat, bug comments, etc, which can give the impression of progress, but never resolve the issue
  2. Multiple and/or cross-functional stakeholders, which can obscure priorities
  3. Multiple people opining on a solution and/or answering questions, which can obscure ownership
  4. A problem that drags on, which can indicate it’s important, but inappropriately owned

This ties into larger discussions around project planning (e.g., managing planned vs unplanned work), and meeting efficiency (e.g., inviting stakeholders, assigning pre-work and clarifying outcomes), but the point here is just to succinctly identify an organizational pattern and when it can be helpful.

The Manager’s Path

I’m an individual contributor, but I want to better understand management’s concerns, so I’m reading Camille Fournier’s excellent The Manager’s Path. These are my notes.

Many think a neutral relationship with management is good because at least it’s not negative, but there is such a thing as a positive relationship w mgmt.

1-1 mtngs:

  • two purposes:
    • Human connection
    • Private conversation, eg feedback
  • I agree w the above two, and would add a third:
    • To ensure time w otherwise busy ppl; the junior person has the priority
  • Not for status
  • Prepare an agenda. I like a living doc, linked from the mtng invite. Items can be added and referenced any time
  • “Regular 1-1s are like oil changes; if you skip them, plan to get stranded …”
  • “Try to keep notes in a shared document” 👍 I like to link an agenda doc from the mtng invite. (Same for most recurring mtngs.)

As you become more senior, feedback decreases.

Appreciate the fact that current peers turn into future jobs.

Uncertainty
– common every 5-10 yrs
– lots of uncertainty in the world
– ultimately, we have to rely on ourselves

People aren’t good at saying what they mean in a way others can understand, so we have to listen carefully to words, and non-verbal cues indicating the person feels understood.

“Be prepared to say anything complex a few times and in diferent ways.” I’ve found such repetition frustrating in the past. It’s validating to see this advice.

Effective teams have good onboarding documents. Have new hires update the docs as their initial contribution.

“What you measure, you improve.”

Beware alpha-geek tendencies. In particular, the tendency to lecture and debate.

Mentorship skills:
– keep an open mind, since the mentee brings fresh eyes
– listen and speak their language. If you can’t hear the question being asked, you can’t provide good answers
– use the mentorship to build your network

“Tech lead is not a job for the person who wants the freedom to focus deeply on the details of her own code.”

“… the tech lead role may be held by many different stages of engineer, and may be passed from one engineer to another without either person necessarily changing his functional job level.”

“… we know from the title that it is expected to be both a technical position and a leadership role.” In other words, it’s not necessarily superlative, ie TL != best.

“The tech lead is learning to be a strong technical project manager… and [is] learning how to handle difficult management and leadership situations”

“Realistically, it is very hard to grow past senior engineer 2 without ever having acted as a tech lead, even on the individual contributor track… people skills are what we’re asking the new tech lead to stretch, more than pure technical expertise.” This stands out to me because of the tension between manager and maker modes, to use Paul Graham’s terminology.

“Being a tech lead is an exercise in influencing without authority …” Including building a psychological skill set for managing associated stresses.

“From now on … balancing is likely to be on of your core challenges.”

Currently, it feels like I’m working two jobs, manager mode during the day, and maker mode in morning and evenings. Regular, project-specific “cadence” meetings have helped reduce ad hoc discussions, fwiw.

Ah, a few lines later: “Some days you’re on maker’s schedule, and some days your on manager’s schedule…It’s very difficult to get into the groove of writing code if you’re interrupted every hour by a meeting.”

“Part of your leadership is helping the other stakeholders … respect the team’s focus and set up meeting calendars that are not overwhelming for individual contributors.” I’m very happy to see this in a book about managing thought workers.

Main roles of tech lead

  • Architect and business analyst. Design the system enough to provide estimates and ensure requirements are met
  • Project planner. The goal is to maximize parallelization
  • Developer and lead. Write code, but not too much. The goal is the project (and team development), not individual tasks

“Sometimes tech leads are tempted to go to heroics and push through obstacles themselves… [but] you should communicate the obstacle first.” I can relate with the former and appreciate the actionable latter.

“Teams often fail because they overworked themselves on a feature their product manager would have been willing to compromise on.” So, communicate.

“… most managers will expect their tech leads to continue writing as much code as before … It’s generally a pure increase in responsibility …”

The goal of a project plan is a “degree of forethought, in places where you can reasonable make predictions and plan … The plan itself … is less important than the act of planning.”

Take time to explain. No one who’s not actively working on a project should be expected to immediately know and understand project details.

Do a premortem as part of project planning. How could the system fail, and what could we do to recover?

“Having the focus to build something big yourself is a distant memory.”

The agile principles can be a healthy alternative to rigid process 👍 I think they’re great.

“… no two great teams ever look exactly alike in process, tools or work style” The best thing I’ve seen is an appreciation of experimentation and iteratively building a style that works for the current team. A basic project plan, ie list of tasks, also seems like a universal business requirement. Put another way, revisiting that plan periodically seems like a reasonable, universal starting point.

Qualities of a great tech lead:

  • Understand the architecture
  • Help build, but involve others
  • Lead decisions, but do so collaboratively
  • Communicate

“You want to encourage others on your team to learn the entire system … but you don’t always need to be self-sacrificing” There’s the need for a sense of balance again.

“Your productivity is now less important than the productivity of the whole team.” But how to improve the productivity of the team without putting on a management hat? Fournier gives an example: “Represent the team in meetings.”

Possession of communication skills differentiates successful leaders.

“Practice repeating things back to people to ensure you understand them.” I like this! I think it pairs well w advice earlier in the book to listen and observe non-verbal cues.

Communicate and listen.

I’d add that the tech lead label can also make one a focal point for questions, eg support, which can disrupt focus work. I like the pattern of having a support rotation, but depending on the company, the convention may be to simply ping the TL.

“Respect the ‘maker schedule’ for reports” 👍 As a general rule, I appreciate biasing toward contiguous meeting blocks.

Autonomy … is an important element of motivation.” I see this w external contributions too. Maximizing an integrating team’s autonomy frees them to meet their goals w minimal bottlenecks.

Politically-charged projects

I attended a talk yesterday that shared best-practices from managing two large projects that suffered from competing priorities:

  • prioritize, prioritize, prioritize <– reminds me of 4DX
  • put goals, tasks, vocabulary, agreements, engineering guidance, etc in writing to clarify communication
  • use pilot programs to clarify requirements
  • hold regular cadence meetings <– reminds me of 4DX
  • focus cadence meetings on clarifying requirements and getting help more than status updates
  • have periodic summits to build team cohesion
  • invite folks from different teams and with different roles to the summits to get a diversity of perspectives
  • identify and support “influencers” in the teams you need help from

One of the problems involved signing a contract before performing any engineering feasibility analysis and then having to turn the org on a dime to meet the deadline. 🦶🔫 I have experience being on the other side of abrupt changes and wondering what was going on, so it was validating to hear the context.

Saying yes to important things

I attended a talk today on the art of saying no and a couple things stood out:

  • saying yes to important things makes it easier to say no to less important things, ie prioritize
  • if the impact of requested work is unclear, request clarification before agreeing to do the work

This reminded me of something a colleague once mentioned: part of the art of saying no effectively is making everyone aware of the costs