linux.conf.au Christchurch January 2019

Monday Jan 21

Toolmaking

Julien Goodwin

  • in your tool set have simple examples for future work

    • real, but simple uses of libraries

  • toolkit for later uses

    • things you can plumb together are great

  • write using whatever tool makes sense and/or you know

    • choose one or two as a primary so you get better at it, and you can share with others in your shop (i.e. try to standardize)

  • keep shell scripts to a few dozen lines at most

  • for other limits, no inherent limit, but use sensible modularity and testing

  • testing

    • testing large operation tools can be hard

    • splitting elements into testable modules, with their own fake implementations for larger system tests can work well

    • try to keep a standard, shared library constants

  • for any tool you will use more than once, also write some documentation

    • at the minimum this means code comments or a CLI help output

  • do some input validation (doesn't have to be perfect, but some basic checks are good)

    • catching gross errors and bailing with a usage message is fine

  • distribution

    • no personal tools for team problems

    • have a way to distribute so all team members can run and update

  • for many ops teams having a single toolmaking-focused engineer is a great idea

    • but a separate team of toolmakers is often bad

    • if scaling to a separate team have that team work on supporting infrastructure

    • in other words, the people that use the tool should be involved in making the tool

Python: Bringing your code to the next level

Jan Groth

  • use Python 3

  • use a tool to manage environments and use a dedicated environment for each project

  • aim for readability

    • use coding conventions

    • use descriptive file names

    • use descriptive names within code and don't rely on comments

    • break into methods

  • aim for beautiful code

    • use the language construct that fits best

      • e.g. use list comprehension instead of a for loop

    • comment the why, not the what

  • don't separate behavior and state: use classes as a blueprint for objects to capture state as well as behavior (methods)

    • re-use code elsewhere

    • use inheritance

    • write unit tests

  • use an IDE

What the dep is going on with Go dependency management?

Adam Harvey

  • several dependency management tools have come out over time

  • Go 1.12 (next stable release) will ship go mod as the default package manager

  • go mod requires semantic versioning

Developer Developer Lightning Talks

Adam Harvey

  • Valgrind automatically detects many memory management and threading bugs, and profiles programs in detail

  • useful for languages without automatic memory management such as C, C++, and Rust

Brenda Wallace

  • When they publish a law, they publish a reference implementation in Python (with unit tests)

  • When the law changes, they can run the tests and if it breaks they can fix the law

  • rules.nz

How Much Do You Trust That Package? Understanding The Software Supply Chain

Benno Rice

  • Supply chain issues

    • package source tampering

    • malware

    • availability (bugs in or unavailability of delivery mechanisms)

    • defects (aka code bugs)

    • lack of maintenance

  • Mitigation

    • support the maintainers

    • have a management process for third-party dependencies

    • have an upgrade/update process

    • have a process to review/audit third-party code

Automated firewall testing

Kristof Provost

  • maintainer of pf, a packet filter (aka firewall) in freeBSD

  • why automated testing?

    • make sure things actually work

    • convenient test case

    • prevent regressions

    • sanity check when making changes

  • tests should be:

    • easy to write

    • easy for everyone to run

    • fast to run

    • integrate with existing test framework

  • vnet is a virtual networking stack you can run tests against

  • jail isolates the environment in a container

Clockwork: Programming with rules

Mike O'Connor

  • language for programming custom industrial machines

  • issues:

    • long term custom hardware support is hard

    • commercial solutions use binary file formats for source configuration

  • requirements:

    • retain control

    • reduce risk

    • be future-proof

    • reuse don't rewrite

    • version control all source code

  • Result:

    • simple programming language

    • program pieces are called machines

    • machine can be simulated easily

    • a machine state resembles a physical machine state

    • reduces risk by programming in house, simulating as developing, reusing what was done before

    • future-proof by using Linux and open source software

    • version control everything

    • effective debugging tools

  • Clockwork

    • latproc language

    • objects are machines

    • describe machines by states

    • surrounding tools (shell, control daemon, interpreter, sampler, etc.)

Tuesday Jan 22 - Docs Down Under Miniconf

Being kind to 3am you

Katie McLaughlin

  • the essentials

    • write stuff down; it doesn't have to be fancy, just get notes in a place you can find them like a wiki

    • it needs to be editable right away (don't use a repo with PR process, for example)

    • make it searchable (e.g. don't put it in a paper notebook)

    • keep it secure (e.g. behind a company firewall) since it likely has content that the public shouldn't see

    • make it discoverable for other people in your organization who need to find it

    • keep your tools readily available; make sure nothing is dangerous, since you are likely to make a mistake

  • integration

    • think about ways to display information in various places (e.g. Slack reminder)

    • make integration contextual

    • perhaps integrate a documentation link with your alert system

  • post mortem

    • always loop back and update your documentation and tools after an incident

    • also help junior engineers with a feedback loop

    • focus especially on recurring issues

    • automate where it make sense

  • share your learnings with your team

  • make sure you always update your docs and flag changes to stakeholders

  • show empathy for your fellow engineers

Making Yourself Redundant on Day One

Alexandra Perkins

  • no matter what you do, when you start a new job you are always entering the unknown and don't know what is going on

  • why?

    • as a new person, you know the right questions to ask when you start

    • you won't remember the information the first time you hear it

    • documenting early makes on-boarding future employees faster and easier

    • robust internal docs fosters an inclusive and diverse workspace; it is an easy way to make people comfortable with finding information

  • what?

    • document anything you find hard about the job or the company

    • who is going to benefit from what you write? Consider the audience and write appropriately.

    • how to book leave?

    • who to ask about which topics?

    • information on workplace social events

  • how?

    • wiki or sharepoint if your company has them

    • files on your personal computer if you don't have a shared place

    • saved support tickets

    • notes to yourself on Slack

    • screenshots of useful Slack conversations

    • keep it simple; informal content is your friend

    • it is internal, so don't worry about making it perfect

    • run through your own content every once in a while just to make sure it is current, and to help refresh your memory

  • future proofing

    • create or improve the place where internal docs are stored

    • docs should be searchable and easily editable

    • transfer all the docs you did on your personal computer into an accessible location

    • make others aware of the work you have done

    • foster a culture of strong documentation. Encourage new hires to participate.

    • "if you email the update, you write the documentation"

  • quality

    • accessible

    • editable

    • searchable

    • peer reviewed

JIT Learning: It's great until it isn't

Tessa Bradbury

  • When should we learn?

  • JIT: Just in time

  • originally a process used in manufacturing to reduce cost and waste

  • JIT compilers are a similar concept

  • common process: write code => hit an issue => define a problem => find a solution => write code

  • but sometimes that doesn't work

  • assumption: you will ask the required questions

  • counter example: you can't ask if you don't know there's a question to ask

  • assumption: you can figure out what you need to learn

  • counter example: it is not always obvious what the problem is so it is difficult to define the problem

  • assumption: you can find what you need to know

  • counter example: it is not always obvious where to look for an answer; this can especially be a problem with proprietary software since you can't go beyond the docs provided (e.g. look in the code)

  • assumption: you know when you've solved the problem

  • counter example: you may not realize your solution hasn't actually solved the problem

  • assumption: the benefit of getting done now outweighs the cost of getting it wrong (aka Agile)

  • counter example: security

  • we need to think critically about what should be trained in advance and what can be learned JIT

Finding your tribe - choosing open source communities

Cintia Del Rio

  • Open Source refers to the product, the development process, and the community

  • just because a product is open source does not mean it has an open source community behind it

  • why you want to contribute can help guide where you will contribute

  • three types of code in public repos

  • source available

    • backed by companies where open source is not their business model, core maintainer and road map controlled by that company

    • you have very limited influence as an external contributor

    • communication and decisions do not happen in public

  • one person band

    • single (or very few) core maintainers

    • common for dev tools and libraries

    • single point of failure

    • very few resources

    • conflicts might not be handled so well

    • not really a community

  • communities

    • communication channels (forums, mail list, chat)

    • code of conduct

    • multiple core developers

    • GitHub org or a website

  • Is a project dead?

    • commits, issues, pull requests, communication channels

  • Culture

    • bad issues, declined pull requests, bad communication => how do people interact?

    • add "can you please" and if it makes a comment polite and not ironic than it is probably ok

    • is the code of conduct enforced? How hard was it for it to be adopted?

    • is non-coding work valued? Doing reviews, answering questions, writing documentations.

    • if material is translated, it is often a sign of a good, diverse culture

    • diverse cultures tend to be better

    • reacting well to minor changes (commas, grammar) is good

    • are there docs/resources for getting started as a contributor?

    • projects in "cool" languages tend to attract more toxic people

    • beware of jerks in leadership; they will form the tone of the community

    • is there someone to ask for help

    • "I usually end up working on things because I need them rather than choosing specific communities, though I have left some."

    • You are a volunteer; remember you can leave if you want or need to.

    • "Does it spark joy?"

Through the looking glass - how user testing can give technical writers better perspective

Fraser Mitchell

  • What is the purpose of technical writing?

    • give customers complex information in a format that can be understood and is engaging

  • technical writers don't read manuals from cover to cover

    • most learn software by testing it out for themselves

    • narrative of help pages are constructed based on this experience, but it is only the writer's experience

    • they also become more familiar with the product as they go along and forget what it is like for new users

  • think about your customer

    • users are searching for help, they're frustrated

    • with each unsuccessful click, they become more agitated

  • user testing lets you see the user experience first hand

  • contextual inquiry is the most common way of performing user testing

    • user is given tasks to perform

    • user is asked questions as they perform the tasks

    • develop a user testing script (unleash your inner talk show host)

    • collaborate with your support team and focus on customer pain points

    • keep the script open ended (e.g. what do you expect to happen when you click this button?)

    • don't use words that are in the UI as that can skew the choices the user makes

  • selecting participants

    • start internally, maybe with family and friends

    • external testers usually require an incentive

  • testing environment

    • monitor actions of the participant

    • have colleagues assist you

    • you only need 5 participants (studies show this finds 80% of issues)

    • keep the testing session short (not longer than 30mins)

    • don't be tempted to help the participant, even though it is normally what we do in our job

  • analysis

    • organize and digest the results

    • apply affinity diagramming methods

    • create storyboards or experience maps

    • what action can you take from what you've learned

The Art and Science of Diagramming

Adam Moszt

  • diagrams are visual communication

  • when you diagram, you need to consider your audience and anticipate their path

  • give them the information they need; don't go off on tangents

  • don't build a duck (aka a diagram is beautiful if it is functional)

  • a diagram that is too big exceeds the cognitive bandwidth of the audience

  • if you exceed the cognitive bandwidth of a person on a diagram they no longer can understand it easily

  • cognitive bandwidth is 7 +- 2 (aka 5 to 9 items)

  • create a consistent visual language for your audience to understand

  • try to prevent lines from crossing

  • keep visual styles simple and clear

  • style distinct elements distinctly or use emphasis

  • Gestalt law of proximity: if you put things close together, people tend to view them as a group

    • this can help extend the cognitive bandwidth

  • Gestalt law of similarity: people tend to view things as a group that look similar

    • for example you can color elements depending on who performs the action

  • ColorBrewer is a resource for picking distinct colors

How to avoid meetings

Maia Sauren

  • stereotypes are not untrue, they are incomplete

  • when you make a private joke without context, someone is left out

  • when you include everyone in a private joke, you create a closer culture

  • whose governance model wins? Consider who gets left out depending on what you choose.

  • it is harder to change a relationship over the phone than to maintain one

  • relationships with a code of conduct are less fragile

  • "how do you want to have arguments?"

  • model good behavior

  • set standards early

  • normalize conflict resolution or you are normalizing unresolved conflict

  • address issues early and often

  • adulting: it's for people who don't want to cry even more later

  • have "pre-meeting meetings" in order to resolve issues before the actual meeting

  • conflict resolution can work differently in different cultures

  • step away sometimes to let the culture grow organically

  • "culture eats strategy for breakfast" Peter Drucker

Any advice for creating relationships over the phone?

Disaster recovery book

Svetlana Marina

  • you can start with a simple guide: map alert types to dashboard logs and documentation

  • consider impact, expectations, SLAs

  • early and regular communication with the customer is crucial to maintaining their trust

  • add to guide: communication details (email or phone numbers, who to contact when)

  • add to guide: run book with resolution steps

  • perform RCA and raise a bug report if required

  • post mortem: this should have guidelines for how it is conducted

  • post mortem feedback should be used to improve can help keep the documentation current

The Bus Plan - Junior Staff Training

Andrew Jeffree

  • the concern is that new staff arrive thinking only about automation (e.g. let's do everything with Puppet)

  • but they don't really understand it

  • how do we give people the skills to execute on the documentation

  • you have to understand what is happening behind the scenes when you push a button in automation

  • this is especially important when the function doesn't already exist

  • it helps to spend some time playing with the technology without automation in order to understand it better

  • do things manually first, before automating

  • training

    • start with a blank server

    • give vague instructions (e.g. install Wordpress)

    • document the steps you used

    • then try extending it (e.g. multiple servers, SSL, backups, etc.)

    • don't be afraid of touching things manually

When Agile Doesn't Work Anymore: Managing a Large Documentation Project

Lana Brindley

  • docs age and we need to update them

Proof of concept - identify what the problem is - figure out how long it will take - get buy in from all the stakeholders

Plan - advertise your plan

Research - who is your audience? - talk to your audience if you can, run a survey

User/task analysis - what tasks do your readers have? - produce a matrix to see how often users consult the docs to complete a task - this helps us know where to focus our attention

Write - keep track of what and who is doing things

Review - What was done? What could be done better? - focus on the problem not on the people

Don't be afraid to break the Agile model when you need to. But if you are going to:

  • do outreach with stakeholders

  • track your work somehow in the Agile process, but don't go overboard

  • keep your work in people's minds and give kudos

Wednesday Jan 23

Keynote: #WeAreNotWaiting: how open source is changing healthcare

Dana Lewis

  • food affects blood sugar in about 15 minutes

  • but insulin can take up to an hour to take effect

  • even with current tools, like a continuous glucose monitor CGM and an insulin pump, there are a lot of decisions to be made throughout the day

  • the hardware options are limited (e.g. work on Windows but not on Mac)

  • hacked the device and uploaded the data to Dropbox then sent appropriate alerts to mobile phone

  • by adding a RaspPi and a battery, they could create a closed loop artificial pancreas system

  • so a computer is in charge of the constant decisions, removing human error

  • open sourced the solution as #OpenAPS (open artificial pancreas system)

  • they had to design for all possible failure modes

  • OpenAPS.org has plain language reference for doing this, as the audience is everyone with Type 1 diabetes and they may not be technical.

  • they have collected over 9 million hours of data from this system, and the data is freely available to anyone who requests it

Plastic is Forever: Designing Tomu's Injection-Molded Case

Sean "xobs" Cross

  • in order to understand how difficult a project is you need to understand it

  • discussion on the process of injection-molding a case for the Tomu

  • they first etch the design in copper, then use EDM to oblate the design into steel

Desktop to Mobile - Developing for Multiple Platforms without Losing Your Mind

Dirk Hohndel

  • Subsurface is a dive log program

  • UI elements on a mobile are different from the desktop. For example, when touching something you cover what you are touching with your finger.

  • QML is simple, but is lacks widgets, and is declarative rather than procedural

  • this makes it very difficult to use for anything beyond basic elements

  • Kirigami provides a tool kit and docs

  • Kirigami does not follow Android or iOS design patterns, which is good because it is consistent between them, but it is bad because it doesn't look native

  • no GUI tool to design the UI

  • QML developers are hard to find

  • connecting to C++

    • challenging documentation

    • full of pitfalls

    • magic names, magic rules

    • object creation heartaches

    • once it works it becomes easy

    • fix bugs once and it will work on all platforms

  • packaging

    • poorly documented

    • frequently broken in new Qt releases

    • magic everywhere

    • incredibly hard to debug

    • maybe the biggest downside compared to using native tooling

    • this is ignoring store and license rules

Game development with Lua, for Beginners

Seth Kenlon

Why linux.conf.au continues to amaze 20 years on!

Steven Hanley, Angus Lees, Hugh Blemings

Right to Not Broadcast

Karen Sandler

  • the experience of having an electronic heart implant changed her opinion from thinking that open source is cool to thinking that open source is essential

  • most of the implanted defibrillators broadcast constantly

  • but even if they don't, the computers that read them and hold data have a large number of security flaws

  • we have very limited options for people who want to opt out, especially when it comes to medical devices

  • we need off switches on broadcasting and aggregating data

  • we need to be able to revisit consent on a regular basis with a presentation of the alternatives

Thursday Jan 24

Keynote: Personal Branding for the Security Conscious

Shannon Morse

  • Hak5 founded on YouTube in 2005

  • if you don't control and manage your brand, others will do it for you

  • so protect yourself and figure out where you draw your privacy line

Open Source. Open Data. Changing lives for the better.

Liz MacPherson

  • Government Chief Data Steward and Government Statistician

  • authorizes all data published

  • data is critical to decision making at all levels (although decisions cannot be made on data alone)

  • trust is a critical element for a data providing body

  • openness is a large part of this

  • openX-ecosystem

  • open government data is open data that is non-personal and non-confidential

  • data is renewable, but it is only sustainable if trust is maintained

  • NZ adopted the Open Data Charter last year

  • data.govt.nz

  • what kind of regulations do you need to build trust so that people will share their data?

  • it is important that the group supplying data can access/use that data, and gains a benefit from it being available

  • this is particularly important for groups that are often underrepresented by data and traditional data gathering methods

  • the government ran an algorithm assessment report to help build trust by trying to ensure data was safe and that the algorithms did not skew results through bias

  • the results showed that the algorithms were generally quite simple, automated decision making tools

  • 'no' decisions tended to bring humans into the process; 'yes' decisions tended to be fully automated

  • recommendations included making both code and data publicly available

  • since it was early in the process, they could start with a mission statement and work towards it

  • legislation generally struggles to keep up

  • but 'soft legislation', like bodies that offer advice, are easier and faster to implement than actual law

  • NZGOAL (New Zealand Government Open Access and Licensing) framework

  • this framework has encouraged a shift to open source to begin for NZ government agencies

  • a strong open source community that is engaged with government data and processes makes a big difference

Finite state machines: a design pattern for FPGAs and React

Tessa Bradbury

  • at university, state machines were generally presented as theoretical concept

  • field programmable gate array (FPGA) is a programmable hardware chip with a lot of gates on it

  • finite state machine

    • a finite set of states

    • an initial state

    • a transition function (state, event => state)

    • a mapping of state to output (not an official part of the definition)

  • examples include TCP, bug tracker, set up wizard, games

  • states make sense to humans, as we understand different behavior in different states, and transitions between states

  • why are state machines so relevant to React and FPGAs? They are both event oriented systems.

  • state can help us avoid a many-to-many relationship in a system, as instead each function can make to the state

  • be intentional about your states, events, and transitions

  • draw a diagram

  • "finite state machines provide a useful abstraction to help us reason about the complex behavior of our systems"

  • "there are often a lot more similarities between different technologies than we think"

Go All-In!

Bdale Garbee

  • the collaborative process is key; no one company is in charge

  • this diversity creates a software commons that we can all benefit from

  • if upstream behaves unacceptably, the community can fork

  • collaborative development allows us to recognize the benefit of collaboration on non-differentiating elements, leaving more resources to deliver meaningful value to customers and users

  • surprisingly, innovation often comes because "amateurs" do something established industry doesn't believe they can do

  • the most successful projects are those built around open contribution

A case study in incrementing a language's major version: PHP

Adam Harvey

  • PHP 7 was a similar update to Python 3, where they wanted to clean up some parts of the language

  • while end users can migrate relatively easily (update, test, deploy), libraries have a harder time, since they often have to support both old and new versions for their users

  • break things cautiously

  • maintain terrible things if it makes life better

  • expand the zone of overlap

Mozilla Project Things Workshop

kathy giori

Friday Jan 25

Keynote: A linux.conf.au Story

Rusty Russell

Growing TypeScript and Rust

Jonathan Turner

Facebook, Dynamite, Uber, Bombs, and You

Lana Brindley

1space: Sharing data across clouds

John Dickinson

STM32 Development Boards (literally) Falling From The Sky

Mark Jessop, Michael Wheeler

  • earth.nullschool.net

  • radiosonde = sensors + radio transmitter

  • measures things like temperature, humidity, pressure, wind speed/direction (via GPS)

  • lifted aloft by weather balloon, usually filled with hydrogen

  • takes about 1.5 hours to ascend at 5m/s

  • burst at ~27000m altitude

  • lands ~30 min later

  • radiosonde keeps transmitting for another 6 hours

  • they descend at different rates depending on various factors

  • radiosonde_auto_rx

  • projecthorus/chasemapper