From the Trenches

Christoph BadurapkgsrcCon 2007

Introduction

We put lots of effort into improving
pkgsrc technically

But don't talk much about our user's needs

Tell about my experiences and thoughts about the matter

A bit of background

Pkgsrc user and NetBSD developer since late 90s

Spend lots of time on business travel in the last years

Use NetBSD on laptop for day-to-day work

Background (cont.)

Limited time to work on NetBSD

pkgsrc user with commit bit

Usage at company

Have some mission critical servers running NetBSD

Usually install & forget

Admins not expected to update perfectly working systems

Old NetBSD branches (2.x until recently)

Old pkgsrc trees

Who are the users?

Hobbyists

Students

business users

Sysadmins

Consultants

What the users want

Install working software with minimal effort

Pkgsrc developers vs. Users

Developers spend lots of time on

Improving the pkgsrc infrastructure

Maintaining packages / fixing bugs

Constantly changing system is OK for them

Do not necessarily use the software they maintain

Neglect installation/update system

Admins aren't SW-developers

Admins often have no formal CS training

Often have trouble compiling software

Low level of automation for system administration

Admins often build machines manually

Automation using YaST or similar installer

Often no automation for local packages/software

SW-Developer's aren't admins

Have no concept of large scale system administration

Often have no understanding for doing system administration at all

Don't build machines

Think that their software materializes magically on production machines

Don't package their software

Benefits of using packages

Show developers/admins that using binary packages is a good thing

Can move software easily between machines

Can automatically install identical machines

Have documentation what the state of a machine is

Corp. users want stable environment

When you admin 600 servers you can't update them daily

Typically machines are install & forget

Only apply security updates

And fixes for bugs that affect production use

People don't get paid to constantly update software

Management expects them to do real work

Risk of down time

In case updated software is buggy

Or not compatible

Quarterly pkgsrc branch is to short-lived

Distribution life time

There's a reason why enterprise Linux distros are maintained for 3-5 years

Pkgsrc needs more man power to maintain branches longer

More a question of attracting man power

Interacting with users

How to get a usable system

Q: I know SuSE/Debian/Ubuntu/etc. How do I start the software installer for the packages on NetBSD?

A: ???

How to update a system

Q: How do I update the packages on my system to the latest ones?

A: ???

Need a package manager

Users want something like aptitude/YaST

Easy selection of packages to install/update/deinstall

Actual installation/deinstallation done in batch mode

Keeps track what the user installed on system

Working with PRs

Creating a PR takes time

Easily 15-30 minutes

Each PR likely means several users have the same experience

Only single person working on the PR

Closing a PR is heavy weight action

Can't be opened again by the user

Additional comments are easily lost

Reasons for closing PRs

Didn't try to understand what the underlying problem is

Didn't understand what the user was trying to say

Making outrageous policy claims

Maintainer not interested in improving usability of pkg

Telling submitter to go away

Bad defaults

/usr/pkg/xorg/bin

Extract xsets; rehash; xterm

Install xorg pkg; rehash; xterm

xterm not found

Both can't meaningfully coexist

Renaming packages

Interferes with updating systems

make update fails

Automation fails because package lists get out of sync

Keeping package lists in sync between different environments

Yearly package update

About 60 packages plus depends

Takes me about 3-5 work days

Just to get where I was before the update

Version mismatch

Update NetBSD to new minor release

Install binary package

this package was compiled for a different OS version. Consider yourself lucky if it works.

Developing packages

Great build system

Bulk builds ensure that most of the packages actually build

Bulk builds ensure that we have lots of binary pkgs

Slower arches have difficulties keeping up, though

Build machinery not separate

Individual package Makefiles still know to much about build machinery/other packages

Difficult to mix old and new versions of pkgsrc

Package options

Better than under FreeBSD

Tends to pop up dialog boxes at inconvenient times

Bad for bulk building binary packages

Unless all options are selected

Otherwise binary packages end up with wrong option set

Postfix w/TLS/SASL anyone?

Need to build from source for common options

buildlinking

How to create a buildlink3 file?

Need short explanation in the guide

developing on slow machines

Getting stuck in clean/extract/patch/configure/compile cycle

Really tedious

Troubleshooting compiler/buildlink issues

How to invoke compiler for a single source file?

Wrapper transformations

Env vars?

Caching DEPENDS

Apparently caches DEPENDS info in WRKDIR

Editing Makefile requires clean/extract/compile cycle

Did I mention slow machines yet?

Local customizations

How to maintain them?

patches/patch-local-??

How to do version control?

Local packages

Can maintain them in pkgsrc/local

But need to keep in sync with rest of tree

Renaming of packages

RPM separates build and package creation infrastructure from package recipes

Fin

Encourage those who want to contribute

A big thanks for providing pkgsrc

Thanks for listening