third-party software management under bsd
TRANSCRIPT
Third-party softwaremanagement under BSD
Andrew Pantyukhin
EuroBSDCon 2006
powered by the FreeBSD Foundation
First things first
●The need for 3d-party software
●Why do we require packages
●The need for package management
●What do we expect to manage
History -Packages in UNIX
●OS is an all-included bundle
●"Users" must figure out how to "use" software
themselves
●SysV/SunOS PKG format
History -Packages in BSD
●JKH committed pkg_install in 1993
●JKH committed ports in 1994
●{Net,Open}BSD snatched both of them
History -Packages in Linux
●Many PM systems appeared in 1993-95, including
RPM and dpkg
●Portage came out in 2002
●A multitude of PM systems were started over since
90s, some of them still maintained, some actively
developed
Current Status
●Dozens of distinctive PM systems
●Focus on binary and/or source packages
●Automation of compilation-related routines
●Automation of installation tasks
FreeBSD ports
●Make + shell
●Multiple scripts in multilevel directories
●Effective automation of common tasks and hacks
NetBSD ports
●Good documentation
●pkginstall framework
●pkg options framework
OpenBSD ports
●Fake build environment
●Immaculate documentation
●Options, flavors, multi-packages
●New pkg_install + a hard-headed maintainer
Looking out of BSD
●RPM
●Dpkg
●Portage
●Other
●Linux Core Consortium - Standard Base
Why bother?
●Isolation has its benefits, but
●No system alone proved to be able to reach even
modest qualitative/quantitative targets
●We have too much in common not to share ideas and
code
What's up?
●Scalability
●Resource management/customization
●Dependencies, capabilities, functionality
●User-side management, interface, Clicks!
●Versioning everywhere
●Saving porter a lifetime (as in more automation)
●Much more...
What packages needand what they provide
●Other packages
●Resources
●Functionality
OS Resourcesfor packages
●Disk space
●TCP/UDP ports at different addresses
●File name spaces
●User/group names/ids
●CPU, RAM, disk throughput, etc.
●Bandwidth
●You name it...
Appliances
●Package + dependencies bundled together
●Jail appliances
●OS appliances
●BSD-based hardware appliances
Package-Provided Resources
●(SQL) Databases
●Virtual Hosts
●Client connections to an (antivirus) engine
●On-screen pixel real-estate
●Technically specific items, like lines in configuration
files
Package-Provided Functionality
●Needed by users directly and/or required by other
packages
●API-oriented, e.g. language modules and extensions,
binary libraries
●Task-oriented, e.g. web servers, mail clients, DBMS
Install-/run-time customization
●Automation of installation tasks based on simple
user-made choices
●Single interface to configure the usage of some
resources, like TCP ports
Multiple package instancing
●Many versions installed at one time for testing and
compatibility
●Several instances of the same version with different
configurations
More flexibility
●Movable packages - at post-install time
●Non-privileged management
●Per-user/per-jail package masking
Versioning
●Easily accessible older versions
●Depending on older versions
●Keeping distfiles in a VCS repo
●Incremental distfile updates
Getting pristine sources
●RCS, VCS repositories
●P2P
●Metalinks
●Rsync
●Guided non-distributable downloads
User Interface
●Centralized management
●Orthogonal switches
●Advanced on-line help/hints system in addition to
manpages and other documentation
●Integrated updating capabilities
●Options handling
●Clicks
User Package Sets
●Storage of package sets, easy access/distribution
●Marking packages as requested or depended on
●Marking some big (sets of) packages as undesirable
to be installed
Technicalities
●distcc
●ccache
●Cross-compilation
Security
●CVE is a dictionary
●We need a database
●Different projects use CVE/database as an overlay
●Or, the database stores info about projects
Exchanging hacks
●Thousands of hacks spread out throughout the PM
systems
●Only a few are documented, most of the other ones
are abandoned and forgotten
●Developers have to reinvent solutions
Collaboration(ism)
●Education
●Spirit
●Communication
Education
●Other PM systems exist, and they do work
●Many of them have good documentation
●Behind projects there are people to talk to and to learn
from
Spirit
●No single PM system dominates the market
●Rich traditions of forks and new-from-scratch solutions
gave us diversity, but prevent us from trying to work
together
●Working with people/approaches we don't necessarily
like
Communication
●Actual developers making contacts
●Looking for solutions (in)
●Sharing our ideas (out)
●Taking part in outside discussions
●Inviting outsiders over to our forums
What's next
●More forays into the wild
●Mentioning the diversity of PM systems in docs
●Learning from others
●Bringing in foreign solutions
●Standardizing on decisions