mac os x 10.4 “tiger” - usenix · 2019. 2. 25. · kinda-sorta-corefoundation so why not cf?...
TRANSCRIPT
![Page 1: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/1.jpg)
Mac OS X 10.4 “Tiger”
What's New for UNIX Users?
![Page 2: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/2.jpg)
General Highlights
Pervasive SearchingAutomatorVoiceOverParental ControlsSyncServices
![Page 3: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/3.jpg)
New and Upgraded Apps
DashboardiChat AV conferencingSafari RSSQuickTime 7 with H.264Mail.app now uses SQLite
![Page 4: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/4.jpg)
UNIX Highlights
Filesystem fun (indexing and attributes)64 bit libSystemPerformance Performance Performance!Developer Tools updateASL “Apple System Logger”launchd “one daemon to rule them all”
![Page 5: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/5.jpg)
Kernel
fine grain locking SMPKPI work
FS locking is no longer per filesystemImproved Unix Conformance64 bit userland supportPerformance
![Page 6: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/6.jpg)
File Systems
Extended attributes (POSIXy superset)EAs are emulated on non supporting FS typesACLs (favoring NT behavior)Higher level Spotlight search APIsUDF closer to 2.5HFS improved built-in de-fragmentation
![Page 7: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/7.jpg)
File System commands
cp, mv and friends are EA awarersync requires the -E flagcvs is not EA aware
![Page 8: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/8.jpg)
Networking
Wide Area Bonjour using DNS updatesEthernet channel bonding/failoverIPSec support for certificatesFirewall logging, ipfw2 and IPv6 firewallingsite to site VPN and support for Kerberosdummynet
![Page 9: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/9.jpg)
Drivers
Improved Power Management APIs64 bit shimming for ABI reasonsAdding a 802.11 familySupport for persistent disk device nodesGPT support
![Page 10: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/10.jpg)
Userland
Perl 5.8.6Python 2.3.5Ruby 1.8.2Tcl 8.4Wait for the Q&A and I can check other tools.
![Page 11: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/11.jpg)
Apple System Logger “ASL”
A system database of log messagesEasy searchingAdvanced pruningMore flexable logging APIPowerful “mixer” like filter control
![Page 12: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/12.jpg)
Service Managementin Mac OS X
![Page 13: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/13.jpg)
Daemons A long running background processesSuper-daemonsA daemon that proxies some execution for other daemonsAgentsDaemons that operate during and only for a given login sessionCommunication handleA Unix socket or Mach port
Terminology
![Page 14: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/14.jpg)
Assumptions
Prior experience writing a daemon in the Mach or Unix worldFamiliarity with Mach IPC or Unix system calls
![Page 15: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/15.jpg)
Introducing launchd
launchd is all about background processesWork directly on behalf of a userWork indirectly on behalf of a user or usersYou need to get your code running at some point in the system
![Page 16: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/16.jpg)
What’s Wrong With the Status Quo?
Daemons deserve better treatmentIn both Unix and Mac OS, daemons were just processes which disassociated them from user input“Faceless background applications” in Mac OS 9 parlance
![Page 17: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/17.jpg)
The Solution:
A new super-daemon to manage themDesigned to do work for youDesigned to be flexibleDesigned to support messaging and control
![Page 18: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/18.jpg)
Launchd Is Open Source
A critical Darwin componentWe want all Unix daemons to adopt this technologyOpen Sourcing it encourages adoption
![Page 19: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/19.jpg)
The issues that a modern daemon writer facesWhat launchd doesWhat launchd doesn’t doHow to port an existing daemon to launchdHow to write a savvy launchd daemon
What will be covered
![Page 20: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/20.jpg)
Unix Historyinetd
Launches IP based daemons on demand at run-timeAssumes only one FD handle per daemon
initMaintains login daemons on ttys at run-time
/etc/rc*A shell script that runs to configure a Unix system. It often runs other scripts which in turn launch daemonsPoor support for run-time control
cron/at/batch:Time centric
![Page 21: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/21.jpg)
Mach History
mach_initLaunches daemons on demand based on Mach port IPC
![Page 22: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/22.jpg)
Today’s ProblemsMissing functionality:
Unix local domain socket supportFile system based events to trigger a daemon launchinit and inetd don’t support user supplied jobs
Multiple event sources:Networking daemons commonly listen on multiple ports these daysSome daemons use both Mach and Unix based IPCUltimately, time, file system, and IPC events need to be supported in the same “super-daemon”
The ability to restart a daemon is important
![Page 23: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/23.jpg)
The Future
One daemon to rule them allSupport for transferable based event sources
Most file descriptorsMach ports
Support for user supplied jobs
![Page 24: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/24.jpg)
So What Does this Mean?
Hopefully less work for youNo need to daemonize
fork() and have the parent exit()setsid()closing stray file descriptorsreopening stdio as /dev/nulletc.
![Page 25: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/25.jpg)
Launch on Demand
Helping you help us save system resourcesWe support keeping your communication handles alive even when you’re not runningThis saves system resourcesIt also improves the system boot-up speed
![Page 26: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/26.jpg)
Parallel Load at Boot
Making boot-up even more dynamicWe register all daemons configuration handles firstThen we let daemons runThis lets us remove the need for externally specified dependancies
![Page 27: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/27.jpg)
User-Agents
Users have their own special needs too!Standardizes the way we launch user-agentsAllows us to launch them on demand too, thus improving login performance
![Page 28: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/28.jpg)
Case Studies
The real world is what matterscupsd
Uses mach APIs to enable automatic restartingwith launchd, no extra code is needed
mDNSResponderuses both Mach ports and Unix file descriptorslaunchd handles both, nothing else does for launch-on-demand
![Page 29: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/29.jpg)
More Case Studies
User examplesssh-agent
Complicated to automate the management ofMost users only want one per sessionlaunchd makes this trivial with small modifications to ssh-agent
![Page 30: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/30.jpg)
What Doesn’t Launchd Do?
Monitor non kernel fundamental event sources:configd’s database key/valuesconfigd’s eventsNetInfo’s database key/valuesBonjour service advertisementsIO Kit’s namespace (which is built upon mach ports)IO Kit events (e.g. power management)etc.
![Page 31: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/31.jpg)
Wait! Not XYZ?!?
This is subject to changeWe have our own internal needs too
![Page 32: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/32.jpg)
Porting
The high level overviewA simple IPC APIA simple RTTI based object system to support message passing
![Page 33: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/33.jpg)
The IPC API
Kinda-sorta-CoreFoundationSo why not CF?
PortabilityMach port and file descriptor passing is not supported by CoreFoundation at the moment
All we need is RTTI, dictionaries and arrays
![Page 34: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/34.jpg)
C APIs
#include <launch.h>
launch_data_t launch_msg(launch_data_t);
int launch_get_fd(void);
![Page 35: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/35.jpg)
C API Semantics
launch_data_t represents an object graphlaunch_msg() is a synchronous API for the common case
Returns NULL and sets errno on failureIf you request asynchronous messages be sent back:
Call launch_msg(NULL) to get an asynchronous messageKeep calling until you get NULL backIf errno == 0, then no more asyncrhonous messages are available for reading
![Page 36: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/36.jpg)
launch_data_t
RTTI and container classes are fun!DictionariesArraysFile DescriptorsMach PortsIntegersReal numbersBooleansStringsOpaque Data
![Page 37: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/37.jpg)
launch_data_t APIs
“Just enough for IPC, and no more”Get/set operations for basic typesDictionaries:
insertlookupremoveinterate
Arrays:set indexget indexget count
![Page 38: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/38.jpg)
XML plist keysLabelUserNameGroupNameProgramNameRootUmaskWorkingDirectoryServiceDescriptionProgramArgumentsEnvironmentVariablesEventSources
![Page 39: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/39.jpg)
What Are EventSources?
Details, details, details…How to setup a given mach port or socketWho to connect to…Where to listen…etc.
![Page 40: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/40.jpg)
XML plist → launch_data_t
Data distillation
UserNames → UIDs
GroupNames → GIDs
“stuff” → file descriptors and mach ports
![Page 41: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/41.jpg)
Example Messages
Dictionaries where the key is the commandSubmitJobRemoveJobGetJobsCheckInSetUserEnvironmentUnsetUserEnvironmentGetUserEnvironment
![Page 42: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/42.jpg)
Rehashlaunchd is the future
Less work for youpre-daemonized when main() is calledJust check-in and goAutomatic restarting
More flexibility in what event sources you react toMultiple Unix file descriptorsMultiple Mach ports
User agentsA powerful concept for per session background processes
![Page 43: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/43.jpg)
Demo
![Page 44: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/44.jpg)
Apple’s Open Source Web Sitehttp://developer.apple.com/darwin/
For More Information
![Page 45: Mac OS X 10.4 “Tiger” - USENIX · 2019. 2. 25. · Kinda-sorta-CoreFoundation So why not CF? Portability Mach port and file descriptor passing is not supported by CoreFoundation](https://reader036.vdocuments.mx/reader036/viewer/2022071213/6039a4c612a5611dc0296663/html5/thumbnails/45.jpg)
Q&A