Download - Writing Well-Behaved Unix Utilities
![Page 2: Writing Well-Behaved Unix Utilities](https://reader034.vdocuments.mx/reader034/viewer/2022050815/547e7bfc5806b5ae5e8b4716/html5/thumbnails/2.jpg)
Who are you?• Head of Digital at Big Fish
• We’re a design, branding and marketing consultancy
• Which is to say:
![Page 3: Writing Well-Behaved Unix Utilities](https://reader034.vdocuments.mx/reader034/viewer/2022050815/547e7bfc5806b5ae5e8b4716/html5/thumbnails/3.jpg)
Who are you?• I’ve been programming in Ruby for
several years now
• But I don’t do much web development any more
• My world is reporting, sysadmin, ops, general tools and utilities
• Ruby is my tool of choice
![Page 4: Writing Well-Behaved Unix Utilities](https://reader034.vdocuments.mx/reader034/viewer/2022050815/547e7bfc5806b5ae5e8b4716/html5/thumbnails/4.jpg)
The command-line spectrum
• One-liner
• Script
• Application
![Page 5: Writing Well-Behaved Unix Utilities](https://reader034.vdocuments.mx/reader034/viewer/2022050815/547e7bfc5806b5ae5e8b4716/html5/thumbnails/5.jpg)
The command-line spectrum
• One-liner
• Script
• Application
![Page 6: Writing Well-Behaved Unix Utilities](https://reader034.vdocuments.mx/reader034/viewer/2022050815/547e7bfc5806b5ae5e8b4716/html5/thumbnails/6.jpg)
What makes something an “application”?
• Intended for use by others, or for the foreseeable future
• General-purpose
• Robust
• Reusable
• Built to last
![Page 7: Writing Well-Behaved Unix Utilities](https://reader034.vdocuments.mx/reader034/viewer/2022050815/547e7bfc5806b5ae5e8b4716/html5/thumbnails/7.jpg)
An aside: “Unix”• A grouping of computer operating
systems that behave in a relatively standard way
• In practical terms:
• Linux (Ubuntu, Debian, Fedora, etc.)
• Mac OS X
• BSDs (FreeBSD, OpenBSD, etc.)
![Page 8: Writing Well-Behaved Unix Utilities](https://reader034.vdocuments.mx/reader034/viewer/2022050815/547e7bfc5806b5ae5e8b4716/html5/thumbnails/8.jpg)
The Unix Philosophy
“…no single program or idea makes [Unix] work well. Instead, what makes it effective is the approach to programming… at its heart is the idea that the power of a system comes more from the relationships among programs than from the programs themselves.
![Page 9: Writing Well-Behaved Unix Utilities](https://reader034.vdocuments.mx/reader034/viewer/2022050815/547e7bfc5806b5ae5e8b4716/html5/thumbnails/9.jpg)
The Unix Philosophy
“Many UNIX programs do quite trivial things in isolation, but, combined with other programs, become general and useful tools.”
— Brian Kernighan, 1984
![Page 10: Writing Well-Behaved Unix Utilities](https://reader034.vdocuments.mx/reader034/viewer/2022050815/547e7bfc5806b5ae5e8b4716/html5/thumbnails/10.jpg)
The Unix Philosophy
“This is the Unix philosophy: write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface.”
— Doug McIlroy, c. 1978
![Page 11: Writing Well-Behaved Unix Utilities](https://reader034.vdocuments.mx/reader034/viewer/2022050815/547e7bfc5806b5ae5e8b4716/html5/thumbnails/11.jpg)
The Unix Philosophy• To summarise: good Unix applications
play well with others
• They respect conventions
• They’re reusable and general in nature
• They interact with other processes and accept interaction in standard ways (files, pipes, sockets)
![Page 12: Writing Well-Behaved Unix Utilities](https://reader034.vdocuments.mx/reader034/viewer/2022050815/547e7bfc5806b5ae5e8b4716/html5/thumbnails/12.jpg)
In Ruby
• What does this mean in the concrete?
• How do we write Ruby applications that fit this philosophy?
• Why is Ruby a great choice for writing command-line applications?
![Page 13: Writing Well-Behaved Unix Utilities](https://reader034.vdocuments.mx/reader034/viewer/2022050815/547e7bfc5806b5ae5e8b4716/html5/thumbnails/13.jpg)
A good application…
![Page 14: Writing Well-Behaved Unix Utilities](https://reader034.vdocuments.mx/reader034/viewer/2022050815/547e7bfc5806b5ae5e8b4716/html5/thumbnails/14.jpg)
• works with standard input and output
• works with files
• communicates via its exit status
• respects resource limits
• handles signals
• accepts command-line options
![Page 15: Writing Well-Behaved Unix Utilities](https://reader034.vdocuments.mx/reader034/viewer/2022050815/547e7bfc5806b5ae5e8b4716/html5/thumbnails/15.jpg)
…works with standard input and standard output
• Pipelines are the foundation of Unix
• The goal:
$ grep foo bar | your-app | head
• Assume that input will come from an arbitrary program, and that your output will be fed into one too
![Page 16: Writing Well-Behaved Unix Utilities](https://reader034.vdocuments.mx/reader034/viewer/2022050815/547e7bfc5806b5ae5e8b4716/html5/thumbnails/16.jpg)
In Ruby$stdin.each do |line| puts some_modification(line) end
• Processes input as a stream — handles arbitrarily large input (other streaming methods work too — each_char, read(bytes), etc.)
• Outputs to $stdout, so can be redirected/piped by the user to a file/another process
![Page 17: Writing Well-Behaved Unix Utilities](https://reader034.vdocuments.mx/reader034/viewer/2022050815/547e7bfc5806b5ae5e8b4716/html5/thumbnails/17.jpg)
…works with files too
• The goal:
$ cat foo.txt bar.txt | your-app
$ your-app foo.txt bar.txt
• The ultimate flexibility. As implemented by cat, grep, and most other Unix utilities
![Page 18: Writing Well-Behaved Unix Utilities](https://reader034.vdocuments.mx/reader034/viewer/2022050815/547e7bfc5806b5ae5e8b4716/html5/thumbnails/18.jpg)
In RubyARGF.each do |line| # process a line of input end
• No more effort than reading from standard input
• If ARGV is non-empty, its contents will be read as files
• If it is empty, standard input will be used instead
![Page 19: Writing Well-Behaved Unix Utilities](https://reader034.vdocuments.mx/reader034/viewer/2022050815/547e7bfc5806b5ae5e8b4716/html5/thumbnails/19.jpg)
…sets an exit status• The goal:
$ your-app
$ echo $?
• Was the process successful or not?
• 0 for success, >0 for failure — so youcan communicate up to 255 different failure states
![Page 20: Writing Well-Behaved Unix Utilities](https://reader034.vdocuments.mx/reader034/viewer/2022050815/547e7bfc5806b5ae5e8b4716/html5/thumbnails/20.jpg)
In Rubyif successful? exit else exit 1 end
• Use exit to stop execution of your script and return a successful exit status
• Pass it an argument to alter the exit status
• Document them!
![Page 21: Writing Well-Behaved Unix Utilities](https://reader034.vdocuments.mx/reader034/viewer/2022050815/547e7bfc5806b5ae5e8b4716/html5/thumbnails/21.jpg)
…respects resource limits
• Processes have resource limits
• You should respect them, but you can also alter them if you need to (be reasonable!)
• Avoid the dreaded Errno::EMFILE: Too many open files
![Page 22: Writing Well-Behaved Unix Utilities](https://reader034.vdocuments.mx/reader034/viewer/2022050815/547e7bfc5806b5ae5e8b4716/html5/thumbnails/22.jpg)
In Ruby
hard, soft = Process.getrlimit(:NOFILE)
• Check what the limits of your current process are (both hard and soft) and take appropriate action
• Here we check EMFILE, the limit of the number of file descriptors that we can have open at one time
![Page 23: Writing Well-Behaved Unix Utilities](https://reader034.vdocuments.mx/reader034/viewer/2022050815/547e7bfc5806b5ae5e8b4716/html5/thumbnails/23.jpg)
In Rubybegin File.open(“foo.txt”) rescue Errno::EMFILE # Do something graceful! end
• Actually handle resource-limit–related errors, and do something sensible
• That might be: increase the limit and try the same operation again
![Page 24: Writing Well-Behaved Unix Utilities](https://reader034.vdocuments.mx/reader034/viewer/2022050815/547e7bfc5806b5ae5e8b4716/html5/thumbnails/24.jpg)
In Ruby
Process.setrlimit(:NOFILE, 4096)
• Change the soft limit if you need to; you’re generally allowed to!
• Applies to the current process and its children, including subshells — so you can make sure third-party code behaves too
![Page 25: Writing Well-Behaved Unix Utilities](https://reader034.vdocuments.mx/reader034/viewer/2022050815/547e7bfc5806b5ae5e8b4716/html5/thumbnails/25.jpg)
…handles signals• The goal: to be able to communicate with our
process using signals
• Common signals:
• SIGINT: interrupt (Ctrl+C)
• SIGCHLD: when a child process terminates
• SIGHUP: when the terminal is closed
• SIGUSR1, SIGUSR2: custom signals
![Page 26: Writing Well-Behaved Unix Utilities](https://reader034.vdocuments.mx/reader034/viewer/2022050815/547e7bfc5806b5ae5e8b4716/html5/thumbnails/26.jpg)
In Rubytrap :INT do # close database connection, # other necessary cleanup exit end
• trap registers a handler for a particular signal (in this case SIGINT)
• When that signal is sent, our block is executed
![Page 27: Writing Well-Behaved Unix Utilities](https://reader034.vdocuments.mx/reader034/viewer/2022050815/547e7bfc5806b5ae5e8b4716/html5/thumbnails/27.jpg)
…accepts command-line options• The goal:
$ your-app --verbose -o ‘file.txt’
• Allow our users to change behaviour based on arguments passed
• Makes our options scriptable — not something that has to be chosen from a menu or read from a config file
![Page 28: Writing Well-Behaved Unix Utilities](https://reader034.vdocuments.mx/reader034/viewer/2022050815/547e7bfc5806b5ae5e8b4716/html5/thumbnails/28.jpg)
In Rubyrequire “optparse” !
options = { verbose: false } OptionParser.new do |opts| opts.banner = "Usage: app.rb [options]" !
opts.on("-v", "--verbose") do |v| options[:verbose] = true end end.parse!
![Page 29: Writing Well-Behaved Unix Utilities](https://reader034.vdocuments.mx/reader034/viewer/2022050815/547e7bfc5806b5ae5e8b4716/html5/thumbnails/29.jpg)
In Ruby
• OptionParser, part of the standard library
• Gives you:
• Options, including short aliases
• Automatic -h help output
• Removes options from ARGV, so ARGF works as you expect
• No need to use a Gem!
![Page 30: Writing Well-Behaved Unix Utilities](https://reader034.vdocuments.mx/reader034/viewer/2022050815/547e7bfc5806b5ae5e8b4716/html5/thumbnails/30.jpg)
Wrapping Up