groovy vfs
DESCRIPTION
A Groovy-based DSL for working with files on remote servers and other virtual filesystems. This is the presentation on v0.5 I did at Greach 2014TRANSCRIPT
© Schalk W. Cronjé Greach 2014
Groovy VFSSchalk W. Cronjé
@ysb33r
© Schalk W. Cronjé Greach 2014
Quickstart
def vfs = new VFS()
vfs {
cp “http://from.here/a.txt”, “sftp://to.there/b.txt”
}
© Schalk W. Cronjé Greach 2014
Groovy VFSCurrent version: 0.5
Source: http://github.com/ysb33r/groovy-vfs
Wiki: https://github.com/ysb33r/groovy-vfs/wiki
Packages: https://bintray.com/ysb33r/grysb33r/groovy-vfs
Gradle Packages: https://bintray.com/ysb33r/grysb33r/vfs-gradle-plugin
License: Apache 2.0
Twitter: #groovyvfs
© Schalk W. Cronjé Greach 2014
Bootstrap@Grapes([
@Grab( 'org.ysb33r.groovy:groovy-vfs:0.5' ),
// If you want ftp @Grab( 'commons-net:commons-net:3.+' ),
// If you want http/https @Grab( 'commons-httpclient:commons-httpclient:3.1'),
// If you want sftp @Grab( 'com.jcraft:jsch:0.1.48' )
])
import org.ysb33r.groovy.dsl.vfs.VFS
© Schalk W. Cronjé Greach 2014
Create directory on remote server
vfs {
mkdir “ftp://a.server/pub/ysb33r”
}
© Schalk W. Cronjé Greach 2014
Directory Listing
vfs {
ls (“ftp://www.mirrorservice.org/sites”) {
println it.name
}
}
© Schalk W. Cronjé Greach 2014
Set Schema Options
vfs {
options {
ftp {
passiveMode true
userDirIsRoot false
}
}
}
© Schalk W. Cronjé Greach 2014
Modify Schema Option per Request
vfs {
ls (“ftp://www.mirrorservice.org/sites?vfs.ftp.passiveMode=1”) {
println it.name
}
}220 Service ready for new user.USER guest331 User name okay, need password for guest.PASS guest230 User logged in, proceed.TYPE I200 Command TYPE okay.CWD /250 Directory changed to /SYST215 UNIX Type: Apache FtpServerPASV227 Entering Passive Mode (127,0,0,1,226,247)LIST150 File status okay; about to open data connection.226 Closing data connection.
© Schalk W. Cronjé Greach 2014
Set Schema Options at Initialisation
def vfs = new VFS (
'vfs.ftp.passiveMode' : true,
'vfs.http.maxTotalConnections' : 4
)
© Schalk W. Cronjé Greach 2014
Schema Options
- Most Apache VFS filesystem options supported
- Follows Groovy property-over-setter/getter convention
Documentation:
https://github.com/ysb33r/groovy-vfs/wiki/Protocol-Options
© Schalk W. Cronjé Greach 2014
Copying & Moving
vfs {
cp “http://a.server/myFile.txt”,
“file:///home/scronje/MyFile.downloaded.txt”
mv “ftp://that.server/myFile.txt”,
“sftp://this.server/in-this-folder&vfs.sftp.userDirIsRoot=1”
}
© Schalk W. Cronjé Greach 2014
Copy DSL
vfs {
cp from_url,
to_url,
overwrite : false,
smash : false,
recursive : false,
filter : defaults_to_selecting_everything
}
Documentation: https://github.com/ysb33r/groovy-vfs/wiki/VFS-Copy-Behaviour
© Schalk W. Cronjé Greach 2014
Interactive Copy
vfs {
cp from_url,
to_url,
overwrite : { from,to ->
println “Overwrite ${to}?”
System.in.readLine().startsWith('Y')
}
}
© Schalk W. Cronjé Greach 2014
Move DSL
vfs {
mv from_url,
to_url,
overwrite : false,
smash : false,
intermediates : true,
}
Documentation: https://github.com/ysb33r/groovy-vfs/wiki/VFS-Move-Behaviour
© Schalk W. Cronjé Greach 2014
Download & Unpack
vfs {
cp “zip:ftp://a.server/with-archive.zip”,
“file:///home/scronje/unpacked”
}
Warning:
Large archives will cause slow-down due to performance bug in Apache VFS
© Schalk W. Cronjé Greach 2014
Working with Contentimport java.security.MessageDigest
vfs {
MessageDigest md5 = MessageDigest.getInstance("MD5")
cat (“ftp://a.server/my-file.txt”) { strm →
strm.eachByte(8192) { buffer, nbytes ->
md5.update( buffer, 0, nbytes )
}
}
println md5.digest().encodeHex().toString()
.padLeft( 32, '0' )
} // From a conversation on groovy-user ML
© Schalk W. Cronjé Greach 2014
Local Files- Prefix with 'file://'
- Use java.io.File instance
- Use org.apache.commons.vfs2.FileObject instance
vfs {
cp new File(“a.txt”), “ftp://to.server/pub”
}
© Schalk W. Cronjé Greach 2014
Authenticationvfs {
// clear password cp new File(“a.txt”), “ftp://${username}:${password}@to.server/pub”
// password encrypted with org.apache.commons.vfs2.util.EncryptUtil cp new File('b.txt'), 'ftp://testuser:{D7B82198B272F5C93790FEB38A73C7B8}@to.server/pub'
}
Warning: User with caution (security …)
Enhancement: ISSUE-17 to provide better authentication DSL support
© Schalk W. Cronjé Greach 2014
Future Ideas For Investigation● Java 7 NIO
– java.nio.file.*– Wait for Apache VFS or go native Groovy?
● Direct SMB support?
– Apache VFS Sandbox– jCIFS license is LGPL
© Schalk W. Cronjé Greach 2014
VFS Gradle Plugin(experimental / incubating)
© Schalk W. Cronjé Greach 2014
Bootstrapbuildscript {
repositories {
jcenter()
}
dependencies {
classpath 'org.ysb33r.gradle:vfs-gradle-plugin:0.5'
classpath 'commons-net:commons-net:3.+' // ftp
classpath 'commons-httpclient:commons-httpclient:3.1' // http/https
classpath 'com.jcraft:jsch:0.1.48' // sftp
}
}
apply plugin : 'vfs'
© Schalk W. Cronjé Greach 2014
Available as Project Extension
task ftpCopy << {
vfs {
cp “ftp://a.server/file.txt”, buildDir
}
}
© Schalk W. Cronjé Greach 2014
Future Ideas For Investigation● vfsTree
– Akin to project.tarTree– Is it even possible to create FileTree?
● Vfs Copy / Move Task– How to define @Input etc. correctly
© Schalk W. Cronjé Greach 2014
Other Plugins ?● Grails & Griffon
– No plugin (yet)● Jenkins
– Time to consolidate various protocol plugins
Who wants to write one?
© Schalk W. Cronjé Greach 2014
Groovy VFS#groovyvfs
Schalk W. Cronjé@ysb33r
http://github.com/ysb33r/groovy-vfs