1. plug-ins for storing and querying large data sequencesand gridded data, and for querying external filesBarrodale Computing Services Ltd. (BCS) 2. www.barrodale.com Dealing with Data - ChallengesData comes in many flavors Gridded, time series, spatial series, Measured data, sensor data, model data, Lots of file formats NetCDF, HDF5, FITS, GRIB, Data volumes can be huge tens of terabytes for LOFAR radio-astronomy data files http://pos.sissa.it/archive/conferences/112/062/ISKAF2010_062.pdf Terabytes/day from a single next generation sequencing run http://cloudfront-blog-cache.bioteam.net/wp- content/uploads/2008/04/gen_apr15_datamanagement.pdf 2 3. www.barrodale.com Our SolutionsThe Grid DataBlade Slice, dice, and reproject your gridded dataDBXten Store and query huge data series efficientlyUniversal File Interface (UFI) Query your external files from inside a database3 4. www.barrodale.com Using the Grid DataBlade4 5. www.barrodale.com Dealing with Data Sequences 5 6. www.barrodale.com DBXten Performance ResultsTaskConventional BCS DBXtenImprovementApproach RatioSize of table 15.6 GB1.4 GBX 11Size of index 6, 605 MB6.8 MBX 971Index creation time 5.25 hours 5 seconds X 3,780Insertion time1.67 minutes 1.2 seconds X 83Retrieval time14.7 seconds 3.8 seconds X4 6 7. www.barrodale.com Dealing with Data in Files7 8. www.barrodale.com UFI in Action Weatherdemo 8 9. www.barrodale.com UFI in Action Weatherdemo 9 10. www.barrodale.com For more information Website: http://www.barrodale.com Contact: BCSInfo@barrodale.com or (250) 412-7428 More: http://www.barrodale.com/DBToolsinDepth.pdf10