network and data security with the use of object storage...7 data in flight: smb data security smb...
TRANSCRIPT
Network and data security with the use of object storage
Shawn Fisher Federal Cloud Architect
"The views expressed in this presentation are those of the author(s) and do not necessarily reflect the official policy or position of the Air Force, the Department of Defense, or the U.S. Government."
2
Agenda
1. A bit about me
2. Data wants to be freea) Secure storage protocolsb) “Checking the box” for Data at rest c) Real data at rest
3. Building a machine learning platform to identify network threats
4. Cloudian storage solutions for a secure environment.
© 2018, Cloudian, Inc.
3
Shawn Fisher
Over 20 years of information technology experience in the service of the Federal Government
1990 - IT consultant
1996 - US Army’s 35th Signal Battalion, after the first gulf war provided communications support to 82nd‘s Airborne Division
2001 - Bethesda Naval, birth of giga-bit networking (token ring, 10Mb 100Mb, 1000base-t, Cat-5/e/6)
2002 - US State Department, IT support to AC-NP-PM-VC: T-bureaus (now AVC), during the diplomatic efforts before the Gulf-War
2004 - Pentagon, IT solutions for PA&E (now CAPE), Start of the second Gulf-War (IT transition – 2006: ITIL)
2007 - DoJ’s JCON-NEXT. Massive effort to consolidate FBI, ATF, DEA and DoJ into single IT infrastructure
2008 - PBGC and FDIC, during the US financial crisis
2011 - NetApp, solutions for DoS. DoI, DoL, NARA, Native American tribes, and many other public sector agencies, watched transition
from spinning disk to xAAS.
2016 - Cloudian: software defined, exa-byte scale
© 2018, Cloudian, Inc.
4
Data wants to be free… “stop it”Millions are being spent on Firewalls, intrusion detection, honey pots, STIGs, scanning, manpower etc…But the bad guys are still getting in and the largest enterprises and agencies are getting hacked everyday.Why are our tools failing us? They can only do so much.
The data that is failing us
Enterprise data security
• Data in flight
• NFS• SMB (CIFS)• Iscsi• Fibre Channel• Others: FTP, Telnet, SSH etc…• TLS and Ipsec• Man in the middle
• Data at rest seems to have only one commercially viable option - SED (self encrypting
drives)
5
Server
Message
Block
TLSIPsec
© 2018, Cloudian, Inc.
6
Data in flight: NFS data securityNFS ports
NFS server: 2049 (TCP and UDP)NFS Port mapper: 111 (TCP and UDP)Cluster status: 1110 (TCP)Client status: 1110 (UDP) NFS lock manager: 4045 (TCP and UDP)
NFS v2 and v3 can be secured through Ipsec. It has to be configured properly on both ends.
NFSv4 now includes Kerberos user and group authentication. Information on portmap is still included, since Linux supports NFSv2 and NFSv3, both of which utilize portmap.
NFSv2 and NFSv3 traditionally passed data insecurely. All versions of NFS now have the ability to authenticate (and optionally encrypt) ordinary file system operations using Kerberos. Under NFSv4 all operations can use Kerberos; under v2 or v3, file locking and mounting still do not use it. When using NFSv4.0, delegations may be turned off if the clients are behind NAT or a firewall. Refer to the section on pNFS in the Storage Administration Guide for information on the use of NFSv4.1 to allow delegations to operate through NAT and firewalls.
© 2018, Cloudian, Inc.
7
Data in flight: SMB data securitySMB 3.1.1SMB 3.1.1 was introduced with Windows 10 and Windows Server 2016.[39] This version supports AES 128 GCM encryption in addition to AES 128 CCM encryption added in SMB3, and implements pre-authentication integrity check using SHA-512 hash. SMB 3.1.1 also makes secure negotiation mandatory when connecting to clients using SMB 2.x and higher.
© 2018, Cloudian, Inc.
8
Data in flight: iSCSI data security
DES 56 and 128 bit-1995Mschap v1 v2
So: Ipsec L2TP VPN?
© 2018, Cloudian, Inc.
9
Data in flight: Fibre Channel data securitySeparate and Obfuscate:Lun masking, mapping, and zoning with MAC/WWNN/WWPN address.
Maybe encrypt?
© 2018, Cloudian, Inc.
10
What to do:Isolate the traffic – vLan, separate, filter, and obfuscate
Vlan Hoppinghttps://resources.infosecinstitute.com/vlan-hacking/
Mac spoofing
Lun masking, mapping, and zoning with MAC/WWNN/WWPN address
© 2018, Cloudian, Inc.
11
Better Solutions: Use modern data in flight protocols designed to work “in the open” with encrypted tunnels.
Tunneling: IPSEC and TLS
IPSec operates at Network Layer of the OSI model and is used to secure network communications while TLS
operates at the transport layer and is designed to secure application communications using a secure web
tunnel. The latest version is 1.3 of the Transport Layer Security (TLS) protocol which allows client/server
applications to communicate over the Internet in a way that is designed to prevent eavesdropping, tampering,
and message forgery.
© 2018, Cloudian, Inc.
12
Tunnels are not perfect
DNS
Man in the Middle
Compromised End Points
Certi
fica
teSSLProxy
Certificate Authorityimpersonation
© 2018, Cloudian, Inc.
Certificate Authority injection
13
So… Encrypt the dataBut SEDs are just “checking the box”
The question that needs to be asked about full-disc encryption is 'What is the attack that it's actually preventing?' If the computer is on and functioning, and someone's actually using it, then full-disc encryption really isn't protecting against anything. A hacker can just go through a web vulnerability or whatever, and get access to all the plaintext stuff," Turner told ZDNet.https://www.zdnet.com/article/encrypting-data-at-rest-is-vital-but-its-just-not-happening/
Trucrypt2014
© 2018, Cloudian, Inc.
14
Real data at rest encryptionEncrypt the actual data or the data container – or both!
S3 bucket
Object
Master keyData keys
Server side Encryption key (regular SSE)• Server manages master key and generates
per-object key that’s stored in object metadata.• Can be integrated with an external key
management system.
Per-object key Encrypt S3 bucket
Customer-provided encryption key (SSE-C)• Encryption key is never stored.• Customer must use same key on PUT and
GET.• Can be integrated with an external key
management system.
Object
EncryptedObject & key
S3 bucketMaster key
AWS SDK
Envelope keys
Client-side Encryption• Client provides and manages master keys. • with AWS SDK, dynamic “envelope” key
generated and used to encrypt object and key.• Encrypted envelope key is sent and stored as
object metadata, and checked on retrieval.
Security should be
automatic and policy based
for the type of data
Object
Object
Key Manager
© 2018, Cloudian, Inc.
15
Data escapes? Rekey and crypto-shred
S3 bucket
Object
Master key
Data keys
Object
Key Manager
© 2018, Cloudian, Inc.
16
All of this is just adding to the overall solutionData at rest and data in flight are just part of the total security solution
• Implement robust network security controls to help protect data in transit. Network security solutions like data segregation, firewalls, network access controls and intrusion detection help secure the networks used to transmit data against snooping, attacks, intrusions and data theft.
• Create policies that categorize and classify data. Data should be secured from internal and external threats, no matter where it resides.
• Don’t rely on reactive security to protect your data. Instead, lock down data and use proactive measures that identify at-risk data and implement effective data protection for data in transit and at rest.
• Choose data protection solutions that use policies to enable user and application prompting, blocking, or automatic encryption for sensitive data in transit.
• Your data platform can help. Make your archives work for you…
© 2018, Cloudian, Inc.
17
Building a machine learning platformThreats have moved from outside to the edge and now they are inside
Smart IDS solution and smart interrogation of logs and alerts using Machine Learning and Smart StorageCapabilities
Current data intrusion tools are amazing. They utilize at-line speed packet/log capture and analytics. They problem is that they are very expensive and can only hold small amounts of data.
The larger the data pool the more information the machine learning tools have to identify anomalies and truly determine “Normal” behavior.
Normal is the “Key” to identifying threats, anomaly detection
© 2018, Cloudian, Inc.
What is “Normal”? Anomaly detection with smart cameras - Wizr and Cloudian
© 2018, Cloudian, Inc. 18
19
IDS and log capture systems
• Information is provided from• inventory management systems• performance analysis tools• security logs• server and appliance reporting • packet captures
• Security information and event management (SIEM)• Some SIEM system are rules-based and can employ a statistical correlation to establish relationships
between log entries• Reporting to offer compliance with the ability to customize and create new compliance reports• Forensics capabilities offer the system to capture additional information about security events by recording the
headers and contents of packets.
BUT…
Logs can only be held for so long due to storage constraints. Do you push your archives to the cloud?
© 2018, Cloudian, Inc.
© 2018, Cloudian, Inc. 20
Data lake inputArchival and compliance
IDS sensors
CSA Agents
RouterSwitches
Network Devices
Packet Capture
PacketBeat
• Keep the data onsite• Lan speeds• Inexpensive storage prices• Limitless expandability• Predictable costs• Difficulties getting data to
archives
21
Data lake input and outputTransition from IDS to IPS
IDS sensors
CSA Agents
Network Devices
Packet Capture
MacAfee, Cisco and others offer IPS solutions using information from IDS to shut down unwanted traffic and intrusion attempts.
Network sensors and honey-pots add to total solution.
Once data is in the lake ML and AI can determine abnormal and normal.
OS
© 2018, Cloudian, Inc.
MapReduce
22
Replay eventsForensics to report and remediate with new signatures
IDS sensors
CSA Agents
Network Devices
Packet Capture
OS
ML AI
Data availability to those who need it, accessing the data they need using the tools they are comfortable with: Forensics, Siem, Data Scientists, CISSO, Compliance
© 2018, Cloudian, Inc.
23
Stop your data lake from becoming a swamp
Auto-tagging data and policy based data expiration allow for stale data to be auto-deleted and as much data as possible to be useful…
Garbage in garbage out.
© 2018, Cloudian, Inc.
Centralised Storage for Global Data-Oceans
© 2018, Cloudian, Inc. 24
MicroServices – CLOUD STORAGE INSIDE DATACENTERDATA CENTER
Backup
NFS/SMB/CIFSSync & Share
Development
Block (FC/ISCSI)
INTERNAL S3 Object StoreON PREM
S3
S3
S3
S3
S3
Analytics/ETL
S3
Secure Cloudian
Policy-based migration
CLOUD
S3
• Predict Cloud $$• Multitenancy• QoS• Chargeback, Accounting• Auto-Tiering• 10TBs to 100s company• Data Compression/Encryption• Bucket-level Consistency Policies• Self-Healing• 100% S3 Compliance (use the S3 SDK)• Replicas & Erasure Coding per bucket
HYPERFILE
25 © 2018, Cloudian, Inc.
26
Storage is changing … forever
Flashhost/array
TraditionalSAN/NAS
ObjectStorage
Scale Out /PetabyteScalability / Big DataScale Up / Terabyte
Low PriceBit PriceHigh Price
Lower performancerequirements
LatencySuper-high
performance
LargeTransaction VolumeLimited
RemoteAccessLocal
CapableRobustness / DRLimited
UNSTRUCTURED DATA GROWTH (IoT/Photos/Videos/Backups/Archive/Big Data)By 2025 the global data-sphere will grow to 163ZB. TEN times the 16.1ZB of data generated in 2016
(Seagate/IDC 2017 report).
© 2018, Cloudian, Inc.
27
S3: de-facto Object Storage API StandardThe Amazon S3 API is evolving as a de facto standard for developers writing storage applications for the cloud. It is supported by leading independent software vendors (ISVs), such as backup, archiving and on-premises object storage vendors, which move data to the AWS cloud.Gartner - Magic Quadrant for Public Cloud Storage Services, 2014
Amazon S3 is 2 times as large as all the other object storage services in this Magic Quadrant combined, as measured by amount of data stored.Gartner - Magic Quadrant for Public Cloud Storage Services
the success of AWS is turning S3 into an enterprise storage ‘must have’Information Age, Chloe Green, 8 February, 2016
Gartner - Magic Quadrant for Public Cloud Storage Services, 2017
Eco-system powers use cases, ROI• Apps• Dev Tools• Developers
© 2018, Cloudian, Inc.
28
S3 Functionality
© 2018, Cloudian, Inc.
29
Can I define how to find my data?
BLOB - Image file stored as jpg/tif..
System meta data;Date - Object creation date.Content-Length - Object size in bytes.Last-Modified - Creation date or the last modified date, whichever is the latest.
User defined meta data;x-amz-meta-Patient: Homerx-amz-meta-Age: 50x-amz-meta-”Scan of”: Brainx-amz-meta-Scanner: Xray 1x-amz-meta-operator: Bart…….
© 2018, Cloudian, Inc.
30
What is Cloudian HyperStore? SOFTWARE
•Intelligent, Scale-Out, On-Prem, Hybrid, Enterprise, Cloud Object Storage Software• Multi-Cloud Capable – AWS / AZURE / GOOGLE• Designed to manage massive amounts of data (10’s of Terabytes to 100’s of Petabytes)
• NTT (80+ PBs, 7 countries & Ms users) ● Interoute (50+ PBs, 4 countries) ● Large TV Affiliate (80+ PBs company)
•100% S3 API Native• Natively supports 100s of S3 applications (CommVault/Veritas/Rubrik/etc.) S3 SDK for API access • Economics/Functionality of AWS S3 INSIDE/OUTSIDE your Datacenter
•PREDICT POTENTIAL CLOUD COSTS (AWS/AZURE/GOOGLE)• Data for distributed in all Cloud Platforms (what make $ sense)• It’s not just per GB - Puts/Gets/Reads/etc
• **$0.09 per GB – Transfer OUT S3**
•Enterprise Features• Multi-Tenancy, Multi-Datacenter Replication, QOS, Chargeback/Show back, Erasure Coding, Replicas, Policy Based
Tiering to cloud (S3/Google/Azure), Data Compression, Encryption (at-rest, in-flight), fully distributed architecture providing no single point of failure, Automation/API everything
© 2018, Cloudian, Inc.
Thank you
32
Sources
• Building Scalable Distributed Intrusion Detection Systems Based on the MapReduce FrameworkHoltz, David and de Sousa Junior, 2011https://pdfs.semanticscholar.org/a854/3920e8cded5d403d409ea79922d6a66194b6.pdf
• Uramova, Jana & Segeč, Pavel & Moravcik, Marek & Papan, Jozef & Mokos, Tomas & Brodec, Marek. (2017). Packet capture infrastructure based on Moloch. 1-7. 10.1109/ICETA.2017.8102538.
© 2018, Cloudian, Inc.