OPERATING SYSTEMS COP 4610
Popular in Course
verified elite notetaker
MTED 5318, Teaching and Learning with Techonology in the Mathematics Classroom
verified elite notetaker
verified elite notetaker
verified elite notetaker
Popular in Computer Programming
This 18 page Class Notes was uploaded by Vito Quigley on Thursday September 17, 2015. The Class Notes belongs to COP 4610 at Florida State University taught by An-I Wang in Fall. Since its upload, it has received 50 views. For similar materials see /class/205474/cop-4610-florida-state-university in Computer Programming at Florida State University.
Reviews for OPERATING SYSTEMS
Report this Material
What is Karma?
Karma is the currency of StudySoup.
Date Created: 09/17/15
ZFS THE LAST WORD IN FILE SYSTEMS Jdl39 Bomtlck Dis nguishad Engima39 Sun Mimicsme ZFS The Last Word In File Systems ZFS Overview Provable data integrity Detects and corrects silent data corruption Immense capacity The world39s first 128bit filesystem Simple administration You39re going to put a lot of people out of work Jarod Jenson ZFS beta customer Smokin39 performance ZFS The Last Word In File Systems Trouble With Existing Filesystems No defense against silent data corruption Any defect in disk controller cable driver or firmware can corrupt data silently like running a server without ECC memory Brutal to manage Labels partitions volumes provisioning growshrink Ietclvfstab Lots of limits filesystemlvolume size file size number of files files per directory number of snapshots Not portable between platforms eg x86 tolfrom SPARC Dog slow Lineartime create fat locks fixed block size naive prefetch slow random writes dirty region logging ZFS The Last Word In File Systems ZFS Objective End the Suffering Data management should be a pleasure Simple Powerful Safe Fast ZFS The Last Word In File Systems nnnnnnnnnnnn i You Can39t Get There From Here Free Your Mind Figure out why it39s gotten so complicated Blow away 20 years of obsolete assumptions Design an integrated system from scratch ZFS The Last Word In File Systems ZFS Design Principles Pooled storage Completely eliminates the antique notion of volumes Does for storage what VM did for memory Endtoend data integrity Historically considered too expensive Turns out no it isn39t And the alternative is unacceptable Transactional operation Keeps things always consistent on disk Removes almost all constraints on IIO order Allows us to get huge performance wins ZFS The Last Word in File Systems Why Vlumes Exist In the beginning Customers wanted more space bandwidth reliability eaCh fileSYStem Rewrite filesystems to handle many disks hard m n in l digkagw a s g e Insert a little shim volume to cobble disks together easy An industry grew up around the FSIvolume model Filesystems volume managers sold as separate products Inherent problems in FSIvolume interface can39t be fixed ZFS The Last Word in File Systems FSIVolume Moel vs ZFS Traditional Volumes ZFS Pooled Storage Abstraction virtual disk Abstraction mallocfree Partitionvolume for each FS No partitions to manage Growshrink by hand Growshrink automatically Each FS has limited bandwidth All bandwidth always available Storage is fragmented stranded All storage in the pool is shared ZFS The Last Word in File Systems FSlVolume Moel vs ZFS FSNolume Stack ZFS ll Stack ObjectBased Transactions Block Device Interface Write this block then that block Make these 7 changes to these 3 objects Allornothing Loss of power loss of ondisk consistency Transaction Group Commit Workaround journaling 39 Again allOFnothing which is SIOW amp complex Always consistent on disk l v quot l 1 l i i v i l l l i l t l i l l l l l l 7 i i i V s l l a l 7 l l 7quot quotw 1 x l 4 l No journal not needed B 0 C k D e V i c e In te Via G e L write each block to each Transaction Group Batch IIO 5363174 13 disk immediatel to kee y p Schedule aggregate mirrors In sync and Issue IIO at Will 39 Loss Of IOGWer resync 39 No resync if power lost Synchronous and slow Runs at platter speed ZFS The Last Word In File Systems ZFS Data Integrity Model Everything is copyonwrite Never overwrite live data Ondisk state always valid no windows of vulnerability No need for fsck1M Everything is transactional Related changes succeed or fail as a whole No need for journaling Everything is checksummed No silent data corruption No panics clue to silently corrupted metadata ZFS The Last Word In File Systems CopyOnWrite Transactions 1 Initial block tree 2 COW some blocks 4 Rewrite uberblock atomic ZFS The Last Word In File Systems Mwm Bonus ConstantTime Snapshots At end of TX group don39t free COWed blocks Actually cheaperto take a snapshot than not Snapshot uberblock gt Current uberblock ZFS The Last Word in File Systems EndtoEnd Checksums Disk Block Checksums Checksum stored with data block Any selfconsistent block will pass 39 Can39t even detect stray writes Inherent FSIvolume interface limitation Disk checksum only validates media smut53 X Phantom Writes 3 misdirected readsaand writes X DMA parity errors X Driver bu39gs 3 Accidental onenmrite ZFS Checksum Trees Checksum stored in parent block pointer Fault isolation between data and checksum Entire pool block tree is selfvalidating ZFS The Last Word in File Systems Traditional Mirroring L Application issues a read A Volume manager passes i Filesystem returns bad data Mirror reads the first disk bad block up to filesystem to the application which has a corrupt block If it39s a metadata block the It can39t tell filesystem panics If not 7 at f V quot t T f i39 r g t If 5 39i i it i 1quot K 1 i u i f Jr g 7 i i t 39 are k mi A a t r i i g i May e x xx 4 3 4 3X 3 J J i p I p 3 1 y y 3 I J 1 ti 1 t N a K 139 F t v K it t i rquot t i I i i 39 i f u t K t pquot I i39 i 39 f v i 7 3quotquot 1 1 xquot i V r J f l it i i i i i J 1 x v u V i 1 t3 4 1 i i i r i i 1 ij t39 rquot 1 U 7 quot ti V ZFS The Last Word in File Systems SelfHealing Data in ZFS L Application issues a read A ZFS tries the second disk ZFS returns good data ZFS mirror tries the first disk Checksum indicates that the to the application and Checksum reveals that the block is good repairs the damaged block block is corrupt on disk ZFS The Last Word In File Systems RAI D Z Dynamic stripe width Each logical block is its own stripe 3 sectors logical 3 data blocks 1 parity block etc 39 Integrated stack is key metadata drives reconstruction Currently singleparity doubleparity version in the works All writes are fullstripe writes Eliminates readmodifywrite it39s fast Eliminates the RAID5 write hole you don39t need NVRAM Detects and corrects silent data corruption Checksumdriven combinatorial reconstruction No special hardware ZFS loves cheap disks ZFS The Last Word In File Systems Disk Scrubbing Finds latent errors while they39re still correctable ECC memory scrubbing for disks Verifies the integrity of all data Traverses pool metadata to read every copy of every block Verifies each copy against its 256bit checksum Selfhealing as it goes Provides fast and reliable resilvering Traditional resilver wholedisk copy no validity check ZFS resilver livedata copy everything checksummed All datarepair code uses the same reliable mechanism Mirror resilver RAIDZ resilver attach replace scrub ZFS Scalability Immense capacity 128bit Moore39s Law need 65th bit in 1015 years Zettabyte 70bit a billion TB ZFS capacity 256 quadrillion ZB Exceeds quantum limit of Earthbased storage Seth Lloyd quotUltimate physical limits to computationquot Nature 406 10471054 2000 100 dynamic metadata No limits on files directory entries etc No wacky knobs eg inodeslcg Concurrent everything Parallel readerite parallel constanttime directory operations etc