We worked to optimize performance and data transfer speeds, but also needed to consider hardware reliability and total cost of ownership. While implementing SMR disks, there were a number of hardware trade-offs we had to consider. Host Managed SMR offer the most control over the way data is stored on the drive and is consistent with how we have built things before. Hosts must explicitly open, fill and close sequential zones. The drive does no copying of new data to sequential zones and no caching of data in the conventional area. Host Managed SMR drives require the host to manage the sequential zones on its own. Host Aware drives offer more control than Drive Managed SMR, which is our defining priority. Hosts can open and close zones, monitor write pointers, partially write sequential zones and avoid conventional area caching and performance bottlenecks caused by rewriting zones. Host Aware drives allow a host that understands SMR disks to control the writing of the sequential zones. This involves reading the data from the sequential zone and writing the original data merged with the new data back to the sequential zone. Non-sequential writes are buffered in the small conventional area on each disk, and then later on transcribed to the sequential zones.
While we originally evaluated both Host Aware and Host Managed SMR disks, we finally settled on Host Managed disks for our fleet.ĭrive or Device Managed SMR disks allow the host to treat them like conventional drives. There are three types of SMR HDDs to consider: Drive/Device Managed, Host Aware, and Host Managed SMR disks. We discuss why we’ve chosen to use SMR, hardware tradeoffs and considerations, and some of the challenges we encountered along the way. In this post, we’ll describe our adoption of SMR HDD technology in the Dropbox Storage platform, Magic Pocket. We believe that future storage innovations, including solid-state-drives (SSDs), will benefit from the same architectural approaches we’re developing for SMRs, so that our investment now will pay off in multiples.
Our ambition here is to use our expertise in software for large distributed systems to enable us to take advantage of the ongoing developments in drive technology before our competitors. That’s a lot of data to write, and the experience has to be fast.Īs with our initial Magic Pocket launch, we attacked the problem through inventive software and server architecture to ensure that this solution matched our standards for annual data durability of over 99.9999999999%, and availability of over 99.99%. After all, our new products support active collaboration between small teams all the way up to the largest enterprise customers. Off the shelf, SMR drives have the reputation of being slower to write to than conventional drives. So the challenge has been to benefit from the cost savings of the denser drives without sacrificing performance. The next step in this evolution is our new deployment of specially configured servers filled to capacity with high-density SMR (Shingled Magnetic Recording) drives.ĭropbox is the first major tech company to adopt SMR technology, and we’re currently adding hundreds of petabytes of new capacity with these high-density servers at a significant cost savings over conventional PMR (Perpendicular Magnetic Recording) drives. We continually look for opportunities to increase storage density, reduce latency, improve reliability, and lower costs.
Magic Pocket, the exabyte scale custom infrastructure we built to drive efficiency and performance for all Dropbox products, is an ongoing platform for innovation.