At Cray, we are a big user and investor in Lustre. Because Lustre is such a great fit for HPC, we deploy it with almost all of our systems. We even sell and deliver Lustre storage independent of Cray compute systems. But Lustre is not (yet) the perfect solution for distributed and parallel-I/O, so Cray invests a lot of time and resources into improving, testing, and honing it. We collaborate with the open-source Lustre community on those enhancements and development. In fact, Cray is a leader in the Lustre community through our involvement in OpenSFS.
OpenSFS is a nonprofit technical organization focused on high-end open-source scalable file system technologies, primarily Lustre. OpenSFS aims to foster collaboration among entities developing and deploying Lustre, drive technologies for future requirements into open, scalable file systems, and deliver Lustre file system releases with a roadmap designed to meet those goals.
Cray is an original founder and promoter of OpenSFS and is committed to the organization’s success. As an official OpenSFS “promoter,” Cray pays $500,000 annual dues and has voting rights on the nonprofit’s board of directors. Cray takes our voting privileges seriously and we try to be good stewards of the organization’s funds. If you join, your contributions will go toward things like education and community outreach, new features, testing, semiyearly releases, and documentation.
The OpenSFS community funds have made Lustre exciting. OpenSFS has funded many popular enhancements and designs for improvement needed by Lustre users. For example, Distributed NamespacE (DNE) “remote directory” metadata scaling improvements were recently added to Lustre version 2.4, which many value-added resellers and vendors, including Cray, are now bringing to market. And OpenSFS has commissioned a new project to simplify the client I/O (CLIO) stack, which should make the code more maintainable and thus more easily supported.
Cray does more than fund Lustre though OpenSFS, though. We are a broad participant in all of the OpenSFS activities and working groups. One of our best contributions is testing for quality improvements. Cray would like to continuously demonstrate excellence in Lustre storage for HPC by constantly improving quality, while maintaining performance and scalability. Cray’s test process and procedures are among the best, and before we release products we establish and meet stringent quality metrics. Our rigorous quality standards continuously evolve to further improve dependability and reliability as we continue releases. Cray also considers performance an integral part of our quality metrics. We test numerous use cases and measure performance, tracking any slip from build to build and release to release.
Cray has worked very hard on its test processes, especially scale and load testing, and we’re working with OpenSFS’s to share our know-how and in turn improve the community test plans and release criteria for future OpenSFS-funded releases. We will share our scale and load test mix with the community.
OpenSFS welcomes you to participate in any way you can. Please consider joining one of the working groups. There is now one main working group for Lustre. The Community Development (CDWG) and Technical (TWG) working groups have just announced that they are merging into the Lustre Working Group (LWG) to better incorporate new feature planning into software development and release planning. Please consider joining the LWG. We could use your input on requirements, to guide community development, and to prioritize and vet future project proposals. You can sign up for the new LWG mail list here. Note: if you were subscribed to the TWG or CDWG lists, you will not automatically be signed up for the new LWG list.
And if you want to collaborate with Lustre users and developers in person, please check out the many events planned thorough the year. Coming next month is LAD ’14, which Cray is sponsoring. We’d love to see you there to talk about Lustre and your ideas. If you register before August 27, you’ll get an early-bird registration discount.
Another great way to get involved is to come work with us at Cray. We have a number of open (and soon to be open) positions around Lustre and storage and we’re looking for bright developers and engineers to help innovate Lustre storage solutions from Cray. Check out the Cray Careers webpage for the most up to date listings.
To learn more about OpenSFS and Lustre, go to http://www.opensfs.org and http://lustre.opensfs.org.
Cory Spitz, Manager, Storage and Data Management R&D File Systems
The post Cray’s Commitment to Lustre and OpenSFS appeared first on Cray Blog.