OrangeFS has the ability to replace HDFS as the file system for Hadoop MapReduce.
While running in this configuration the Hadoop MapReduce code does not need any modifications to run.
As explained in this technical article originally published on HPCwire.com,
recent tests performed at Clemson University achieved a 25 percent improvement in Apache Hadoop Terasort run times by replacing the Hadoop Distributed File System (HDFS) with an OrangeFS configuration using dedicated servers.
The availability of OrangeFS on AWS Marketplace offers customers the simplicity of Running Big Data, HPC
and other high-performance and data-intensive applications on AWS. With the ease of 1-Click Launch for a Single Instance (1.28TB) or advanced
options to leverage the AWS Marketplace OrangeFS AMIs in 4-Instance (5TB), 8-Instance (10TB) or 16-Instance (20TB) configurations, customers can
build High Performing Cloud File Systems for demanding I/O applications in minutes.Get these new product offerings at AWS MarketPlace.
OrangeFS 2.9 Beta
Adds New Security Modes,
Distributed Directory Entries
A recent beta release introduces many of the advanced features that continue to be fine-tuned for OrangeFS version 2.9,
including two new modes of security and distributed directory metadata.
In key-based security mode, OrangeFS uses public and private keys to authenticate client systems. Each server and client
has a key pair (a public and private key, cryptographically related through openSSL). In certificate-based security mode, each user has a certificate that is uniquely associated with a CA (certificate authority). The certificates are mapped to Linux UIDs/GIDs, using an LDAP (Lightweight Directory Access Protocol) directory.
The beta also provides distributed directory entries across multiple metadata servers, which will further minimize bottlenecks for
accessing large numbers of files with many concurrent tasks. For more information on these features, see the ChangeLog file included in the package, which is
available at the OrangeFS.org Download Page.
The recent maintenance update (Version 2.8.7), added to the June 18 release of
Version 2.8.6, yield significant improvements in performance—especially with small files
and unaligned requests, extended Linux Kernel support (including 3.x) and new Direct Interface libraries
for POSIX and low-level system calls. Also get support for
WebDAV and S3 in the initial offering of Web Pack,
a separate download that will include more Web-based tools in future releases. Get these new packages at the
OrangeFS.org Download Page.
New Online Documentation Debut
The new OrangeFS
Installation Instructions is our first offering in an eventual complete
set of online documentation.
OrangeFS: Past, Present, Future
What the PVFS2 foundation provided:
- Distributed metadata; modular design
- MPI and standard Linux kernel support
- Highly scalable storage, excellent for large data-sets and large files
- Performance speeds significantly faster than NFS using MPI-IO interface
What OrangeFS has already added:
- Commercial-grade support; focused development team
- Windows client; separate metadata location; replicate on immutable
- Improved small-file performance
- Enhanced client libraries; client-side caching and WebDAV support
What is currently being added:
- Distributed directories for improved access time
- Capability based access control
More on OrangeFS Roadmap
Why OrangeFS Stands Out
What features make OrangeFS
right for your environment?
- Object-based design
- Separation of data and metadata
- Optimized MPI-IO requests/support
- Multiple network architectures
- Stateless (lockless) servers
- User-level implementation
- Support for standard Linux (no custom kernels needed)
- Reliable, scalable, portable, flexible
- Hardware independent; painless deployment; research platform
- Metadata operations
- Small, unaligned accesses
- Cross-server redundancy
- Secure access control
- Improved documentation
More on OrangeFS Features
What Omnibond Offers
Paid, commercial-grade services
available for OrangeFS
With an OrangeFS HPC subscription, customers can:
- Get fast escalation for bug fixes
- Get access to the support team with available developer escalation for
support issues, including 24x7 support for critical issues
- Get remote installation and design assistance
- Get unlimited named, designated support contacts
- Help fund the future of free and open
object-based parallel file systems on which you can build your future
- Provide direct input on future features and directions
- Drive the direction and derive the benefits of functionality and performance
testing scenarios for OrangeFS
We offer these subscriptions at various levels for:
- Commercial Entities
- Government, Research Labs and Educational (Non-Teaching)
- Classroom, Instructional, Hobby, Testing and QA
Visit The Development Community
Get More Information
Please make at least one entry per section.