Alluxio
From Wikipedia, the free encyclopedia
Alluxio is an open-source virtual distributed file system (VDFS). Initially as research project "Tachyon", Alluxio was created at the University of California, Berkeley's AMPLab as Haoyuan Li's Ph.D. Thesis,[2] advised by Professor Scott Shenker & Professor Ion Stoica. Alluxio is situated between computation and storage in the big data analytics stack. It provides a data abstraction layer for computation frameworks, enabling applications to connect to numerous storage systems through a common interface. The software is published under the Apache License.
| Original author(s) | Haoyuan Li |
|---|---|
| Developer(s) | UC Berkeley AMPLab |
| Initial release | April 8, 2013 |
| Stable release | v2.9.4
/ June 11, 2024[1] |
| Repository | https://github.com/Alluxio/alluxio |
| Written in | Java |
| Operating system | macOS, Linux |
| Available in | Java |
| License | Apache License 2.0 |
| Website | www |
Data Driven Applications, such as Data Analytics, Machine Learning, and AI, use APIs (such as Hadoop HDFS API, S3 API, FUSE API) provided by Alluxio to interact with data from various storage systems at a fast speed. Popular frameworks running on top of Alluxio include Apache Spark,[3] Presto, TensorFlow, Trino,[4] Apache Hive, and PyTorch, etc.[citation needed]
Alluxio can be deployed on-premise, in the cloud (e.g. Microsoft Azure, AWS, Google Compute Engine), or a hybrid cloud environment. It can run on bare-metal or in containerized environments such as Kubernetes, Docker, Apache Mesos.
Alluxio also plays a key role in accelerating AI-related workloads, particularly in the areas of model training and model distribution. In machine learning and AI workflows, model training often requires access to large datasets stored across multiple platforms, including on-premises and cloud storage. Alluxio addresses this challenge by providing a unified data layer that caches frequently accessed data in memory, reducing data retrieval latency and eliminating I/O bottlenecks. This leads to faster and more efficient model training. Additionally, Alluxio facilitates model distribution by enabling seamless data access across heterogeneous storage systems, making it easier to share models and datasets between different computational frameworks and environments. By integrating with popular AI frameworks such as TensorFlow and PyTorch, Alluxio ensures fast data access, regardless of its physical location, which is essential for the efficient training and distribution of AI models at scale.
History
Summarize
Perspective
Alluxio was initially started by Haoyuan Li at UC Berkeley's AMPLab in 2013, and open sourced in 2014. Alluxio had in excess of 1000 contributors in 2018,[5] making it one of the most active projects in the data eco-system.
| Version | Original release date | Latest version | Release date |
|---|---|---|---|
| 0.2 | 2013-04-08 | 0.2.1 | 2013-04-25 |
| 0.3 | 2013-10-21 | 0.3.0 | 2013-10-21 |
| 0.4 | 2014-02-02 | 0.4.1 | 2014-02-25 |
| 0.5 | 2014-07-20 | 0.5.0 | 2014-07-20 |
| 0.6 | 2015-03-01 | 0.6.4 | 2015-04-23 |
| 0.7 | 2015-07-17 | 0.7.1 | 2015-08-10 |
| 0.8 | 2015-10-21 | 0.8.2 | 2015-11-10 |
| 1.0 | 2016-02-23 | 1.0.1 | 2016-03-27 |
| 1.1 | 2016-06-06 | 1.1.1 | 2016-07-04 |
| 1.2 | 2016-07-17 | 1.2.0 | 2016-07-17 |
| 1.3 | 2016-10-05 | 1.3.0 | 2016-10-05 |
| 1.4 | 2017-01-12 | 1.4.0 | 2017-01-12 |
| 1.5 | 2017-06-11 | 1.5.0 | 2017-06-11 |
| 1.6 | 2017-09-24 | 1.6.1 | 2017-11-02 |
| 1.7 | 2018-01-14 | 1.7.1 | 2018-03-26 |
| 1.8 | 2018-07-07 | 1.8.2 | 2019-08-05 |
| 2.0 | 2019-06-27 | 2.0.1 | 2019-09-03 |
| 2.1 | 2019-11-06 | 2.1.2 | 2020-02-04 |
| 2.2 | 2020-03-11 | 2.2.2 | 2020-06-24 |
| 2.3 | 2020-06-30 | 2.3.0 | 2020-06-30 |
| 2.4 | 2020-10-19 | 2.4.1 | 2020-11-20 |
| 2.5 | 2021-03-10 | 2.5.0 | 2021-03-10 |
| 2.6 | 2021-06-23 | 2.6.2 | 2021-09-17 |
| 2.7 | 2021-11-16 | 2.7.4 | 2022-04-19 |
| 2.8 | 2022-05-04 | 2.8.1 | 2022-08-17 |
| 2.9 | 2022-11-16 | 2.9.3 | 2023-03-27 |
Legend: Old version, not maintained Old version, still maintained Latest version Latest preview version | |||
Enterprises that use Alluxio
The following is a list of notable enterprises that have used or are using Alluxio:
See also
References
External links
Wikiwand - on
Seamless Wikipedia browsing. On steroids.