Astronomer, the corporate behind Astro, the main knowledge orchestration and observability platform powered by Apache Airflow®, in the present day introduced the findings from its State of Airflow Report 2025. Primarily based on responses from over 5,000 knowledge practitioners – making it the biggest knowledge engineering survey so far – the report reveals the rising criticality of information orchestration in enterprise operations: from superior analytics to revenue-generating, external-facing knowledge merchandise to manufacturing AI and MLOps.
Additionally Learn: AI and Large Information Governance: Challenges and High Advantages
Apache Airflow has firmly established itself because the business commonplace for knowledge orchestration, a place bolstered by its outstanding progress and widespread adoption. The State of Airflow Report 2025 clearly exhibits a surge in Airflow reputation, utilization and group progress by having a look at metrics from November of final yr and the identical time interval within the yr 2020, when the final main launch–Airflow 2.0–got here out. As of November 2024, Airflow noticed:
- Great improve in reputation: Airflow had greater than 31M+ downloads in November 2024, in comparison with 888K+ in November 2020. The truth is, there have been extra Airflow downloads in 2024 than all earlier years mixed.
- A rising contributor group: Airflow has 3,000+ Airflow contributors, in comparison with 1,300+ in November 2020. Airflow has the most contributors amongst all Apache Software program Basis open supply tasks, together with others like Apache Kafka® and Apache Spark®.
- Growing pull request quantity: 29K+ pull requests in November 2024, in comparison with 10K+ in November 2020
- Exponential enterprise adoption: 77K+ organizations have been utilizing Airflow as of November 2024, in comparison with 25K+ in 2020.
This outstanding progress underscores each Airflow’s growing reputation amongst knowledge professionals in addition to its continuous enlargement of use, as organizations in each business flip to Airflow to assist extra enterprise features.
“The State of Airflow Report 2025 confirms what we’re seeing throughout industries – leveraging knowledge successfully is turning into a aggressive necessity,” mentioned Andy Byron, CEO, Astronomer. “Information orchestration has moved from a enterprise have to a strategic crucial, serving because the spine of contemporary enterprise infrastructure. With over 90% of information professionals citing Airflow as essential to their enterprise, it’s clear that orchestration is foundational for operationalizing knowledge–fueling the whole lot from superior analytics to AI-driven innovation–and that Airflow is the usual for knowledge orchestration.”
Notable themes that emerged from the report embody:
- Information platform variety is the brand new multi-cloud – Whereas market analysts could also be searching for a transparent winner within the cloud knowledge platforms battle, enterprises have clearly chosen a multi-solution technique–similar to they earlier decided that multi-cloud would far outpace standardization on any single cloud supplier. Snowflake (28%), Databricks (29%), and Google Cloud’s BigQuery (27.6%) have practically equal shares of adoption as enterprises transfer in direction of a heterogenous know-how stack with Airflow because the connective tissue.
- Airflow is business-critical – Airflow isn’t simply extraordinarily fashionable with knowledge groups (90%+ advice fee), it’s a core part of mission-critical knowledge operations at main enterprises. Nearly all of customers in organizations with 50,000+ staff report driving business-critical workloads on Airflow.
- Airflow has prolonged past analytics – Whereas Airflow stays a highly-effective framework for ETL and ELT–and it’s nonetheless a wonderful answer for superior analytics and inner dashboards–it’s quickly evolving right into a essential platform for deploying machine studying and generative AI options. Over 85% of surveyed customers anticipate a rise in exterior or revenue-generating options constructed on Airflow within the subsequent yr.
- Orchestration-first organizations ship manufacturing AI and ML – At a time when organizations in each business are challenged with delivering a return on their AI investments, Airflow maturity and knowledge orchestration foundations are key to getting AI and MLOps options in manufacturing. Amongst Airflow customers with a minimum of 5 years of expertise, 30.6% run MLOps workloads on Airflow and 13.3% use Airflow for GenAI. Amongst Astronomer’s Astro clients, 55% use Airflow for ML and AI. Amongst clients which have used Astro for two+ years, the quantity jumps to 69%.
The entire State of Airflow Report 2025 has extra findings across the rising function of information groups, how massive enterprises are leveraging heterogenous tech stacks, and key traits round knowledge orchestration.
Additionally Learn: The Position of AI and Machine Studying in Streaming Expertise
[To share your insights with us as part of editorial or sponsored content, please write to psen@itechseries.com]
