Apache-AIRFLOW

 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++    +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++  Airflow require a python version above 3.6  check python3 version > python3 --version  python environment > python3 -m view py_env & source the python environment Go to Airflow website  The installation command is in there ..... https://github.com/apache/airflow pip install ' apache-airflow==2.7.0 ' \ --constraint " https://raw.githubusercontent.com/apache/airflow/constraints-2.7.0/constraints-3.8.txt " after successful installation we must , we are going to initialize the database for Airflow.  Initilizing the db for Airflow This command will create a SQLLite database , a log folder and some configuration files. Next start Airflow web server : > airflow webserver -p 8080 You can use any port you wish 8082, 83 as you wish, this will show up a link -- lets open the link on a brow...

   +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ https://www.youtube.com/watch?v=ZET50M20hkU This one : https://www.youtube.com/watch?v=K9AnJ9_ZAXE 00:00 - Airflow Introduction 03:06 - Run Airflow in Python Env 10:44 - Run Airflow in Docker 17:55 - Airflow Basics and Core Concepts 21:55 - Airflow Task Lifecycle 26:19 - Airflow Basic Architecture 28:14 - Airflow DAG with Bash Operator 40:09 - Airflow DAG with Python Operator 45:04 - Data Sharing via Airflow XComs 52:53 - Airflow Task Flow API 57:56 - Airflow Catch-Up and Backfill 01:02:09 - Airflow Scheduler with Cron Expression 01:07:25 - Airflow Connection to Postgres 01:08:58 - Airflow Postgres Operator 01:19:30 - Airflow Docker Install Python Package 2 ways 01:29:34 - Airflow AWS S3 Sensor Operator 01:42:37 - Airflow Hooks S3 PostgreSQL 02:00:43 - Course Bonus +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ What is Apache Airflow ? Apache Airflow is an open-source p...