Skip to main content
European Commission logo
Research and innovation
Big Data Test Infrastructure (BDTI)

Use case code (DAG)

A DAG (Directed Acyclic Graph) is the core concept of Airflow, collecting Tasks together, organized with dependencies and relationships to say how they should run.

1) Import relevant libraries

from airflow import DAG
from airflow.operators.python import PythonOperator
from airflow.operators.bash import BashOperator
from datetime import datetime,timedelta

 

2) Define DAG default_args

default_args = {
    'owner': 'airflow',
    'depends_on_past': False,
    'email_on_failure': False,
    'email_on_retry': False,
    'retries': 0,
    'retry_delay': timedelta(minutes=1)
}

3) Define the DAG

dag = DAG(
    dag_id='meteo_data_fetching',
    default_args=default_args,
    schedule_interval="@daily",
    start_date=datetime(2022, 1, 1),
    catchup=False
)

4) Define the task to be performed through the BashOperator when the DAG is triggered - to execute the data fetching script

t1 = BashOperator(
    task_id='testairflow',
    bash_command='python /opt/airflow/dags/UAT_Scenario_5.py', 
    dag=dag
)