bash – 如何设置Airflow的电子邮件配置以发送有关错误的电子邮
发布时间:2020-12-15 21:35:41 所属栏目:安全 来源:网络整理
导读:我试图通过传入一个不起作用的 Bash行(thisshouldnotrun)来故意使一个Airflow任务失败并输出错误.气流输出如下: [2017-06-15 17:44:17,869] {bash_operator.py:94} INFO - /tmp/airflowtmpLFTMX7/run_bashm2MEsS: line 7: thisshouldnotrun: command not fo
我试图通过传入一个不起作用的
Bash行(thisshouldnotrun)来故意使一个Airflow任务失败并输出错误.气流输出如下:
[2017-06-15 17:44:17,869] {bash_operator.py:94} INFO - /tmp/airflowtmpLFTMX7/run_bashm2MEsS: line 7: thisshouldnotrun: command not found [2017-06-15 17:44:17,869] {bash_operator.py:97} INFO - Command exited with return code 127 [2017-06-15 17:44:17,869] {models.py:1417} ERROR - Bash command failed Traceback (most recent call last): File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/models.py",line 1374,in run result = task_copy.execute(context=context) File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/operators/bash_operator.py",line 100,in execute raise AirflowException("Bash command failed") AirflowException: Bash command failed [2017-06-15 17:44:17,871] {models.py:1433} INFO - Marking task as UP_FOR_RETRY [2017-06-15 17:44:17,878] {models.py:1462} ERROR - Bash command failed Traceback (most recent call last): File "/home/ubuntu/.local/bin/airflow",line 28,in <module> args.func(args) File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/bin/cli.py",line 585,in test ti.run(ignore_task_deps=True,ignore_ti_state=True,test_mode=True) File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/utils/db.py",line 53,in wrapper result = func(*args,**kwargs) File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/models.py",in execute raise AirflowException("Bash command failed") airflow.exceptions.AirflowException: Bash command failed Airflow会发送这类错误的电子邮件吗?如果没有,发送这些错误的电子邮件的最佳方式是什么? 我甚至不确定airflow.cfg是否设置正确…由于最终目标是测试电子邮件警报通知,我想确保正确设置airflow.cfg.这是设置: [email] email_backend = airflow.utils.email.send_email_smtp [smtp] # If you want airflow to send emails on retries,failure,and you want to use # the airflow.utils.email.send_email_smtp function,you have to configure an # smtp server here smtp_host = emailsmtpserver.region.amazonaws.com smtp_starttls = True smtp_ssl = False # Uncomment and set the user/pass settings if you want to use SMTP AUTH # smtp_user = airflow_data_user # smtp_password = password smtp_port = 587 smtp_mail_from = airflow_data_user@domain.com 什么是smtp_starttls?我在文档或在线中找不到任何相关信息.如果我们有查看电子邮件所需的双因素身份验证,那么这对Airflow来说是个问题吗? 这是我的Bash命令: task1_bash_command = """ export PATH=/home/ubuntu/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin export rundate=`TZ='America/Los_Angeles' date +%F -d "yesterday"` export AWS_CONFIG_FILE="/home/ubuntu/.aws/config" /home/ubuntu/bin/snowsql -f //home/ubuntu/sql/script.sql 1> /home/ubuntu/logs/"$rundate"_dev.log 2> /home/ubuntu/logs/"$rundate"_error_dev.log if [ -e /home/ubuntu/logs/"$rundate"_error_dev.log ] then exit 64 fi 我的任务是: task1 = BashOperator( task_id = 'run_bash',bash_command = task1_bash_command,dag = dag,retries = 2,email_on_failure = True,email = 'username@domain.com') 解决方法
smtp_starttls基本上意味着使用
TLS
如果要使用SSL,请将此设置为False并将smtp_ssl设置为True.您可能需要smtp_user和smtp_password. Airflow不会处理2步验证.但是,您使用的是AWS,您可能不需要它,因为您的SMTP(SES)凭据与您的AWS凭据不同. 见here. 编辑: 在这里看到例如: def throw_error(**context): raise ValueError('Intentionally throwing an error to send an email.') t1 = PythonOperator(task_id='throw_error_and_email',python_callable=throw_error,provide_context=True,email_on_failure=True,email='your.email@whatever.com',dag=dag) (编辑:李大同) 【声明】本站内容均来自网络,其相关言论仅代表作者个人观点,不代表本站立场。若无意侵犯到您的权利,请及时与联系站长删除相关内容! |