workflow - Falcon & Oozie - How to configure job.properties for oozie in falcon -
i have oozie workflow calls sqoop , hive action. individual workflow works fine when run oozie command line. since sqoop , hive scripts vary, pass values workflow.xml using job.properties file.
sudo oozie job -oozie http://hostname:port/oozie -config job.properties -run
now want configure oozie workflow in falcon. can please me in figuring out can configure or pass job.properties?
below falcon process.xml
<process name="demoprocess" xmlns="uri:falcon:process:0.1"> <tags>pipeline=degingestdatapipeline,owner=hadoop, externalsystem=svservers</tags> <clusters> <cluster name="democluster"> <validity start="2015-01-30t00:00z" end="2016-02-28t00:00z"/> </cluster> </clusters> <parallel>1</parallel> <order>fifo</order> <frequency>hours(1)</frequency> <outputs> <output name="output" feed="demofeed" instance="now(0,0)" /> </outputs> <workflow name="dev-wf" version="0.2.07" engine="oozie" path="/apps/demo/workflow.xml" /> <retry policy="periodic" delay="minutes(15)" attempts="3" /> </process>
i not find on web or falcon documentation regarding this.
i worked on development in falcon did not try falcon vanilla lot, understand tutorial below:
http://hortonworks.com/hadoop-tutorial/defining-processing-data-end-end-data-pipeline-apache-falcon/
i try creating oozie-workflow.xml accepts job.properties dynamically. place properties file in respective hdfs folder workflow.xml picks , can change every process. can use falcon process.xml , call command line using:
falcon entity -type process -submit -file process.xml
also in path=/apps/demo/workflow.xml need not specify workflow.xml explicitly. can give folder name, example:
<process name="rawemailingestprocess" xmlns="uri:falcon:process:0.1"> <tags>pipeline=churnanalysisdatapipeline,owner=etlgroup,externalsystem=uswestemailservers</tags> <clusters> <cluster name="primarycluster"> <validity start="2014-02-28t00:00z" end="2016-03-31t00:00z"/> </cluster> </clusters> <parallel>1</parallel> <order>fifo</order> <frequency>hours(1)</frequency> <outputs> <output name="output" feed="rawemailfeed" instance="now(0,0)" /> </outputs> <workflow name="emailingestworkflow" version="2.0.0" engine="oozie" path="/user/ambari-qa/falcon/demo/apps/ingest/fs" /> <retry policy="periodic" delay="minutes(15)" attempts="3" />
Comments
Post a Comment