maestro 0.7.0 introduces conditional pipelines

Wait 5 sec.

[This article was first published on data-in-flight, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)Want to share your content on R-bloggers? click here if you have a blog, or here if you don't. The 0.7.0 release of maestro is out, and with it is the ability to conditionally run pipelines. In a nutshell, this is especially useful when you have DAG pipelines and you want to branch or only execute parts of a DAG when the output meets a particular condition.If you haven’t heard of maestro, it’s a script orchestrator for R. You can learn more about it here.Get it from CRAN:install.packages("maestro")Conditional pipelines with @maestroRunIfA pipeline can be made conditional using the maestroRunIf tag. The value of the tag must be an R expression that returns a single TRUE or FALSE. What makes this tag special is that it can be combined with DAG pipelines.Let’s say we have a two-step DAG pipeline for getting weather bulletins and sending out a notification in the event of an active bulletin. The first step of the DAG is to extract the bulletins from an open data source. The second step is to send the notification. However, we only want the second step to execute if there’s an active bulletin.The code below shows this conditional logic. The pipeline send_notification has a maestroRunIf tag that checks the truthiness of grepl("Alert", .input$bulletin_title). Note the use of .input to get the value from the inputting pipeline.# ./pipelines/bulletins.R#' @maestroFrequency 15 minutes#' @maestroStartTime 00:00:00get_bulletins xml2::as_list() bulletin_title