This is a quick post to share a few scripts to find what is currently executing in Azure Data Factory. These PowerShell scripts are applicable to ADF version 1 (not version 2 which uses different cmdlets).
Prerequisite: In addition to having installed the Azure Resource Manager modules, you'll have to register the provider for Azure Data Factory:
#One-time registration of the ADF provider #Register-AzureRmResourceProvider -ProviderNamespace Microsoft.DataFactory
Check for Unfinished ADF Jobs
This script finds unfinished jobs (i.e., not in a ready state), after the start date, sorted by status (i.e., window state):
#-----------------------------------------
#Input Area
$subscriptionName = 'YourSubscriptionName'
$resourceGroupName = 'YourResourceGroupName'
$dataFactoryName = 'YourDataFactoryName'
$runStartDateTimeUTC = '2018-01-20T14:00:00Z'
#-----------------------------------------
#Manual login into Azure
Login-AzureRmAccount -SubscriptionName $subscriptionName
#-----------------------------------------
Get-AzureRmDataFactoryActivityWindow `
-DataFactoryName $dataFactoryName `
-ResourceGroupName $resourceGroupName `
-RunStart $runStartDateTimeUTC `
| ? {$PSItem.WindowState -ne 'Ready'} `
| SELECT PipelineName, ActivityName, ActivityType, WindowState, `
PercentComplete, RunStart, RunEnd, Duration `
| Sort-Object WindowState | Format-Table
For brevity, the rest of this post skips repeating the input area and login as shown in the first script above.
Check Status of All ADF Jobs
This script shows status of all jobs after the start date, sorted by pipeline name:
Get-AzureRmDataFactoryActivityWindow ` -DataFactoryName $dataFactoryName ` -ResourceGroupName $resourceGroupName ` -RunStart $runStartDateTimeUTC ` | SELECT PipelineName, ActivityName, ActivityType, ` WindowState, PercentComplete ` | Sort-Object PipelineName | Format-Table
Find Status of Certain ADF Jobs Based on a Naming Convention
If you have implemented a nice consistent naming convention (yes, I'm totally a naming convention junkie), you can search based on the name of an activity or pipeline. This example is looking for the status of the jobs which copy data from Oracle into Azure Data Lake Store:
Get-AzureRmDataFactoryActivityWindow `
-DataFactoryName $dataFactoryName `
-ResourceGroupName $resourceGroupName `
-RunStart $runStartDateTimeUTC `
| ? {$PSItem.ActivityName -like 'ORA*' `
-Or $PSItem.ActivityName -like 'DW*'} `
| SELECT PipelineName, ActivityName, ActivityType, WindowState, `
PercentComplete, RunStart, RunEnd, Duration `
| Sort-Object PipelineName | Format-Table
