AZCopy Sync With Task Scheduler

In this blog we will setup the auto sync between the blob storage container(source) to local machine(destination) by using AZCopy Sync and windows task scheduler.

AZCOPY

AzCopy is a command-line utility that you can use to copy blobs or files to or from a storage account. This article helps you download AzCopy, connect to your storage account, and then transfer files.

Windows Task Scheduler

Task Scheduler is a component of Microsoft Windows, including the latest version Windows 10, that provides the ability to schedule the launch of computer programs or scripts at per-defined times or after specified time intervals: job scheduling.

Create a storage account

To create an Azure storage account with the Azure portal, follow these steps:

1. From the left portal menu, select Storage accounts to display a list of your storage accounts.
2. On the Storage accounts page, select Create.

Options for your new storage account are organized into tabs in the Create a storage account page.

Storage account

Create a container inside the storage account

To create a container in the Azure portal, follow these steps:

1. Navigate to your new storage account in the Azure portal.
2. In the left menu for the storage account, scroll to the Data storage section, then select Blob containers.
3. Select the + Container button.
4. Type a name for your new container.
5. Set the level of public access to the container. The default level is Private (no anonymous access).
6. Select OK to create the container.

Create a containers

Create a SAS Key in container

1. Open the Azure Storage Explorer app on your local machine and navigate to your connected Storage Accounts.
2. Expand the Storage Accounts node and select Blob Containers.
3. Expand the Blob Containers node and right-click on a storage container node or to display the options menu.
4. Select Get Shared Access Signature… from options menu.
5. In the Shared Access Signature window, make the following selections:

• Select your Access policy (the default is none).
• Specify the signed key Start and Expiry date and time. A short lifespan is recommended because, once generated, an SAS can’t be revoked.
• Select the Time zone for the Start and Expiry date and time (default is Local).
• Define your container Permissions by checking and/or clearing the appropriate check box.
• Review and select Create.

Generate a SAS key

Blob SAS URL: https://whatever.blob.core.windows.net/newconain?sp=r&st=2021-08-14T16:45:44Z&se=2021-08-15T00:45:44Z&spr=https&sv=2020-08-04sr=csig=ZUdpQXWi2zAGtI%2Bmw97o7%2BIPbQIHghdfhdgQmWjNcQ%3D

Note: while creating the SAS key give the permission of read and list. Without list permission we can’t scan the directory file while sync.

Install AZCopy in the local system

Download the AZCopy executable to the local system and place it on the directory where you need to execute the commands.

Link: https://docs.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-v10

Steps to create powershell script for AZCopy sync

1. In the first step, replace the path of your local azcopy.exe file path.
2. Replace your storage container SAS URL.
3. At last replace the Destination path.
4. Save this powershell script as nameyouwish.ps1 on the same directory of Azcopy.exe.

launch_process.ps1

<local azcopy.exe file path>\azcopy.exe sync “https://godhhhhhdkdutest.blob.core.windows.net/file?sp=rl&st=2021-08-13T15:11:52Z&se=2021-08-13T23:11:52Z&sv=2020-08-04sr=csig=8dzwNkPZgHIjztGpT%2F2%2B3Jaxhqj6LsqG0m%2BrJeB%2BX%2FE%3D" “<Destination path>”

Steps to create XML file to configure Task Scheduler

In this XML file we will define our configurations of the Windows Task scheduler

1. In Start boundary we can give the time which will need to trigger the task.
2. Days Interval we will define the time interval in which the task need to run.
3. At last in arguments we will mention the file path which need to execute at the specific time.

Task.xml

<?xml version=”1.0" encoding=”UTF-16"?>
<Task version=”1.2" xmlns=”http://schemas.microsoft.com/windows/2004/02/mit/task">
<RegistrationInfo>
<Date>2021–08–14T00:37:46.1335413</Date>
<Author>SahayaGodson</Author>
<URI>\MAXERrecurrentDownload</URI>
</RegistrationInfo>
<Triggers>
<CalendarTrigger>
<StartBoundary>2021–08–14T01:13:00</StartBoundary>
<Enabled>true</Enabled>
<ScheduleByDay>
<DaysInterval>1</DaysInterval>
</ScheduleByDay>
</CalendarTrigger>
</Triggers>
<Principals>
<Principal id=”Author”>
<UserId>S-1–12–1–1926020177–1169294588–4151075990–3368394568</UserId>
<LogonType>InteractiveToken</LogonType>
<RunLevel>HighestAvailable</RunLevel>
</Principal>
</Principals>
<Settings>
<MultipleInstancesPolicy>StopExisting</MultipleInstancesPolicy>
<DisallowStartIfOnBatteries>false</DisallowStartIfOnBatteries>
<StopIfGoingOnBatteries>true</StopIfGoingOnBatteries>
<AllowHardTerminate>true</AllowHardTerminate>
<StartWhenAvailable>true</StartWhenAvailable>
<RunOnlyIfNetworkAvailable>false</RunOnlyIfNetworkAvailable>
<IdleSettings>
<StopOnIdleEnd>true</StopOnIdleEnd>
<RestartOnIdle>false</RestartOnIdle>
</IdleSettings>
<AllowStartOnDemand>true</AllowStartOnDemand>
<Enabled>true</Enabled>
<Hidden>false</Hidden>
<RunOnlyIfIdle>false</RunOnlyIfIdle>
<WakeToRun>false</WakeToRun>
<ExecutionTimeLimit>PT72H</ExecutionTimeLimit>
<Priority>7</Priority>
<RestartOnFailure>
<Interval>PT1M</Interval>
<Count>3</Count>
</RestartOnFailure>
</Settings>
<Actions Context=”Author”>
<Exec>
<Command>powershell</Command>
<Arguments>-ExecutionPolicy ByPass -File replace_with_your_scrpit_path\launch_process.ps1</Arguments>
</Exec>
</Actions>
</Task>

Expected file structure

It is mandatory that the files should be present in the same folder as shown below.

Procedure to Configure the task scheduler

To export, do the following:

• Press Windows key + R to invoke the Run dialog.
• In the Run dialog, type taskschd.msc
• Press CTRL+SHIFT+ENTER key combo to open Task Scheduler in admin mode.
• Open the Task Scheduler in administrative mode.
• Click to expand the Task Scheduler Library on the left pane.
• Right-click on the folder for the task and choose Import Task.
• Now, browse the XML file location and click on Open.

Task Scheduler

After successful execution of trigger, the files will get automatically sync up with the azure storage container.

Powershell execution page