Azure DevOps Build Pipelines: Run Windows UI Automation & CI

In order to run test automation through Azure DevOps, we’ll need to install a special piece of software on our test run machine.

This software is called a build agent. In this video, we’ll download and install a build agent on a VM.

After downloading a build agent, we also need to configure it to work with Azure DevOps.

When you install a build agent on your own machine, it is called a “self-hosted agent.”

The alternative is called a “Microsoft-hosted agent”.

If you got TFS 2015 installed locally, you will need a different agent. But that’s beyond the scope of this course.

We are working with Azure DevOps.

Build agents are an organization-wide construct.

They can be used for various projects inside their parent organization.

Let me open the Azure DevOps portal on my test VM and sign in here.

I will select the Org for which I want to download the agent.

Build agents are configured specifically to work with an organization.

On the organization home page, there’s “Organization Settings”.

I’m going to click this icon.

Next, I’ll click the “Agent Pools” link under the “Pipelines” section”.

https://vstsagentpackage.azureedge.net/agent/2.169.1/vsts-agent-win-x64-2.169.1.zip

Inside here, I’ll select a pool; I’ll click the “Default Pool” to pick it.

Now I’ll click the “New Agent” button.

The agent version is different for 32-bit and 64-bit machines.

Since I have a 64-bit machine, I’ll select x64.

I have opened the Azure DevOps portal on my test machine, so I’m going to download the agent here.

I will click the “Save” button.

You may click the “Copy URL to Clipboard” button.

This URL will let you download a build agent on a test machine without login into your Azure account.

While the download is in progress, you can see detailed instructions about configuring an agent by clicking this link here.

Since we created this organization, we have permission to install an agent.

This fancy PowerShell shown on the Microsoft Azure DevOps website is just unzipping the Azure agent file.

Let’s first download the agent.

The download can take a few minutes, depending on your internet speed.

Once the download is complete, open the downloads folder.

I’ll unzip the package and give it a shorter name.

I’ll call it AzDAgent

Longer names can sometimes contribute to problems in building sources.

I’ll go one level up and

I’ll copy the newly created folder to the root of the C drive.

You may use a different drive.

That’s it for this video; in the next one, we’ll work to configure the agent.

Don’t log out from Azure DevOps just yet.

In this video we’re going to configure the build agent which we downloaded in the last video.

Before the next step, we need to open PowerShell with the option “Run as Administrator”

I missed this small detail and I had to spend half an hour trying to figure out what had gone wrong

So, make sure you start PowerShell as an administrator.

I will click the Windows button and type PowerShell

I will right click PowerShall and select “Run as Administrator”

<Open file explorer>

If you remember from the previous video, we copied the extracted agent files to the C drive in a folder named AZDAgent.

I’ll open the folder and copy it’s path

Next I’m going to go to the PowerShell and type the CD command and give a space here.

I’ll right click in the PS window to paste the path which I just copied and I’ll hit enter.

A quick dir command will show you the contents of this folder.

First thing we’ll configure the agent by using the config.cmd file.

It is going to ask us the URL of our Azure DevOps organization and a PAT.

Let’s run the config.cmd file, I’ll type config and press tabs key and then hit enter.

It will ask you for the server URL, it will be the URL of your organization

Let’s go to the browser and get your server URL

You can find it by going to the home of your Azure DevOps portal and clicking your organization.

Let’s hit the enter key when the command prompt asks you if you want to use a PAT as authentication type.

Now let’s go back to the Azure DevOps and generate a PAT quickly.

A PAT is just a string and it can be configured to work for all organizations or specific organizations pertaining to an account or specific organizations.

We can specify what actions which can be performed by a user who is specifying a certain PAT to access Azure DevOps.

The PAT generated here will be used to authenticate a build agent which will be installed on our test machine.

More on this later.

<TODO: Request ZH to create a PowerPoint slide for PAT thing and edit it in>

Let’s go back to the Azure DevOps browser.

I’ve selected the organization we’re going to set up for working with the agent which we downloaded in the last video.

I’m going to click on “User Settings”.

Right now it is the icon on the left of my profile picture, the location may change in the future.

Here I’ll select “Personal Access Tokens”.

Let’s click the “New Token” button.

Let me specify a name here (UdemyCoursePAT)

The organization which we just created will be selected here.

In Expiration, I’ll let it be 30 days.

The “Scope” can be very important in keeping your repository safe from malicious users.

That’s why, it’s important to pick the access rights carefully.

You can select “Full Access” for test purposes.

But, here’s how to make it a little more secure.

Firstly, I’ve opened an article about agent download( opened it in the other tab).

I’ll put the URL in resources section of this video too.

You can see that the website says we need to give our PAT some rights in Agent Pools and Deployment Group.

I’ll click “Read & manage” in Agent Pools

Under “Deployment Groups”, there’s only one option and I will check it.

Now Let’s click “Create” button. <llatest one>

I’ll click the copy button to copy the PAT and save it on a safe place because we’ll need it shortly.

I’ll paste the PAT in a notepad file and I’ll click the “Close” button here.

The next page will tell you that make sure you copy this token and save it to a safe place

because Microsoft won’t store it and you won’t be able to see it ever again.

You’ll also get an email about the new token, but, it ironically, it won’t contain the token

Now go back to the PowerShell

paste your PAT and hit enter.

It will ask you to specify an agent pool, I’ll press enter for default.

Next I’ll specify the agent name which will be shown on the server, this string should be meaningful.

For example the name of application under test hyphen agent.

I’ll make it VM10Pro-Agent

If the command prompt doesn’t respond in a few minutes, hit enter again.

Leave the work folder default and hit enter

Next, it will ask if the agent will run as a server, we’ll hit enter here for No.

Press N when it asks for “Configure autologon and run agent on startup?”

After this the setup will be completed.

This is it for the current video, we’ll continue working on PAT in the next video.

The next video is short and very important.

I highly recommend that you do watch the next video now.

For the purpose of this course, we’re going to use a C#.Net UI Test automation code sample which you can download from the “Resources” section of this video.

Please note that right now I am not working on the testing VM.

This machine is different from the one where we installed Azure build agent.

Theoretically speaking you can use the same PC to run build pipelines and do dwebevelopment.

But I’d still recommend you keep both machines separate.

The project which we are using as an example is a Unit Test project which consumes the WinAppDriver package through Nuget.

We will run some tests on the built in Windows Calculator application.

This project is created using .Net Framework 4.5.

Please download the project and open it in Visual Studio 2019 community edition.

You’ll need to have installed “.Net desktop development tools” and .Net 4.5 development tools as well.

I’ll open Visual Studio.

I will click “Continue without code”.

Now open the Team Explorer. If its not there just click “View” and select “Team Explorer”.

I will click “Manage Connections”.

And I’ll click the “Connect” link down below.

I will need to sign in with the account which I’m using for Azure DevOps.

Once we’re in, you will be able to see that the dev.azure.com is the root of the tree shown here.

It’s got a cute little cloud with it.

Underneath, there are the organizations associated with this account.

We have “WADTFS” which we created earlier, listed underneath

TFSRepoTestAutomationTV.

I will select this node.

The purple icon is for TFS Version Control, which is basically the older kind of source control.

The one above it, the orange one that looks like a traffic sign is for a Git repo.

I’ll come to that in later sections.

So, I’ll click the WADTFS code note and click “Connect” button.

And I will click “Connect”.

I’ll click “Map and Get” button.

Now I’ll click the “Local Path”, it will open the folder associated with this repository.

Now let’s copy the solution files to this folder.

Just a friendly reminder, no offense. Please unzip the source code package before you copy it over.

I’ll go back to the TFS and right click the WADTFS node and select “Add Items to Folder”.

I’ll select the folder which contains our source and click “Next”.

I’ll exclude all items in the Packages folder.

Now click “Finish”.

Let’s open the new folder and the solution inside of it.

It will load in solution explorer.

Let met build it.

I’ll go back to the team explorer and here I’ll click “Pending Changes”.

The check-in comment is a piece of information you want to go with your source code to the Azure DevOps repository for reference purposes.

Now I’ll click the “Check In button”.

I will select the node which we just created, “WinAppDriver testing with Azure DevOps”

I’ll click “OK”

Now I’ll click “Team Explorer” and click “Pending Changes”

I’ll type a comment here and press “Check In” button. \

Click “OK” on the next popup.

A check-in confirmation pop-up will show and I’ll select “Yes” on it.

Now let’s open the browser to see if the files which we checked-in jus now have made it to the Azure DevOps server or not.

I’ll go to my Azure DevOps organization and open the project on which we’re working right now.

I’ll click the “Repos” link here.

You will be able to see that our code has made it to the Azure source code repository.

The commit comment which you specified is also there.

Let’s dig deep and open a code file as well.

So here’s the C# code of our test script.

It is not advised to change your code on the fly through the web browser directly, but sometimes you have to do it in an emergency situation.

So keep in mind, this editor must be used with caution.

time to create an actual build pipeline

I will go to our Azure DevOps Project in browser

I’ll click the Pipelines link, right now it's the blue rocket.

I will click the “Create Pipeline” button.

The first question asked by the Azure DevOps portal is “Where is your code?”

You can see it supports multiple options like GiHub, Azure Repos Git, SubVersion etc.

Since we stored our code in a Team Foundation Version Control on Azure, I’m going to select this option.

I’m going to select workspace mappings.

The local path doesn’t have a $ at the start.

We also need to map the solution file here.

So I’ll click the + button here.

Let me open “Repos” and open “Files”

You can see that under WADTFS I have a CalcRunnerUITest folder, then another CalcRUnnerUITest folder, and under it CalcRunnerUITest.sln.

If I hover the mouse on the solution item, you’ll be able to see it’s path in source control.

This is going to go on the left hand side text box.

Let me set these values as well.

$/WinAppDriver Testing with Azure DevOps/CalcRunnerUITest

/WinAppDriver Testing with Azure DevOps/CalcRunnerUITest

$/WinAppDriver Testing with Azure DevOps/CalcRunnerUITest/CalcRunnerUITest.sln

/WinAppDriver Testing with Azure DevOps/CalcRunnerUITest/CalcRunnerUITest.sln

I’ll click “Continue” button.

From template selection, I’ll select .Net Desktop and click “Apply”

I’ll click “Agent job 1” and set the value of Agent pool to “Default”

Now I’ll save the pipeline configuration.

This is all you need to run your test automation scripts on a build agent, through Azure DevOps Pipelines.

In the next video, I’ll give you an overview of the build pipelines tasks and UI.

Let us open the pipeline which we created in the previous video.

As I told you earlier, an Azure build pipeline is just a stack of tasks which run in a sequential way.

If one task fails, all others below it are skipped and the build is considered failed.

The top most item in the stack reads “Pipeline”, the right side of the screen shows various options related to this pipeline.

First one is name, you can change the name to anything you like.

Having a descriptive pipeline name is helpful for future references and search purposes.

Then there’s the Agent pool, an agent pool is a set of machines which can be used to run your build pipelines.

By default this drop down contains the value “Azure Pipelines”.

Let’s open it to see the contents

It contains only one other value, that is “Default”.

The drop down below it contains the text vs2017-win2016

This is a VM created by Microsoft in the cloud, and you may use it if you want to build your code with Visual Studio 2017 build tools on a Windows 2016 box.

But, that box won’t have Windows Application Driver installed and running on it, and it will be a problem.

Because our tests depend on WinAppDriver

It is highly likely that the Microsoft box won’t have your application under test installed either.

Although, the Windows calculator application will be there.

Just for sake of exploration let’s open the drop down to see what is nside.

You’ll see that various machines are readily available to build your code.

It includes macOS and ubuntu machines.

Anyway, I’m going to close this drop down

I will select the value “Default” in “Agent pool”.

Next we have the build task “get sources”. If I click it you’ll see the same settings related to our code which we performed in the previous video.

We are providing some key information about our source code here.

This is how the pipeline will know from where it is supposed to pick up the source code folder containing your test code.

Remember we stored your test automation code in Azure Repos

Once the code is read from the server side repo, it will be transported to the test run agent.

Before the agent can run your test automation scripts, it will need to store the downloaded folder locally.

That’s what we’re doing here.

The line below specifies the mapping for the solution file because the build agent needs to know what solution to build.

After that, there are two Nuget tasks to install packages related to your solution.

Not much happening here.

After getting the source code and performing Nuget, comes the task “Build solution”.

It is just going to build the solution we just told it to.

We won’t change anything here.

Please note that, if you have any compile time error in your source code, the build task  will fail and next steps will be skipped.

Once the build solution is done, we will have our test assemblies ready to run our test automation scripts.

The expression specified in the “Test files” field will be used to run tests from all dll files which contain the word test in their name.

The sample used in this course is “CalcRunnerUITest”.

It produces a dll file named CalcRunnerUITest.dll

This is a special type of dll and the VSTest task knows how to handle it.

Any tests available in the assembly “CalcRunnerUITest.dll” will be run by Azure DevOps build pipeline.

If you don’t want to run all the tests, you may specify a test filter criteria, I will cover it in a later video.

Then there’s a deceptive checkbox, “Test mix contains UI tests”.

Let me click the little i right next to it. It will reveal that this checkbox won’t do any magic, it is here just for informational purposes.

There are two more steps down below but they are not relevant to the core theme of this course.

So, let’s end the video here.

In the next video, we’ll run the build pipeline.

Hey! It’s showtime!

Before running the pipeline, it is mandatory that we login to the PCor virtual machine which we will use for running our WinAppDriver based test automation

.

Once logged in to the test run machine, we’ll also need to run the Azure DevOps agent.

This is the agent which we configured earlier.

I’ll go to the agent folder and run the run.cmd file.

It is entirely possible to launch the run.cmd file on boot if you like to.

You can do the same through PowerShell if you like.

Wait until the agent console shows the message “Listening for Jobs”.

This means the agent has been successfully connected to the Azure DevOps backend.

For running WinAppDriver tests, it is mandatory to have the WinAppDriver.exe running.

We can start the WinAppDriver through build tasks too but for now let’s start it manually.

I’ll show you how to do this in a later video.

I’ve logged on to my Azure DevOps account and I’ve opened the organization which I’m using for these demos.

I’ll hover the mouse on our test project and here I’ll select the build pipelines icon, the blue rocket.

I got only one build pipeline here.

The one which we created recently.

It failed a bunch of times due to internet connectivity issues earlier.

I’ll click the three dots menu here and click “Run a Pipeline”.

In Agent Pool I’ll select Default.

And I’ll click Run.

Soon afterwards, the agent console will show “Running job 1”. If I click in the browser, I’ll be able to see what is currently going on at the build agent. (put VM window below and overlap browser window on right side)

The glorious moment when your automation scripts light up the screen, might take a while to arrive, it depends on the size of your code and your internet speed.

You can see in the browser that the tasks of your build pipeline are getting executed one by one.

In a top down fashion.

When the VSTest task is running, you’ll see the WinAppDriver window will start printing stuff to the console.

And soon afterwards, the calculator application will open and our automation scripts will start performing various operations on it.

Please note that the version of .Net Framework with which you built your solution must be installed on the test agent machine.

If the build fails, it is going to show an error.

Whether a build is passed, or failed. You’re going to get an e-mail notification from Azure.

The time taken to run the job is showing on the top

You will see on the powershell console that the agent is now printing “Job Agent job 1 completed with result: Succeeded”.

Here you see, I also got the mail about it.

Let’s end this video here and explore the test results etc. in the next video.

I’ll pick where we left in the last video.

I’m on the job run page, after the job has finished running.

On the root node titled “Agent Job 1”we’ll be able to see various pieces of information on the top

Right after the job run, we can see the test results by clicking the tests icon here.

It says “100% tests passed”. I’ll open the link in a new tab.

On the test results page you will be able to see how many tests passed and how many failed.

The time it took to run the tests is also shown here.

Let me clear the filter.

I can click any test in the list and see it’s log.

Any Azure Board work items associated with the test will also be shown here.

It is also possible to upload various files as attachment to the build run as well.

Let me go back to the test run tab.

If any build task fails, you can click it and see the log printed on the right side to figure out what went wrong.

It is possible that someone checked-in a code with compile time errors. In that case, the “Build solution” task will fail.

Let me show you the email notification as well.

Here you see, the latest one has succeeded.

Until about 4 AM yesterday, the same build was failing due to internet connectivity issues.

You can see the notifications below.

Let me open the email. It shows

what build was run

what was it’s pass failed status

who started it

how many tests passed

and what version of code committed to the repository was used.

You can view the results by clicking the “View Results” button.

Here if you click the “job name with a green icon, it will show you the particular job run.

This is it for the current video. I hope you’d found it to be useful.

If you have any questions, please feel free to ask. I’d love to be helpful.

?What happens if we click “Enable system diagnostics”?

Please note that very long path names can be a problem so be careful.

Continuous Integration, or CI is THE buzzword these days.

Enabling CI with your Azure DevOps build pipeline is very simple.

You see, everything that makes it to the source code repository must be tested ASAP automatically to make CI work well.

How do we perform these tests? Well, you guess it…

Your test automation scripts will do the testing.

If we enable continuous integration in our pipeline, the pipeline will be triggered automatically whenever there’s check-in on the repository related to the pipeline.

We need to flick a switch.

To do this, let’s go edit the pipeline.

I’ll go to the “Triggers” section and select our project name under the “Continuous Integration” section.

Let me check the “Enable Continuous Integration” and click save here.

I’m going to save the changes.

Now I’ll spin up the VM which contains the test agent configured to work with this build pipeline.

Once inside, I’ll start the build agent as well.

After this, I’ll return to my test development machine and open the WinAppDriver-based test project we’re using in our course.

I’m going to make a pointless change in our test file here. It will be just a comment.

After this, I’ll save all the files and go to Team Explorer.

I’ll click “Home,” and select “Pending Changes.”

Over here, I’ll put a comment and click the Check In button.

Let me minimize the VS.

Once I do this, all of your code will be pushed to the server, and shortly afterward, you’ll see that the build agent console will say “Running Job.”

If we go to the Pipelines and drill down, we’ll see a blue icon next to the job running.

If I click this icon, I’ll be taken to the Azure DevOps page, which shows us the pipeline running right now.

The icon might change in the future, but I believe you get my point.

So this is how we can enable continuous integration on our code check-ins.

Please note that build timeout can be changed through the options section of the build.

Let me show you.

By default, it is 60 minutes.

If, within 60 minutes, an agent is not made available to run the job, the build will fail.

But since the agent was available, the pipeline got triggered, and everything went fine.

I also got the email about the good news.

If I open and see the “Summary” section, the “Reason” field says “Continuous Integration.”

The commit which was tested is also given down below.

If you click the commit link, you can go to the source code repo and see what has changed.<don’t do on video>

If I click on a build email sent due to a build I triggered, the summary section says “Manual” in the reason field.

This is the end of the current video. See you at the next one.

In this video, I’m going to show you how to start and stop WinAppDriver using PowerShell.

A task is available on Azure marketplace to do the same but it doesn’t work on 32 bit machines.

That’s why I myself wrote a tiny piece of PowerShell script to start and stop WinAppDriver.

For this purpose, I’ve opened the build pipeline which we’ve created so far.

I’ll click the + sign right next to “Agent job 1” and I’ll search for “PowerShell”.

Here, I’ll select the task “PowerShell”. The description says “Run a PowrShell script on Linux, macOS, or Windows”.

I’ll click the “Add button”

I’ll drag the task to the position right before “VsTest” task.

Let’s rename it to “Start WinAppDriver”.

I’ll change the type to “Inline” and type in the script.

The script used herein is attached to the resources section of this video. You may download it from there instead of typing with me .

First, I’ll have an if condition to see if the WinAppDriver folder is present in Program Files (x86), which is the case in 64 bit machines.

We’ll use the Test-Path PowerShell command.

In this case, we’ll use the Start-Process command to run the WinAppDriver

We’ll use the Stop-Process PowerShell command to stop the running instance of WinAppDriver.

That’s it. Now you can close the PowerShell on your test machine.

Save the pipeline and run it.

The WinAppDriver will be started & stopped by your PowerShell script.

In the previous section, we created a build pipeline using a Team Foundation Servier Version Control-based repository.

Team Foundation Server Version Control is also called TFSVC.

TFVC is a source code management system based on Microsoft’s proprietary technologies.

It has lost the source control management wars to Git, which is the open-source king of the ring.

So much so that Microsoft provides you with its own hosted Git solution known as Azure Git Repos.

Not to mention Microsoft’s purchase of the world’s largest Git server, GitHub, in 2018.

In the current section, I will show you how to use Azure Git Repos to store your code and create a build pipeline using it.

Since Visual Studio 2013, support for Git version control has been available in VS right out of the box.

You don’t need to install anything to use Git through Visual Studio.

As you’ve seen earlier, when we create a project in Azure DevOps, we can specify the source control it will use.

Let us create a new project in Azure DevOps under our organization.

We will then create a copy of our source code used in the previous section and upload it to our new project’s Azure Repos Git.

So let’s get started and create a new project in Azure DevOps.

I’ve opened my Azure DevOps organization’s home page.

Let’s click the “New Project” button.

I’ll call the one:  WADGit

I’ll expand the “Advanced” button below. We can see that by default the “Version Control” is set to “Git.”

I will keep it private and click “Create.”

Let’s click the “Repos” link.

You will see that the Git Repos page is different from the one we saw in the previous section in the case of a TFS repository.

In the next video, we’ll add some code to this repository.

I’ve opened the WadGit project in Azure DevOps which we’re using for our Git-based example.

I’ll click the “Pipelines” icon.

Here it is inviting you to create your first pipeline.

At the time of this recording, the web page says, “Automate your build and release processes using our wizard, and go from code to cloud-hosted within minutes,”

I’m going to click “Create Pipeline.”

it is asking, “Where is your code?”

We can see there are multiple options listed below,

I’ll select the “Azure Repos Git” option

The process for selecting a Git repo is a little different, but it is essentially the same.

Next, I’ll select the WADGit repo.

I’m going to select .Net Desktop.

This time, you see the words “Review your pipeline YAML” and user interfaces different from what you saw in the past.

There’ll be a lot of text.

This is the new generation of pipeline configuration. It is easier to maintain source control.

Let me explain the important concepts here to you.

The word “trigger:” specifies “master” which means whenever a change is pushed to the master branch, this pipeline will trigger.

The “pool” lets us specify an agent pool.

The value goes underneath the heading.

By default, it’s got “vmImage: ‘windows-latest’.”

Since we want to run our test automation on a self-hosted agent, we will change this value.

Let me remove the existing value.

Instead, I’m going to put a different name-value pair here, which is

name:’Default’

We have configured an agent and added it to the default agent pool.

Then there are tasks for NuGet that we don’t need to change.

Then there’s a task for building our solution. We don’t need to change it either.

These tasks have the word “Settings” written on top of them.

The related settings will be shown on the right side if we click it.

You can see that there’s a section titled “task:VSTest@2”

Let’s see the settings of the "VSTest" task.

We have things like test filter criteria and test settings over here.

There’s also a checkbox that says, “Test mix contains UI tests” you may check it, but keep in mind this flag will not affect the test run behavior.

Now I’m going to go to the machine where I’ve installed the build agent.

I’ll run the build agent, and I’ll also run the WinAppDriver.

Remember, WinAppDriver needs to stay running for our automation code to run successfully.

Now I’ll go back to the AzureDevOps and click the “Save and Run” button.

Let’s commit the settings.

Now we need to give the pipeline permission to access a resource. Let’s click the view button.

I will click the “Permit” button

This needs to be done only the first time.

The agent console will show the status “Running job.”

The browser window will keep updating the job status.

When the VSTest task runs, you’ll see that the WinAppDriver window will start getting text, and shortly afterward, your application under test will launch.

Automation actions will be performed on it.

Once the task is finished, you’ll see Agent will show the status “Job Completed with the result: Succeeded”

A complete test run report will be compiled and published for you to see online.

You’ll get an email in your account with the Build status and a link to see build results.

Let me click the Pipelines icon and select the “Runs” link.

I’ll click the topmost run.

It will contain a summary view.

Right next to it is a “Tests” tab. I’m going to click it.

You can see how many tests were passed and failed here.

By default, it is filtering on failed and aborted tests. I can click the filter icon and deselect these.

Now you will be able to see all the passed tests and how much time they took.

This is it for the current video and the current.

I hope you have found it to be helpful.

I’ve opened the WadGit project in Azure Devops which we’re using for our Git based example.

I’ll click the “Pipelines” icon.

Here it is inviting you to create your first pipeline.

At the time of this recording, the web page says “Automate your build and release processes using our wizard, and go from code to cloud-hosted within minutes”,

I’m going to click “Create Pipeline”

it is asking, “Where is your code?”

We can see there are multiple options listed below,

I’ll select the “Azure Repos Git” option

The process for selecting a Git repo is a little different, but it is essentially the same.

Next, I’ll select the WADGit repo.

I’m going to select .Net Desktop

This time around you see the words “Review your pipeline YAML” and a user interfaces different from what you saw in the past.

There’ll be a lot of text.

This is the new generation of pipeline configuration, it is easier to maintain in source control.

Let me explain the important concepts here to you.

The word “trigger:” specifies “master” which means whenever a change is pushed to the master branch, this pipeline will trigger.

The “pool” lets us specify an agent pool.

The value goes underneath the heading.

By default, it’s got “vmImage: ‘windows-latest’”.

Since we want to run our test automation on a self-hosted agent, we will change this value.

Let me remove the existing value.

Instead, I’m going to put a different name-value pair here, which is

name:’Default’

We have already configured an agent and added it to the default agent pool.

Then there are tasks for NuGet that we don’t need to change for now.

Then there’s a task for building our solution, we don’t need to change it either.

These tasks have the word “Settings” written on top of them.

If we click it, the related settings will be shown on the right side.

You can see that there’s a section titled “task:VSTest@2”

Let’s see the settings of the VSTest task.

We have things like test filter criteria and test settings over here.

There’s also a checkbox that says “Test mix contains UI tests”, you may check it but keep in mind this flag is not going to have any effect on the test run behavior.

Now I’m going to go to the machine where I’ve installed the build agent.

I’ll run the build agent, and I’ll also run the WinAppDriver.

Remember, WinAppDriver needs to stay running for our automation code to run successfully.

Now I’ll go back to the AzureDevOps and click “Save and Run” button.

Let’s commit the settings.

Now we need to give the pipeline permission to access a resource. Let’s click the view button.

I will click the “Permit” button

This needs to be done only the first time.

The agent console will show the status “Running job”

The browser window will keep updating job status.

When the VSTest task will run, you’ll see that WinAppDriver window will start getting text, and shortly afterward your application under test will launch.

Automation actions will be performed on it.

Once the task is finished, you’ll see Agent will show the status “Job Completed with result: Succeeded”

A complete test run report will be compiled and published for you to see online.

You’ll get an email in your account which will have Build status and a link to see build results.

Let me click the Pipelines icon and select the “Runs” link.

I’ll click the topmost run.

It will contain a summary view.

Right next to it is a “Tests” tab. I’m going to click it.

You can see how many tests were passed and failed here.

By default, it filters on failed and aborted tests. I can click the filter icon and deselect these.

Now you will be able to see all the passed tests and how much time they took.

This is it for the current video and the current.

I hope you have found it to be helpful.