Building AI-Powered Blazor Applications

In this Lecture we will

• Give a brief overview of some the new features and improvements specific to Blazor in the new .NET 9 Framework update

  • NET 9 Blazor WebAssembly Apps now startup 25% faster

  • Blazor Server Apps are more responsive due to web socket message compression

  • Static web assets (usually stored in the wwwroot folder)  are now optimised so that your static assets can be efficiently downloaded and cached (this occurs when your Application is Published)

  • You can now Detect the component render mode at runtime (See BlazorWebAppNET9InterativeAuto and RegistrationNET9 application)

  • The EditForm component InputNumber now supports a slider ... type=range (See RegistrationNET9 application)

  • The QuickGrid component now supports and OverscanCount property

    • <QuickGrid Items"@people" Virtualize OverscanCount="3" >

    • Using Virtualize allows the Grid to only displays data necessary for what is currently rendered with the OverscanCount property adding rows before and after  to pad out the scroll a little bit

  • Blazor Hybrid applications now have a new Template called .NET MAUI Blazor Hybrid and Web App (See BlazorHybridWebAppNET9  and HoneyDoNET9)

    • Recall that Blazor Hybrid apps take the best of .NET MAUI and Blazor and allow you to create Native experiences with web UI and code reuse across devices (desktop and mobile) and web browsers.

• Create a simple Blazor Web App (Interactive Render mode : Auto Server and WebAssembly)  using .NET 9 (BlazorWebAppNET9InteractiveAuto) ... hands on

  • In the Counter page we implement the RenderInfo commands to detect the components render mode at runtime

    • <h2>Render mode: @RendererInfo.Name</h2>

      
      

      <p role="status">Current count: @currentCount</p>

      
      

      @*During Pre-Rendering there may be a delay...here is how we deal with it*@

      
      

      @if(RendererInfo.IsInteractive)

      {

          <button class="btn btn-primary" @onclick="IncrementCount">Click me</button>

      }

      else

      {

          <p>... Loading please wait</p>

      }

    • Execute application and observe results ... Initially RenderMode is Static ( Static Server Side Rendering ... which means that there is no interactivity at the moment because the page is rendered on the server and returned to the client without interactivity ... non-interactive render mode)  then it switches to either Server/WebAssembly Interactive Render Mode

      • The Auto render mode decides how to render the component at runtime. First, the component is rendered using InteractiveServer. In the background, the .NET runtime and the app bundle are downloaded and cached client-side.

        Next time the component renders, it will use the client executable and render the component using InteractiveWebAssembly.

• RegistrationNET9 (demo only)

  • This is an update (Blazor WebAssembly Standalone App using .NET 9)  to the Application created in Lecture 125 where we create a more sophisticated Registration type form (Registration) which implemented among other things

    • A drop down list for name title (Mr./Mrs/Miss/Ms)

    • A confirm password check

    • An "Accept Terms" checkbox

    • The use of EditContext. Its the way EditForm keeps track of the current state of the form (which fields have been modified) and any validation errors which have been triggered. You can use it instead of the Model attribute assignment.

    • Storing the entered information in JSON format by calling the  JsonSerializer and re-displaying it with a simple alert command called from a Javascript function.

  • Note on the Home.razor page the RendererInfo commands ... You can now through code detect the component render mode at runtime.

    • Welcome to your new .NET 9 app 

      <br />

      Render Mode: @RendererInfo.Name   @(RendererInfo.IsInteractive? "Interactive RenderMode":"No Interactive RenderMode")

    • Name ... returns the location (Server/Client...WebAssembly) where the component is executing

    • IsInteractive indicates if the component supports interactivity at the time of rendering

  • Note in the link "Registration Form App" (RegForm.razor)  the addition of the InputNumber component with type=range .... But unlike the input html tag where we could add oninput for a real time update effect , this does not work now.

    • Basically bind-Value:event="oninput" is not supported in InputNumber or any other Blazor EditForm Component, but you can easily derive a new control from 'InputNumber' which does update on input.

    • You create a new custom Razor Component ... called MyInputNumber.razor , have it inherit Forms.InputNumber and then use the basic html input with additional attributes ... See the RegForm.razor page and MyInputNumber.razor page for all the details

    • MyInputNumber.razor

      • @inherits Microsoft.AspNetCore.Components.Forms.InputNumber<int>

        <input type="range" max="200" min="10" step="10" @attributes="@AdditionalAttributes" class="@CssClass" @bind="@CurrentValueAsString" @bind:event="oninput" />

      • By inheriting from InputNumber<int>, the MyInputNumber component gains all the functionality of the InputNumber<int> base class. This base class is a Blazor component designed to handle numeric input (int in this case)

      • @attributes="@AdditionalAttributes":

        This allows additional attributes to be passed dynamically to the <input> element. For example, if you pass id="mySlider" or data-custom="value" from the parent component, they will be added to the rendered <input> element.

      • class="@CssClass":

        This binds the CssClass property (inherited from InputNumber<int>) to the class attribute of the <input> element. It allows you to dynamically apply CSS classes to style the slider.

      • @bind="@CurrentValueAsString":

        This binds the slider's value to the CurrentValueAsString property (also inherited from InputNumber<int>). When the user moves the slider, the value is updated in the component, and vice versa.

    • RegForm.razor

      • <div class="form-group row">

            <div class="form-group col-5">

                <label class="mt-2">Age ... using InputNumber:@(model.Age == 0 ? "" : model.Age)</label>

                <InputNumber type="range" max="100" min="1" step="1" @bind-Value="model.Age" />

                <ValidationMessage For="@(() => model.Age)" />

            </div>

            <div class="form-group col-5">

                <label class="mt-2">Age ... using MyInputNumber:@(model.Age == 0 ? "" : model.Age)</label>

                <MyInputNumber @bind-Value="model.Age" />

                <ValidationMessage For="@(() => model.Age)" />

            </div>

        </div>

  • RegNET9InputNumberOninputTime

    • Note the addition of the time picker (MyInputTime) in the Registration Form App. The HTML input type=time is not available in Blazor Edit Components

  • RegNET9MarkdownEditor

    • In this update ... Note the addition of Simple Markdown Editor in the Registration Form App. SimpleMDE ... a simple and embeddable Javascript editor... You may need to run it without the Debugger (Ctrl+ F5) to see it in action)

      Here are the Details

      • In the wwwroot folder we have updated the index.html file to point to the js code and stylesheet

        • We are using an External URL: Instead of hosting the JavaScript file locally in your project (e.g., in a js folder), you are pointing to a Content Delivery Network (CDN) URL.

        • We also point to the local js file (located in the js folder) . This file is used to create an instance of the Editor for use in the BlazorSimpleMde.razor component.

      • In the wwwroot folder we have added a javascript file blazor-simplemde.js

      • In the Pages folder we have added a new Razor Component called BlazorSimpleMde.razor

      • In this BlazorSimpleMde.razor file we have added a textarea element and a code block to initialize the markdown editor and subsequently get the value (contents) of the editor

      • ... and finally we add BlazorSimpleMde component (the Markdown Editor) to our form

• Take a look at the new Blazor Hybrid Template

  • .NET 9 now has a new template called .NET MAUI Blazor Hybrid and Web App

  • This new Blazor Hybrid template gives you with a single click an automatically configured project with .NET MAUI, Blazor Web App, and a Razor class library all setup! ... plus a Client Project if you pick WebAssembly Render Mode

  • ... Recall Lecture 166 Introduction to Blazor Hybrid ... Where we implemented the .NET MAUI Blazor Hybrid App which consisted of a Single Project (.MAUI) and we manually added a Razor Class Library Project and Server/WASM project after the fact (HybridTodoLibNET8updated)

  • BlazorHybridWebAppNET9  ... hands on

    • Start up Visual Studio 2022 and choose .NET MAUI Blazor Hybrid and Web App ...NET 9 ...Interactive Render Mode WebAssembly

    • The Solution now contains

      • The .NET MAUI project  ... Blazor Hybrid App  (MauiApp1)

        • Windows Machine (Windows Desktop App)

        • Mobile

      • A Web Project (MauiApp1.Web) ... standard Blazor Web App (Server)

      • A Shared Project (MauiApp1.Shared) ... Razor Class Library

        • Most of the UI lives in the Shared project

        • Note how all the Razor Component pages live here in the Pages subfolder

          • Counter and Weather pages are basically the same as previous versions

          • Home page has some new additions
            

            • @page "/"

              @using ToDoHybridNET9.Shared.Services

              @inject IFormFactor FormFactor

              
              

              <PageTitle>Home</PageTitle>

              
              

              <h1>Hello, world!</h1>

              
              

              Welcome to your new app running on <em>@factor</em> using <em>@platform</em>.

              
              

              @code {

                  private string factor => FormFactor.GetFormFactor();

                  private string platform => FormFactor.GetPlatform();

              }

            • The Interface IFormFactor is located in the Services folder of this Shared project

            • This Interface is registered in the Program.cs file of the Web project and in MauiProgram.cs of the MAUI project
              // Add device-specific services used by the ToDoHybridNET9.Shared project

              builder.Services.AddSingleton<IFormFactor, FormFactor>();

            • The Implementation of FormFactor is located in the Services folder of the MAUI project
              using ToDoHybridNET9.Shared.Services;

              
              

              namespace ToDoHybridNET9.Services

              {

                  public class FormFactor : IFormFactor

                  {

                      public string GetFormFactor()

                      {

                          return DeviceInfo.Idiom.ToString();

                      }

              
              

                      public string GetPlatform()

                      {

                          return DeviceInfo.Platform.ToString() + " - " + DeviceInfo.VersionString;

                      }

                  }

              }

            • .... and the Services folder of the Web Project
              using ToDoHybridNET9.Shared.Services;

              
              

              namespace ToDoHybridNET9.Web.Services

              {

                  public class FormFactor : IFormFactor

                  {

                      public string GetFormFactor()

                      {

                          return "Web";

                      }

              
              

                      public string GetPlatform()

                      {

                          return Environment.OSVersion.ToString();

                      }

                  }

              }

              
              

        • Note how all the Layout pages live here (Shared project)  in the Layouts subfolder

      • ... Client Project (MauiApp1.Web.Client) ... WebAssembly Project .. since we chose WebAssembly Render Mode

    • When you execute the application it will default to the Blazor Hybrid App (Windows Machine / Mobile ).

      • Welcome to your new app running on Desktop using WinUI - 10.0.26100.2314.

      • Welcome to your new app running on Phone using Andriod-15

    • You can change the Startup Project and have it point to the Web App and it will run as a typical Blazor Web App with Rendermode InteractiveWebAssembly

      • Welcome to your new app running on WebAssembly using Other 1.0.0.0.

      • Check out App.razor in MauiApp.Web folder
        <body>

            <Routes @rendermode="InteractiveWebAssembly" />

            <script src="_framework/blazor.web.js"></script>

        </body>

  • HoneyDoNET9 (demo only ... Optional Enrichment )

    • Simple To Do application with different views/UI depending on device or Web

      • Card View (Mobile) or QuickGrid (Desktop or Web)

      • Note the use of the #if preprocessor conditional directive (on Home.razor/All.razor and Edit.razor pages)
        @{ #if ANDROID || IOS }

            <MyCardView HoneyDos="@filteredHoneyDos" />

        @{ #else }

            <MyQuickGrid HoneyDos="@filteredHoneyDos" />

        @{ #endif }

      • #if is a preprocessor directive to control conditional compilation.  Opens a conditional compilation where code is compiled only if the specified symbol is defined .The C# compiler compiles the code between the #if directive and #endif directive

        
        

    • Key Highlights

      • Most of the coding happens in the Shared project

        • In the Models folder ... HoneyDoModel.cs class

        • In the Interface folder

          • ILocalStorage.cs
            public interface ILocalStorage

            {

                public string GetItem(string key);

                public void SetItem(string key, string value);

                public void RemoveItem(string key);

            }

          • IPhotoManager.cs
            public interface IPhotoManager

            {

                public Task<string> TakePhotoAsync();

                public Task<string> PickPhotoAsync();

            }

        • In the Services folder

          • LocalStorage

          • PhotoManager.cs

          • HoneyDoServices
            internal class HoneyDoService

            {

                private static List<HoneyDoModel>? myModel = null;

                private const string ModelName = "HoneyDoModel";

            
            

                public static List<HoneyDoModel> GetModel(ILocalStorage storage)

                {

                    if (myModel == null)

                    {

                        string jsonString = storage.GetItem(ModelName);

                        if (jsonString != null)

                        {

                            myModel = JsonSerializer.Deserialize<List<HoneyDoModel>>(jsonString);

                        }

                    }

                    if (myModel == null)

                    {

                        myModel = new List<HoneyDoModel>();

                        myModel.Add(new HoneyDoModel { Id = 1, Task = "Task 1", Description = "Description 1", IsComplete = false, DueDate = DateTime.Now.AddDays(1), AssignedTo = "Nick", CreatedBy = "Beth", CreatedDate = DateTime.Now, UpdatedDate = DateTime.Now });

                        myModel.Add(new HoneyDoModel { Id = 2, Task = "Task 2", Description = "Description 2", IsComplete = false, DueDate = DateTime.Now.AddDays(2), AssignedTo = "Nick", CreatedBy = "Beth", CreatedDate = DateTime.Now, UpdatedDate = DateTime.Now });

                        myModel.Add(new HoneyDoModel { Id = 3, Task = "Task 3", Description = "Description 3", IsComplete = false, DueDate = DateTime.Now.AddDays(2), AssignedTo = "Nick", CreatedBy = "Beth", CreatedDate = DateTime.Now, UpdatedDate = DateTime.Now });

                    }

                    return myModel;

                }

                public static HoneyDoModel GetModelById(int id, ILocalStorage storage)

                {

                    if (myModel == null)

                    {

                        myModel = GetModel(storage);

                    }

                    var item = myModel?.FirstOrDefault(m => m.Id == id);

                    return item;

                }

                public static void SaveModel(ILocalStorage storage)

                {

                    if (myModel == null)

                    {

                        return;

                    }

                    string jsonString = JsonSerializer.Serialize(myModel);

                    storage.SetItem(ModelName, jsonString);

                }

                public static HoneyDoModel AddItem()

                {

                    var model = new HoneyDoModel();

                    if (myModel != null)

                    {

                        myModel.Add(model);

                        if (model.Id == 0)

                        {

                            model.Id = myModel.Max(m => m.Id) + 1;

                        }

                    }

                    return model;

                }

                public static void AddItem(HoneyDoModel model)

                {

                    if (myModel != null)

                    {

                        myModel.Add(model);

                        if (model.Id == 0)

                        {

                            model.Id = myModel.Max(m => m.Id) + 1;

                        }

                    }

                }

                public static void RemoveItem(HoneyDoModel model)

                {

                    myModel?.Remove(model);

                }

            
            

                public static List<HoneyDoModel> ReloadModel(ILocalStorage storage)

                {

                    myModel = null;

                    return GetModel(storage);

                }

            }

        • Components folder

          • MyCardView.razor
            @using Shared.Models

            @using Shared.Interfaces

            @using Shared.Services

            @inject NavigationManager NavigationManager

            
            

            @if (honeyDos == null || honeyDos.Count() == 0)

            {

                <p>No tasks due. Enjoy the day! ?</p>

            }

            @foreach (var honeyDo in honeyDos)

            {

                <div class="card @GetCardBorderColor(honeyDo)" >

                    <div class="card-body @GetCardBodyColor(honeyDo)">

                        <h4 class="card-title">@honeyDo.Task</h4>

                        <p class="card-text">@honeyDo.Description</p>

                        <p class="card-text">Due: @honeyDo.DueDate.ToShortDateString()</p>

                        @if (honeyDo.IsComplete)

                        {

                            <p class="card-text">Completed</p>

                        }

                        else

                        {

                            <p class="card-text">Not completed</p>

                        }

                        <img class="card-img-bottom img-thumbnail" style="width:100px" src="@honeyDo.Image" alt="Card image">

                        <a href="#" class="stretched-link" @onclick="() => Edit(honeyDo.Id)"></a>

                    </div>

                </div>

            }

            @code {

                [Parameter]

                public IQueryable<Shared.Models.HoneyDoModel>? honeyDos { get; set; }

            
            

                private void Edit(int id)

                {

                    NavigationManager.NavigateTo($"/edit/{id}");

                }

                public string GetCardBorderColor(HoneyDoModel honeyDo)

                {

                    if (honeyDo.IsComplete)

                    {

                        return " border-secondary mb-3";

                    }

                    else if (honeyDo.DueDate.Date < DateTime.Now.Date)

                    {

                        return " border-danger mb-3";

                    }

                    else

                    {

                        return "";

                    }

                }

                public string GetCardBodyColor(HoneyDoModel honeyDo)

                {

                    if (honeyDo.IsComplete)

                    {

                        return " text-secondary";

                    }

                    else if (honeyDo.DueDate.Date < DateTime.Now.Date)

                    {

                        return " text-danger";

                    }

                    // else if (honeyDo.DueDate.Date == DateTime.Now.Date)

                    // {

                    //     return " text-primary";

                    // }

                    else

                    {

                        return "";

                    }

                }

            }

          • MyQuickGrid.razor
            @using Shared.Models

            @using Shared.Interfaces

            @using Shared.Services

            @using Microsoft.AspNetCore.Components.QuickGrid

            @inject ILocalStorage LocalStorage

            @inject NavigationManager NavigationManager

            
            

            @if (honeyDos == null || honeyDos.Count() == 0)

            {

                @*emoji character are added in Windows 11 by

                    pressing Window logo key and . (period)

                    this will display the Emoji keyboard ?

                *@
                    <p>No tasks due. Enjoy the day! ?</p>

            }

            <QuickGrid Items="@honeyDos" Class="table table-sm table-bordered table-striped">

            @{#if ANDROID || IOS || WINDOWS || MACCATALYST }

                <TemplateColumn Title="Image">

                    <img class="img-thumbnail" style="width:100px" src="@context.Image" alt="Card image">

                    <div class="click-area" @onclick="() => Edit(context)"></div>

                </TemplateColumn>

            @{#endif}

                <TemplateColumn Title="Task" Sortable="true">

                    <div class="@GetTaskColor(context)">@context.Task</div>

                    <div class="click-area" @onclick="() => Edit(context)"></div>

                </TemplateColumn>

                <PropertyColumn Property="@(item => item.DueDate)" Format="yyyy-MM-dd" Sortable="true" />

                 <TemplateColumn Title="Complete?">

                    <input type="checkbox" @bind="@context.IsComplete" @onclick="() => Complete(context)" />

                </TemplateColumn>

                <PropertyColumn Property="@(item => item.Description)" Sortable="true"/>

                <PropertyColumn Property="@(item => item.AssignedTo)" Sortable="true"/>

            </QuickGrid>

            @code {

                [Parameter]

                public IQueryable<Shared.Models.HoneyDoModel>? honeyDos { get; set; }

            
            

                private void Edit(HoneyDoModel model)

                {

                    NavigationManager.NavigateTo($"/edit/{model.Id}");

                }

                private void Save()

                {

                    HoneyDoService.SaveModel(LocalStorage);

                }

                private void Complete(HoneyDoModel model)

                {

                    model.IsComplete = !model.IsComplete;

                    Save();

                }

                public string GetTaskColor(HoneyDoModel honeyDo)

                {

                    if (honeyDo.IsComplete)

                    {

                        return " text-secondary";

                    }

                    else if (honeyDo.DueDate.Date < DateTime.Now.Date)

                    {

                        return " text-danger";

                    }

                    else

                    {

                        return "";

                    }

                }

            }

        • Pages folder

          • All.razor

            @page "/all"

            @using HoneyDo.Shared.Services

            @using HoneyDo.Shared.Interfaces

            @using HoneyDo.Shared.Models

            @using HoneyDo.Shared.Components

            @inject ILocalStorage LocalStorage

            @inject NavigationManager NavigationManager

            
            

            <PageTitle>All "Honey Do" Items</PageTitle>

            <h3>All Tasks</h3>

            <p>

                Show completed tasks: <input type="checkbox" @onclick="FilterChanged" />

            </p>

            @{ #if ANDROID || IOS }

                <MyCardView HoneyDos="@filteredHoneyDos" />

            @{ #else }

                <MyQuickGrid HoneyDos="@filteredHoneyDos" />

            @{ #endif }

            
            

            @code {

            
            

                private IQueryable<HoneyDoModel> honeyDos;

                private IQueryable<HoneyDoModel> filteredHoneyDos;

                private bool showCompleted = false;

            
            

                protected override void OnInitialized()

                {

                    honeyDos = HoneyDoService.GetModel(LocalStorage)

                                              .AsQueryable()

                                              .OrderBy(item => item.DueDate);

            
            

                    filteredHoneyDos = honeyDos.Where(item => !item.IsComplete);

                }           

                private void FilterChanged()

                {       

                    showCompleted = !showCompleted;

                    if (showCompleted)

                    {

                        //Show all

                        filteredHoneyDos = honeyDos.Where(item => true);

                    }

                    else

                    {

                        filteredHoneyDos = honeyDos.Where(item => !item.IsComplete);

                    }       

                }

            }

          • Home.razor

          • Edit.razor (uses Route Parameters ... so that it handle handle edits and new ToDos)
            @page "/edit/{Id}"

            @page "/add"

            @using HoneyDo.Shared.Services

            @using HoneyDo.Shared.Interfaces

            @using HoneyDo.Shared.Models

            @inject NavigationManager NavigationManager

            @inject ILocalStorage LocalStorage

            @inject IPhotoManager PhotoManager

            
            

            <PageTitle>"Honey Do"" Task</PageTitle>

            <h3>Edit Task</h3>

            
            

            <EditForm class="form-control-sm" Model="@honeyDo" @ref=form>

                <DataAnnotationsValidator />

                <ValidationSummary />

            
            

                <div class="form-group">

                    <label class="col-form-label" for="Task">Task</label>

                    <InputText id="Task" @ref="taskBox" class="form-control" @bind-Value="@honeyDo.Task" />

                </div>

                <div class="form-group">

                    <label class="col-form-label" for="Description">Description</label>

                    <InputTextArea id="Description" class="form-control" @bind-Value="@honeyDo.Description" />

                </div>

                <div class="form-group">

                    <label class="col-form-label" for="DueDate">Due Date</label>

                    <InputDate id="DueDate"  class="form-control" @bind-Value="@honeyDo.DueDate" />

                </div>

                <div class="form-group">

                    <label class="col-form-label" for="IsComplete">Completed?</label>

                    <InputCheckbox id="IsComplete" class="form-check-inline" @bind-Value="@honeyDo.IsComplete" />

                </div>

                @{#if ANDROID || IOS || WINDOWS }

                <table>

                    <tr>

                        <td >

                            @{#if ANDROID || IOS }

                            <button class="btn btn-secondary" @onclick="@TakePhoto">Photo</button>

                            @{#endif}

                        </td>

                        <td><button class="btn btn-secondary" @onclick="@PickPhoto">Pick</button></td>

                        <td><button class="btn btn-secondary" @onclick="@ClearPhoto">Clear</button></td>

                    </tr>

                    <tr>

                        <td colspan="3" >                   

                            <img src="@honeyDo.Image" width="250" />

                        </td>

                    </tr>     

                </table> 

            @{#endif}

                <p></p>       

                <button class="btn btn-primary" @onclick="@Save">Save</button>

                <button class="btn btn-danger" @onclick="@Cancel">Cancel</button>

            </EditForm>

            
            

            @code {

                private HoneyDoModel honeyDo;

                [Parameter]

                public string? Id { get; set; }

                private EditForm? form { get; set; }

                private InputText taskBox;

            
            

                protected override async Task OnAfterRenderAsync(bool firstRender)

                {

                    if (firstRender)

                    {

                        await taskBox.Element.Value.FocusAsync();

                    }

               }

                protected override void OnInitialized()

                {

                    if (Id == null)

                    {

                        Id = "0";

                    }
            
            //When adding a NEW ToDo

            
            

            
            

            //When editing uri will NOT contain add

            
            

                    if (!NavigationManager.Uri.Contains("add"))

                    {           

                        var id = int.Parse(Id);

                        honeyDo = HoneyDoService.GetModelById(id, LocalStorage);       

                    } 

                    if (honeyDo == null)

                    {

                        honeyDo = HoneyDoService.AddItem();

                    }

                }

                private async void TakePhoto()

                {

                    honeyDo.Image = await PhotoManager.TakePhotoAsync();

                    StateHasChanged();

                }

                private async void PickPhoto()

                {

                    honeyDo.Image = await PhotoManager.PickPhotoAsync();

                    StateHasChanged();

                }

                private void ClearPhoto()

                {   

                    honeyDo.Image = "_content/HoneyDo.Shared/placeholder-image.jpg";

                    StateHasChanged();

                }

                private void Save()

                {     

                    if (form.EditContext.Validate())

                    {

                        HoneyDoService.SaveModel(LocalStorage);

                        NavigationManager.NavigateTo("/");

                    }

                }

                private void Cancel()

                {

                    HoneyDoService.ReloadModel(LocalStorage);

                    NavigationManager.NavigateTo("/");

                }

            }

            
            

      • Note the use of 

        • Blazored LocalStorage to save ToDos in the Web and Client Projects

          • added to the Dependencies in the Web Project and Client Project

          • ... and Program.cs
            using HoneyDo.Shared.Interfaces;

            using HoneyDo.Web.Components;

            using HoneyDo.Web.Services;

            using Blazored.LocalStorage;

            
            

            var builder = WebApplication.CreateBuilder(args);

            
            

            // Add services to the container.

            builder.Services.AddRazorComponents()

                .AddInteractiveServerComponents()

                .AddInteractiveWebAssemblyComponents();

            
            

            //Use Blazored OSS library for browser-local storage

            builder.Services.AddBlazoredLocalStorage();

            
            

            // Add device-specific services used by the HoneyDo.Shared project

            builder.Services.AddScoped<ILocalStorage, LocalStorage>();

            builder.Services.AddSingleton<IPhotoManager, PhotoManager>();

          • In Services folder LocalStorage.cs contains version specific to using Blazored LocalStorage on a typical Web App

            • using Blazored.LocalStorage;

              
              

              namespace HoneyDo.Web.Services

              {

                  public class LocalStorage : Shared.Services.LocalStorage

                  {

                      private ILocalStorageService LocalStorageService;

              
              

                      public LocalStorage(ILocalStorageService _localStorageService)

                      {

                          LocalStorageService = _localStorageService;

                      }

              
              

                      protected override string ReadFromStorage()

                      {

                          ValueTask<string> task = LocalStorageService.GetItemAsync<string>(StorageKeyName);

                          return task.GetAwaiter().GetResult();

                      }

              
              

                      protected override void SaveToStorage(string jsonString)

                      {               

                          LocalStorageService.SetItemAsync(StorageKeyName, jsonString);           

                      }

                  }

              }

          • Preferences to store data locally in .MAUI Project (Windows Desktop, Mobile Apps)

            • public class LocalStorage : HoneyDo.Shared.Services.LocalStorage

              {

                  protected override string ReadFromStorage()

                  {

                      return Preferences.Get(StorageKeyName, "");

                  }

                  protected override void SaveToStorage(string jsonString)

                  {           

                      Preferences.Set(StorageKeyName, jsonString);            

                  }

              }

        • QuickGrid to display results in Web app and Windows Desktop

          • added to the Shared Project and the Web Project

• Supplementary Demos

  • ToDoNET9WasmNewLayout

    • This is a simple To Do application that uses the Blazor WebAssembly Standalone Template and .NET 9.0. This version is a bit more stylized with a Top Navigation Menu instead of the classical Side Navigation Menu. The application stores the "Todos" locally using Javascript. This is in contrast to the HoneyDo application in this Lecture , which stored "Todos" locally by implementing the NuGet Package BlazoredLocalStorage for the Web App and the .MAUI Preferences command for the Mobile and Desktop Apps.

      Here are some the key teaching points/features of the application related to the Coding:

      • In the Models folder the class TaskItem.cs is declared. A simple class with 3 properties ... Id,Name,IsComplete

      • In the Interface folder we declared a simple Interface called ITaskService.cs ... 4 Tasks are declared ... GetTaskItemsAsync,SaveTaskItemsAsync,AddTaskItem,RemoveTaskItem

      • Now in the Services folder we create the Service/Class called TaskService which implements the Interface above

        • GetTaskItemsAsync ... returns a List of TaskItems from Local Storage using Javascript ... localStorage.getItem

        • SaveTaskItemsAsync ... saves the List of TaskItems to local Storage using Javascript ... localStorage.setItem

        • ... Recall we need to declare this Service in Program.cs so it can be injected in our desired Pages

      Now we focus on the UI Development

      • We will not use the traditional Side menu , but rather a Top Navigation Technique.

      • First in App.razor we change the DefaultLayout to TopNavLayout.razor

      • Now we go into the Layout folder and create TopNavLayout consisting of a TopNavMenu and Footer layout

      • TopNavMenu.razor will contain the Home and To Do Task Links

      • Footer.razor will contain 3 columns ... Id Section, Udemy section,Social Media links

      • Now we focus on the Pages folder

        • The Home page is updated with a new Welcome

        • The TaskPage is the focus. It implements an EditForm to display an InputText control and an Add Button. Once a Todo is entered we cycle through the entire list and display the current Todos in a separate component called TaskLine. This TaskLine component (located in the UI folder) will have a Delete button and a Checkbox to mark the Todo as complete. Note the use EventCallbacks in the TaskLine component since Delete and CheckBox actions will occur in the Parent component (TaskPage.razor)

        • The Styling for each of these pages occurs in local stylesheets (isolated stylesheet) connected to each Razor Component Page
          ... TaskLine.razor.css/Taskpage.razor.css
          ... TopNavMenu.razor.css/Footer.razor.css

  • FormValidDisplayNameNET9

    • This is a simple demo application which reviews and extends our knowledge of how to use forms and validations in Blazor WebAssembly.

      Click on the Forms and Validations link in the navigation bar to see the demo.

      ... Features to note are:

      • DisplayName="Car Drivers Age" hard coded right in the InputNumber component

      • The Addition of some SEO (Search Engine Optimization ... Check out the index.html file in the wwwroot folder

      • The use of C# conditional preprocessor directives and predefined constants you can use to check the Blazor target framework

        • @code {

             

              private string? blazorVersion{ get; set; }

          
          

              protected override void OnInitialized()

              {

                  #if NET6_0

                      blazorVersion = "NET6.0";   

                  #elif NET7_0

                      blazorVersion = "NET7.0";   

                  #elif NET8_0

                      blazorVersion = "NET8.0";   

                  #elif NET9_0

                      blazorVersion = "NET9.0";   

                  #endif

              }   

          }

  • BlazorAutoCompleteNET9

    • A demonstration of the concept of AutoComplete , implementing a simple Service (although I called it SuggestionsApi) which we inject into this page to fetch matching names from a list

      Here are the Details

      • When you type a name in the input box, the service is called to fetch matching names

      • When you click on a suggestion, the name is selected and added to the input box and the suggestions are cleared

      • First we create a class called DataSeed which will contain a list of name suggestions ... not exhaustive but a fair size... See MoreNames for a large list of Male only names (2700+)

      • Then we create a class called SuggestionsApi (Service) which will have a method FindAsync to fetch matching names

      • Next we register the service in the Program.cs file

      • Finally we inject the service into this page and use it to fetch matching names

      • On the main page, IntroduceYourself.razor the key ideas are: The use of bind:after (in the inputbox) to execute the task AutocompleteAsync (accesses the FindAsyc Task from the injected Service),and making the suggestion names list clickable in the foreach statement.

  • BlazorMusicPadUpdated

    • This project is a simple music pad that plays sounds when the corresponding key is pressed or button clicked. This application reviews and extends our knowledge of Interfaces, Dependency Injection, and JS Interop.

    • First we created a Models folder to hold our classes,and interface

      • Our base class is called PadModel. A class containing 4 properties Id, SoundName,SoundUrl,Key and a constructor

    • Next we jump over to the wwwroot folder and create a folder called js to holder our javascript code

      • Our javascript code is called audioplayer.js and it contains a function called playaudio

      • Make sure you add a reference to audioplayer.js in the index.html file

    • Now we go back to our Models folder and create a simple Interface called ISoundPlayer with a Task called Play

      • ... and finally we create a class/service called SoundPlayer. This class implements the ISoundPlayer interface and uses IJSRuntime to call the JavaScript function playaudio . Remember we are going to inject this service into our MusicPage and MusicPad components so we need to register it in the Program.cs file

    • Now we focus on the UI

      • We create a non-routable Razor Component called MusicPad which implements a parameter referencing the PadModel class. This component is used to display the pads on the screen. Each pad has a name, sound url, and key. The key is the key on the keyboard that will trigger the sound. Note the use of a css file to style the pads. Also note how the sound is played when the pad is clicked using the OnPadClicked method which is called when the pad is clicked. The sound is played using the injected soundPlayer

      • Finally we create a routable Razor Component called MusicPage which is the main page of the application. This page contains a list of pads of type PadModel created using the PadModel constructor. Each pad has a name, sound url, and key. The pads are displayed on the screen using the MusicPad component referenced in the foreach loop in the HTML section. Note how on this page we are listening for Keypresses and playing the sound of the pad using the injected soundPlayer

  • HySqLiteN9profile

    • .NET MAUI Blazor Hybrid .NET 9 Application with full CRUD ability using an SQLite DB

    • Displays a table of user profiles (Id, Name, Email) and also a QuickGrid display

    • Supports creating, loading, editing and deleting profiles (with confirmation dialogs).

    • Responsive UI: Action buttons in Table Display and QuickGrid (Edit/Delete) are hidden on Android, instead we us row taps for options which pops up a MAUI DisplayActionSheet

    • Navigation to add/edit profile page (UpdatePage.razor)

    • UpdatePage uses route parameters ie page "/addOrEdit/{id:int}"

In this Lecture we will

• Discuss how the integration of ChatGPT in modern web applications adds to engagement and a more personalized experience for users.

• Learn how to implement (integrate) OpenAI (ChatGPT) into a Blazor Web App

  • Learn that the first step in the process is creating an account on the OpenAI Developer Platform site

    • 
      

  • Learn how to generate an API key that we need to use in our Blazor app ... once you login go here:

    • Newer OpenAI keys can be as long as 164 characters in size

      • Make sure to make a copy of your key right away because you will not be able to see it again after it is created.

      • You can of course delete this key and create a new one if necessary

    • Next make note of  the OpenAI’s API endpoint for Chat Completions: ... choose API Reference ...  Chat Completions )

      • we will need this endpoint when we use the HttpClient technique (one of two techiques) to access the API ... recall Lectures 112-113 "Accessing Data Services that Implement the JSON:API"

      • Then click on the </> Code button and select curl ... you will see the Endpoint right at the top

      • You will also see a sample Request body which gives us a clue how we will code the request later in Blazor

      • You will also see a sample Response

      • Note: Circa May 2025 OpenAI now recommends trying Responses instead of Chat Completions (See Lecture 213)

    • Take a look at a number of the OpenAI API models available

      • GPT-4o

      • GPT-4o mini

    • Pricing

      • Accessing  the OpenAI API is not free and cost varies depending on usage and on the model being accessed.

      • Go to Settings (Cog wheel image) ... then Billing

        • Here you can use "Pay as you go" ... you add your payment method and setup an initial amount as little as 5-10 dollars which can be recharged when your credit balance reaches zero.

        • See the Pricing link   ... note there is a separate cost for input and output based on 1 million tokens (a basic unit of text used by the model to process information ... a token is a chunk of text that can be a whole word, part of a word or even a single character depending on how the text is tokenized)

        • Check out Usage (Dashboard)

  • Create our first simple Blazor application which utilizes the OpenAI API (key removed in demos in Resources ... you supply your own)

    • BlazorAInet9 (WebAssemblyStandalone App)

      • Introductory example with an input box and button. Users can type a prompt into the input field and hit the button to get a response

      • This first application uses the HttpClient technique which calls the OpenAI endpoint (we determined this endpoint in the introductory steps above)

      • The key here is we need to know the format of the "Response" from the API just like we have done before when accessing other data service APIs

        • Go to API Reference (top right menu) ... Chat Completions (left side menu) ... Create Chat Completion

        • I copied the JSON response (bottom right side display)  and then returned to my  application, created a Models page, added a class called ChatResponse and performed     Paste special -> Paste JSON as classes
          

          Below is the FULL response ... BUT I only selected some of the Key elements (properties) from each class
          

          ChatbotResponse .... public Choice[] choices {get;set;}

          Choice ... public Message message {get;set;}

          Message ... public string role {get;set;} and public string content {get;set;}

          
          

          //public class ChatbotResponse

          //{

          //    public string id { get; set; }

          //    public string _object { get; set; }

          //    public int created { get; set; }

          //    public string model { get; set; }

          //    public string system_fingerprint { get; set; }

          //    public Choice[] choices { get; set; }

          //    public string service_tier { get; set; }

          //    public Usage usage { get; set; }

          //}

          
          

          //public class Usage

          //{

          //    public int prompt_tokens { get; set; }

          //    public int completion_tokens { get; set; }

          //    public int total_tokens { get; set; }

          //    public Completion_Tokens_Details completion_tokens_details { get; set; }

          //}

          
          

          //public class Completion_Tokens_Details

          //{

          //    public int reasoning_tokens { get; set; }

          //    public int accepted_prediction_tokens { get; set; }

          //    public int rejected_prediction_tokens { get; set; }

          //}

          
          

          //public class Choice

          //{

          //    public int index { get; set; }

          //    public Message message { get; set; }

          //    public object logprobs { get; set; }

          //    public string finish_reason { get; set; }

          //}

          
          

          //public class Message

          //{

          //    public string role { get; set; }

          //    public string content { get; set; }

          
          

          //    // used to instruct the JSON serializer to ignore the marked property

          //    // when serializing or deserializing JSON data.

          //    [JsonIgnore]

          //    public bool IsUser => role == "user";

          //}

      • ChatGPTpage

        • All the remaining key coding is done in this razor component ... no Services or appsettings.json are used. (I highlight some more  advanced applications in the next lecture which implement these techniques)

        • Again one of the key concepts is how to talk to the OpenAI API ... the clue is here:

          • Go to API Reference (top right menu) ... Chat (left side menu) ... Create Chat Completion

            • Look at the format of the JSON Request ... we will use a slightly modified version of this written in C#
              
              completion = client.chat.completions.create(

                model="gpt-4o",

                messages=[

                  {"role": "developer", "content": "You are a helpful assistant."},

                  {"role": "user", "content": "Hello!"}

                ]

              )

              print(completion.choices[0].message)

            • Notice all the Request parameters that are available. We will use Temperature.

        • Let's start off with the code section and some simple declarations

          @code {

              private string userPrompt=string.Empty;

              private string? responseText;

        • Now let's go up and set up our UI in the HTML section
          @page "/chatgptpage"

          @using System.Text.Json

          @using System.Text

          @inject HttpClient Http

          
          

          <h3>ChatGPT API Integration</h3>

          
          

          <div class="mb-3">

              <label for="userInput">Enter your Question</label>

              <input @bind="userPrompt" id="userInput" class="form-control"/>

              <button @onclick="SubmitPrompt" class="btn btn-primary mt-3">@buttonText</button>

          </div>

          
          

          @if (responseText !=null)

          {

              <div class="alert alert-info">

                  <p>@responseText</p>       

              </div>

          }

        • Finally we focus on the remaining code
          
          private readonly string apiKey = "yourKey";

          
          

          private async Task SubmitPrompt()

          {

              buttonText = "Sending ... ";

          
          

              //Sending a Request to the ChatGPT API endpoint for chat completions

              var url = "";

          
          

              //The requestData object specifies:

              //the model (gpt-4o-mini),

              //the role of each message (system/user)

              //and the user's input

              //The system message is used to define ChatGPT's role as a helpful assistant

              //temperature is set to 0.7 to control the randomness of responses

              //... lower values are more deterministic, higher values make the responses more creative

          
          

              //Key ideas came from the OpenAI API documentation (See Request Body)

           

          
          

              var requestData = new

              {

                  model = "gpt-4o-mini",

                  messages = new[]

                  {

                      new{role="system",content="Your are a helpful assistant"},

                      new{role="user",content=userPrompt}

                  },

                  temperature=0.7

              };       

          
          

              //Here we are adding an Authorization header to the HTTP request message.

              //The header includes a Bearer token, which is a type of access token used for authentication.

              //The token is stored in the apiKey variable.

              //...and finally we serialize the request data as JSON file using UTF-8 (Unicode Transformation Format - 8-bit)

              //which is a variable-width character encoding used for electronic communication.

              // It can represent every character in the Unicode character set.

              var requestMessage = new HttpRequestMessage(HttpMethod.Post, url);

              requestMessage.Headers.Add("Authorization", $"Bearer {apiKey}");

              requestMessage.Content = new StringContent(JsonSerializer.Serialize(requestData), Encoding.UTF8, "application/json");

          
          

              try

              {

                  //Send the request to the OpenAI API

                  var response = await Http.SendAsync(requestMessage);

          
          

                  //if the request succeeds we use ReadAsStringAsync() to serialize the content to a string

                  //then we parse the response (Deserialize) using the ChatResponse class

                  //and display the first response message from the choices array

                  //This array contains possible responses from ChatGPT and we show the one

                  //with the highest likelihood

                  if (response.IsSuccessStatusCode)

                  {

                      var responseContent = await response.Content.ReadAsStringAsync();

                      var completion = JsonSerializer.Deserialize<ChatResponse>(responseContent);

                      responseText = completion.choices[0].message.content;

                  }

                  else

                  {

                      responseText = " Error: " + response.StatusCode;

                  }

              }

              catch (Exception ex)

              {

                  responseText = "An error occured: " + ex.Message;

              }

              finally

              {

                  buttonText = "Send";

              }

              //This block is executed after the try block and any associated catch blocks,

              // regardless of whether an exception was thrown or caught.

              // It is used for cleanup code that must run no matter what.

    • Test application with various inputs

      • note that the response is not nicely formatted ... here's the fix

        • BlazorAInet9HTMLupdate

        • var requestData = new

          {

               model = "gpt-4o-mini",

               messages = new[]

               {

                   new{role="system",content="Your are a helpful assistant" + "format every response in HTML"},

                   new{role="user",content=userPrompt}

               },

               temperature=0.7

          };   

        • @if (responseText !=null)

          {

              <div class="alert alert-info">

                  <p>@((MarkupString)responseText)</p>       

              </div>

          }

• Supplementary Demos

  • BlazorAIwasmImagesHttpClientTechn/BlazorAIwasmImagesSpeechHttpClientTechnUpdate

    • This example reviews and extends our knowledge of connecting to the OpenAI API using Blazor WebAssembly and the HttpClient Technique. In this demo we implemented a Service (OpenAIServices) to handle all the API calls

      • One to handle the Text Response  GenerateTextAsync and one to handle the Image Generation GenerateImageAsync and one to handle Text to Speech Generation ConvertTextToSpeechAsync

      • Then we used the service in our Blazor component (Home) to generate the required responses from the OpenAI API.

    • Also note that we did not create any classes to represent the API text response,instead using
      var json = await response.Content.ReadFromJsonAsync<JsonElement>();

      json.GetProperty("choices")[0].GetProperty("message").GetProperty("content").GetString();

      • Since JsonElement is used, the code does not rely on a predefined class like ChatResponse. Instead, it manually accesses properties in the JSON response using methods like GetProperty().

        and then simply storing the response in a string on the Home page.

    • ... Also, Check out the AI Image  and Text to Speech Links as an additional Enrichment demo

      • For the OpenAI image generation API, you do not need to specify the model in the request body.

        The model is implicitly defined by the endpoint you are using.

        The  endpoint is specifically for generating images, and it does not require a model parameter.

        • var requestBody = new

          {

               prompt = prompt,

               n = 3, //number of images to generate

               size = "512x512"

          };
          
          var requestContent = new StringContent(JsonSerializer.Serialize(requestBody), Encoding.UTF8, "application/json");
          requestContent.Headers.ContentType = new MediaTypeHeaderValue("application/json");
          httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", _apiKey);
          var response = await _httpClient.PostAsync("", requestContent);
          
          var json = await response.Content.ReadFromJsonAsync<JsonElement>();
          
          if (json.TryGetProperty("data", out var dataArray) && dataArray.ValueKind == JsonValueKind.Array)

          {

          
          

              //Extracts the image URLs from the JSON response and returns them as a LIST of strings.

              var imageUrls = dataArray.EnumerateArray()

                  .Select(img => img.GetProperty("url").GetString())

                  .Where(url=> !string.IsNullOrEmpty(url))

                  .ToList();

          
          

              return imageUrls;

          }

      • For the Text to Speech Generation again you do specify a model in the requestBody plus we reference the endpoint in our Post request.

        The endpoint is specifically for generating speech, and it does not require a model parameter.

        • var requestBody = new

          {

              model = "tts-1",

              input = text,

              voice = voice

          };
          
          var jsonBody = JsonSerializer.Serialize(requestBody);
          _httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", _apiKey);
          var content = new StringContent(jsonBody, Encoding.UTF8, "application/json");
          var response = await _httpClient.PostAsync("", content);
          
          var audioBytes = await response.Content.ReadAsByteArrayAsync();//Asynchronously reads the HTTP content and stores it in a byte array.

          var base64Audio = Convert.ToBase64String(audioBytes);          //Converts the byte array to a base64-encoded string.

          var audioUrl = "data:audio/mp3;base64," + base64Audio;         //Constructs a data URL for the audio file.

          return audioUrl;

        • Note the use of Voices in the SpeechPage.razor (we use a drop down list ... select) to display all the choices
          //

          //This list is used to populate the dropdown list of available voices

              private readonly List<string> AvailableVoices = new List<string> { "alloy", "echo", "fable","onyx","nova","shimmer","ash","sage","coral" };   

        • Note the use of Javascript to play the AudioUrl
          //fetch the new speech URL

          AudioUrl = await OpenAIServices.ConvertTextToSpeechAsync(TextInput,SelectedVoice);
          await JSRuntime.InvokeVoidAsync("eval", "new Audio('" + AudioUrl + "').play();");

        • Note in the HTML section where we allow the user to download/save the audio generated
          @if (!string.IsNullOrEmpty(AudioUrl))

          {

              @*Here the we download instead of link to href ... download location depends on browser default*@

              <div class="text-center">

                  <a href="@AudioUrl" download="output.mp3" class="btn btn-success mt-3">Download Speech</a>

              </div>

          }

  • BlazorHybridAIimagesHttpClientTechn-2026

    • .NET MAUI Blazor Hybrid App version of previous demo focusing on Image Generation

    • Uses a Tabbed Menu along the top (modified in MainPage.xaml)  with the link OpenAI Image API ... this a Native .MAUI page (NewPage2.xaml)  not a basic Razor

      • We use a bit of XAML to update the NewPage2.xaml and add some Text (Labels) , some Buttons, a Image container , an Entry container (like a text box) and a Switch.

      • Note the associated code-behind necessary to sense the Clicked events

      • We also added A Text to Speech command button which details the purpose of this page (Image Generation)

      • We have the Switch respond to a state change (Toggled) by making the loaded in image appear and disappear

      • Added ...VerticalOptions="Center" ...HorizontalOptions="Center" to VerticalStack and BackgroundColor to ContentPage to tweak appearance somewhat

        • <ContentPage
          BackgroundColor="LightBlue">

      • Added a new button to Close MAUI page if called Modally from Counter page ... does not work if page is simply accessed via Tabbed Menu

        • Associated code behind for button uses command ... Navigation.PopModalAsync();

        • Code on Counter page uses ...
          App.Current.MainPage.Navigation.PushModalAsync(new Pages.NewPage2());

  • BlazorAInet9UsingOpenAInuget

    • A redo of today's Lecture demo application (BlazorAInet9) using the OpenAI .NET Library

      • 
        

    • The OpenAI .NET library provides convenient access to the OpenAI REST API from .NET applications. It implements a number of classes/methods which reduce the coding necessary to send and receive responses from the OpenAI API.

      • First we must install the OpenAI NuGet Package

    • Next we store the OpenAI Key in a class called Constants.cs

      • public class Constants

        {

            public const string OpenAIKey = "yourKey";

        }   

    • The rest of the coding is in the razor component OpenAInuget.razor

      • @page "/openainuget"

        @using OpenAI.Chat 

        
        

        <h5>Implementing the OpenAI .NET Library (NuGet Package)</h5>

        
        

        <div class="mb-3">

            <label for="userInput">Enter your Question</label>

            <input @bind="userPrompt" id="userInput" class="form-control" />

            <button @onclick="SubmitPrompt" class="btn btn-primary mt-3">@buttonText</button>

        </div>

        
        

        @if (responseText != null)

        {

            <div class="alert alert-info">

                <p>@((MarkupString)responseText)</p>

            </div>

        }

        
        

        @code {

        
        

            private string userPrompt = string.Empty;

            private string? responseText;

            private string buttonText = "Send to ChatGPT";

        
        

            private async Task SubmitPrompt()

            {

                buttonText = "Sending ... ";

        
        

                if (!string.IsNullOrEmpty(userPrompt))

                {

                    var modelName = "gpt-4o-mini";

                    var client = new ChatClient(modelName, Constants.OpenAIKey);

                    var response = await client.CompleteChatAsync(userPrompt + " format every response in HTML");

                    string answer = response.Value.Content[0].Text;

                    responseText = answer;

                    buttonText = "Send to ChatGPT";

                }

                else

                {

                    buttonText = "Send to ChatGPT";

                }

            }

        }

  • BlazorAInet9UsingOpenAInugetEnhancedUpd

    • This updated version keeps track of our entire conversation with ChatGPT. This way the Bot has a frame of reference if a followup question is asked.

      ... for instance ... What is the tallest building in the world? ... whats the next?

    • Key updates in Code Section

      • //The ChatMessage class is typically used to represent a message in a chat conversation.

        // In the context of the OpenAI .NET library, it is used to structure the messages exchanged with the OpenAI API.

        // Each ChatMessage object usually contains information about the sender (e.g., user or assistant)

        // and the content of the message.

        List<ChatMessage> chatMessages = new List<ChatMessage>();   

      • private async Task SubmitPrompt()

        {

            buttonText = "Sending ... ";

        
        

        
        

            if (!string.IsNullOrEmpty(userPrompt))

            {

                // Add the User's message to the chatMessages list so we can keep track of the conversation history.

                // This helps the OpenAI API to maintain context across multiple interactions.

                // Input messages from the caller are typically paired with the assistant's responses.... See below

                 chatMessages.Add(new UserChatMessage("Make sure to format every response in HTML"));       
                  chatMessages.Add(new UserChatMessage(userPrompt));

                           

                var modelName = "gpt-4o-mini";

                var client = new ChatClient(modelName, Constants.OpenAIKey);

               

                // Send the entire conversation to the OpenAI API ... instead of just current userPrompt

                var response = await client.CompleteChatAsync(chatMessages);   

                //var response = await client.CompleteChatAsync(userPrompt + " format every response in HTML");

               

                string answer = response.Value.Content[0].Text;

                chatMessages.Add(new AssistantChatMessage(answer));
        // Add the Assistant's response to the chatMessages list

        // Input messages from the caller are typically paired with the assistant's responses.

        // This way the OpenAI API can maintain context across multiple interactions.

                responseText += "<b>ChatGPT:</b>" + answer + "<br><br>";

                buttonText = "Send to ChatGPT";

            }

            else

            {

                buttonText = "Send to ChatGPT";

            }

        
        

        }

  • BlazorAutoGenOpenAInugetNBA

    • NBA Teams (History) is a new menu option in this application, designed to showcase the power of AI-generated content. Our .NET App Generates Content for itself via AI.

      • AI-Powered Summaries: The application uses advanced AI to generate detailed histories for NBA teams, providing unique and insightful content.

      • JSON Data Storage: All initial NBA team data is stored in a JSON file, structured according to the GameItem.cs model. This ensures data consistency and easy updates.

      • Service Architecture:

        • Service folder contains the IOpenAIService interface and its implementation OpenAIService, which handles communication with the AI backend. We are using the OpenAI nuget package for this purpose.

        • GameItemService is responsible for loading and updating the NBA teams data from the JSON file.

      • Main Page: The GameItems.razor page displays all NBA teams and allows users to request an AI-generated history for any team.

      • Team History Modal: The generated history is shown in a non-routable component called ItemsDescriptionModal, which receives parameters to display the selected team's details.

In this Lecture we will

• Extend our AI implementation skills through a series of Hands On and Demo applications

• BlazorChatbotAInet9 (demo)

  • Advanced version of a Chatbot specifically used to answer questions related to C# programming and only C# programming.

  • Note how the class ChatbotResponse in the Models folder was developed

    • copied JSON response from OpenAI API (Paste special ... Paste JSON as classes)

      • 
        

    • A second class called ChatOptions is also added to the Models folder

      • public class ChatOptions

        {

            public string ApiKey { get; set; } 

            public string ApiUrl { get; set; } 

            public string GtpModel { get; set; }   

        }

  • We store our key values in the file appsettings.json in the  wwwroot folder
    {

      "OpenAI": {

        "ApiKey": "yourKey",

        "ApiUrl": "",

        "GtpModel": "gpt-4o-mini"

      }

  • This application still uses the HttpClient technique but puts most of the coding in a Service (class) called OpenAIService in the Services folder
    public class OpenAIService

    {

    
    

        private readonly HttpClient _httpClient;

       

        private readonly IOptions<ChatOptions> _options;

        //IOptions<T> is a feature in ASP.NET Core used for accessing configuration

        //settings. It provides a way to retrieve configuration values that are bound to a specific class.

        //Recall that ChatOptions is a class that holds configuration settings for the ApiKey,ApiUrl and GtpModel. 

       

        //The actual values are stored in the appsettings.json file.   

        //In the Program.cs we have ChatOptions refer to the OpenAI (key) section in the

        //appsettings.json configuration file. 

        //builder.Services.Configure<ChatOptions>(o => builder.Configuration.GetSection("OpenAI").Bind(o));

    
    

        //Class constructor that initializes a new instance of the class

        //with the required HttpClient and IOptions<ChatOptions> objects. 

        public OpenAIService(HttpClient httpClient,IOptions<ChatOptions> options)

        {

            _httpClient = httpClient;

            _options = options;           

        }

    
    

        //Note how CreateChatCompletion method is used in the ChatBot.razor.cs file

        //private void AddUserQuestionToConversation()

        //{

        //    _conversationHistory.Add(new Message { role = "user", content = _userQuestion });

        //}

    
    

        //private async Task CreateCompletion()

        //{

        //    _isSendingMessage = true;

        //    var assistantResponse = await OpenAIService.CreateChatCompletion(_conversationHistory);

        //    _conversationHistory.Add(assistantResponse);

        //    _isSendingMessage = false;

        //}

        public async Task<Message>CreateChatCompletion(List<Message>messages)

        {

            //Method used in BlazorAInet9 app (from previous Lecture)

            //var requestData = new

            //{

            //    model = "gpt-4o-mini",

            //    messages = new[]

            //{

            //    new{role="system",content="Your are a helpful assistant"},

            //    new{role="user",content=userPrompt}

            //},

            //    temperature = 0.7

            //};

            //---------------------------------------------------------------------------------

            var request = new { model = _options.Value.GtpModel, messages = messages.ToArray() };

    
    

            //Method used in BlazorAInet9 app (from previous Lecture)

            //var requestMessage = new HttpRequestMessage(HttpMethod.Post, url);

            //requestMessage.Headers.Add("Authorization", $"Bearer {apiKey}");

            //requestMessage.Content = new StringContent(JsonSerializer.Serialize(requestData), Encoding.UTF8, "application/json");

            //var response = await Http.SendAsync(requestMessage);

            //---------------------------------------------------------------------------------

            _httpClient.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", _options.Value.ApiKey);

            var response = await _httpClient.PostAsJsonAsync(_options.Value.ApiUrl, request);

    
    

            //EnsureSuccessStatusCode throws an exception if the IsSuccessStatusCode property for the HTTP response is false.

            response.EnsureSuccessStatusCode();

    
    

            //Method used in BlazorAInet9 app (from previous Lecture)

            //var chatCompletionResponseString = await response.Content.ReadAsStringAsync();

            //var chatCompletionResponse = System.Text.Json.JsonSerializer.Deserialize<ChatbotResponse>(chatCompleteResponseString);

            //---------------------------------------------------------------------------------

            var chatCompletionResponse = await response.Content.ReadFromJsonAsync<ChatbotResponse>();

    
    

            //return chatCompletionResponse?.choices[0].message;   

            return chatCompletionResponse?.choices.First().message;

        }

    
    

    }

  • Next we declare the Service in Program.cs
    / this line ensures that the settings defined under the "OpenAI" section in your configuration files

    // are loaded into an instance of the ChatOptions class, which can then be injected and used throughout

    // your application. This is useful for centralizing and managing configuration settings in a structured way.

    builder.Services.Configure<ChatOptions>(o => builder.Configuration.GetSection("OpenAI").Bind(o));

    builder.Services.AddScoped<OpenAIService>();

  • Now we focus on the main UI page ... ChatBot.razor/ChatBot.razor.cs

    • @page "/"   

      
      

      <div class="container d-flex flex-column vh-100">

         

          @*Only display the Opening Prompt at the start when no questions have been asked *@

          @if (Messages.Count==0)

          {

              <div class="text-center">

                      <h3>Welcome to the Chat Bot</h3>

                      <p>I'm your C# learning assistant ... ask me anything</p>

               </div>

          } 

         

          @*Cycle through all the questions and responses

              The user's questions are displayed in a blue long oval bubble with a person image on the left

              The bot's responses are displayed in a white long oval textarea with a robot image on the left

              The Messages (list) is a filtered _conversationHistory list that contains the user's questions and the bot's responses

          *@

          <div class="flex-grow-1">

      
      

              @foreach (var message in Messages)

              {

                  <div class="row p-2">

                      <div class="col-1 rounded-circle bg-dark-subtle text-center m-2 p-0" style="width:50px;height:50px;">

                          <span class="bi fs-2 @(message.IsUser ? "bi-person-fill": "bi-robot")" ></span>

                      </div>

                      <div class="col-11 rounded p-3 shadow-sm @(message.IsUser ? "text-bg-primary" : "text-bg-light")">

                         @if(message.IsUser)

                         {

                             @message.content

                         }

                         else

                         {

                             @( (MarkupString)message.content)

                         }

                      </div>

                  </div>

      
      

                  @*MarkupString forces Blazor to render the string as HTML rather than encoding it as plain text.

                  This is useful when you want to include HTML content in your Blazor components.*@

              }

      
      

          </div>

      
      

          @*The sticky-bottom class ensures that the input area is always at the bottom of the screen

            The input area consists of a button to clear the conversation, a textarea to enter the user's question

            and a button to send the question to the bot*@   

          <div class="sticky-bottom w-50 p-3 rounded mx-auto">

             

              @if (_isSendingMessage)

              {

                  <div class="text-center">

                      <div class="spinner-border spinnder-border-sm" role="status">

                          <span class="visually-hidden">Loading ...</span>

                      </div>

                  </div>

              }
      

              <div class="input-group">       

                  <button class="btn btn-primary" @onclick="ClearConversation">

                      <span class="bi bi-eraser"></span>

                  </button>       

                  <textarea rows="2" class="form-control" placeholder="Ask me anything ..."

                  @bind-value="_userQuestion" @bind-value:event="oninput" @onkeypress="HandleKeyPress"

                  disabled="@_isSendingMessage">

                  </textarea>

              </div>

      
      

          </div>

      
      

      </div>

    • using BlazorChatbotAInet9.Models;

      using BlazorChatbotAInet9.Services;

      using Microsoft.AspNetCore.Components;

      using Microsoft.AspNetCore.Components.Web;

      using System.Security.Principal;

      using System.Text.Json.Serialization;

      
      

      namespace BlazorChatbotAInet9.Pages

      {

          public partial class ChatBot

          {

              private string _userQuestion = "";

              private readonly List<Message> _conversationHistory = new List<Message>();

              private bool _isSendingMessage;

              private readonly string _chatBotKnowledgeScope = "" +

                 "Your name is CsharpBot, You are an assistant that help users learn C#." +

                 "When user's question is not related to C# or the .NET framework, reply politely that you can not answer" +

                 "format every response in HTML.";

      
      

              //The _chatBotKnowledgeScope field above defines the initial context and behavior for the chatbot.

              //It sets the chatbot's identity and scope of knowledge. Specifically, it instructs the chatbot to:

              //1. Identify itself as "CsharpBot".

              //2. Act as an assistant that helps users learn C#.

              //3. Politely decline to answer questions that are not related to C# or the .NET framework.

              //4. Format every response in HTML.

      
      

              protected override Task OnInitializedAsync()

              {

                  _conversationHistory.Add(new Message { role="system",content=_chatBotKnowledgeScope});

                   return Task.CompletedTask;

              }

      
      

              //The purpose of this property is to provide a list of messages that excludes any messages with the role "system".

              //This is useful for displaying only user and assistant messages in the chat interface,

              //ignoring any system messages that might be used for internal purposes.

      
      

              public List<Message> Messages => _conversationHistory.Where(c=>c.role is not "system").ToList();

      
      

              //Recall the Message class from the ChatbotResponse.cs file

              //public class Message

              //{

              //    public string role { get; set; }

              //    public string content { get; set; }

      
      

              //    // used to instruct the JSON serializer to ignore the marked property

              //    // when serializing or deserializing JSON data.

              //    [JsonIgnore]

              //    public bool IsUser => role == "user";

              //}

      
      

              [Inject]

              public OpenAIService OpenAIService { get; set; }

      
      

              private async Task HandleKeyPress(KeyboardEventArgs e)

              {

                  if (e.Key is not "Enter") return;

                  await SendMessage();   

              }

      
      

              private async Task SendMessage()

              {

                  if (string.IsNullOrWhiteSpace(_userQuestion)) return;

                  AddUserQuestionToConversation();

                  StateHasChanged();

                  await CreateCompletion();

                  ClearInput();

                  StateHasChanged(); 

              }

      
      

              private void AddUserQuestionToConversation()

              {

                  _conversationHistory.Add(new Message { role="user", content = _userQuestion });

              }

             

              private async Task CreateCompletion()

              {

                  _isSendingMessage = true;

                  var assistantResponse= await OpenAIService.CreateChatCompletion(_conversationHistory);

                  _conversationHistory.Add(assistantResponse);

                  _isSendingMessage = false;

              }

      
      

              private void ClearInput()

              {

                  _userQuestion = "";

              }   

      
      

              private void ClearConversation()

              {

                  ClearInput();

                  _conversationHistory.Clear();   

              }

          }

      }

  • Note the update to the MainLayout.razor page which has removed the reference to the NavMenu.razor ... so it will default to "/" ... this page directive has been added to the ChatBot.razor page
    @inherits LayoutComponentBase

    <main>

        @Body

    </main>

• BlazorWASMAppOpenAI (.NET 8 WebAssembly Standalone App)

  • Blazor WASM Application using the Azure OpenAI NuGet package

    • Simple "Tourist Recommendation" application that prompts for the input of  a City and then allows you choose between Restaurants,Hotels or Attractions.

  • Lets first add an appsettings.json file to the wwwroot of the project. This will automatically be picked up by Blazor. The content should look like this.
    {

      "OpenAIKey": "yourKey",

      "OpenAIEndpoint": null

    }

  • Next we install the Azure.AI.OpenAI nuget Package (use 1.0.0-beta.8)

  • Now create a Services folder and create a class called OpenAIService

    • using Azure.AI.OpenAI;

      using Azure;
      
      namespace OpenAIRecommendationAppBlazor.Services;

      public class OpenAIService

      {

          //HttpClient vs OpenAIClient   

         

          //HttpClient is a more general-purpose class for sending HTTP requests and receiving HTTP responses from a resource identified by a URI.

      //Can be used to interact with any web service or API by manually constructing HTTP requests.

      //Highly flexible, but requires more boilerplate code to handle specific API interactions, including setting headers,

          //handling authentication, and parsing responses.

      
      

          //OpenAIClient is a class provided by the Azure.AI.OpenAI library

          //and is specifically designed to interact with OpenAI's API services.

          //Provides higher-level abstractions for OpenAI-specific operations like generating completions, chat completions, etc.

          //Simplifies the process of making requests to OpenAI by encapsulating the details of the

          //API endpoints and request/response handling.

      
      

          OpenAIClient client;

         

          //static readonly char[] trimChars = new char[] { '\n', '?' };

      
      

          //This method is responsible for initializing the OpenAIClient instance, which is used to interact with OpenAI's API services

          //OpenAIService is initialized with an API key and a custom endpoint.

          //If the custom endpoint is not needed, we just use the API key.

         

          //This Initialize method is called from the Program.cs file to set up the OpenAIService instance before running the Blazor app.

          //Program.cs is the entry point for the Blazor WebAssembly application and is responsible for setting up the app's configuration and services.

         

          // Set up the OpenAI client

          //OpenAIService svc = new OpenAIService();

          //builder.Services.AddSingleton<OpenAIService>(svc);

          //var app = builder.Build();

          //var config = builder.Configuration;

          //var openAIKey = config["OpenAIKey"]!;

          //var openAIEndpoint = config["OpenAIEndpoint"];

          //svc.Initialize(openAIKey, openAIEndpoint);

          //await app.RunAsync();

      
      

          public void Initialize(string openAIKey, string? openAIEndpoint = null)

          {

              client = !string.IsNullOrWhiteSpace(openAIEndpoint)

                  ? new OpenAIClient(

                      new Uri(openAIEndpoint),

                      new AzureKeyCredential(openAIKey))

                  : new OpenAIClient(openAIKey);

          }

      
      

          //This Task is called from the TouristPage.razor file to generate a chat completion based on the user's input.

          //async void OnSubmit()

          //{

          //    responseText = "Working on it...This can take a little while based on the model selected.";

          //    var message = await openAIService.CallOpenAIChat(recommendation.RecommendationType, recommendation.City);

          //responseText = message;

          //    this.StateHasChanged();

          //}

      
      

          internal async Task<string> CallOpenAIChat(string recommendationType, string location)

          {

              //This is the prompt we want to send to the OpenAI API ...

              //for example ... Give me 5 recommendations for "restaurant" near "New York" format every response in HTML

              string prompt = GeneratePrompt(recommendationType, location);

             

              ChatCompletionsOptions options = new ChatCompletionsOptions();

      
      

              //options.ChoiceCount = 1;

      
      

              //Recall from BlazorAInet9 example how we set up our requestData object

              //var requestData = new

              //{

              //    model = "gpt-4o-mini",

              //    messages = new[]

              //    {

              //        new{role="system",content="Your are a helpful assistant" + "format every response in HTML"},

              //        new{role="user",content=userPrompt}

              //    },

              //    temperature = 0.7

              //};

      
      

              //Here is how we set up your request data using Azure.AI.OpenAI

              options.Messages.Add(new ChatMessage(ChatRole.User, prompt));

              var response = await client.GetChatCompletionsAsync("gpt-4o-mini",options);

      
      

              // StringWriter is used to accumulate the responses from the OpenAI API into a single string,

              // which is then returned by the CallOpenAIChat method.

              StringWriter sw = new StringWriter();

      
      

              foreach (ChatChoice choice in response.Value.Choices)

              {

                  //var text = choice.Message.Content.TrimStart(trimChars);

                  var text = choice.Message.Content;

                  sw.WriteLine(text);

              }

              var message = sw.ToString();

              return message;

          }

      
      

          private static string GeneratePrompt(string recommendationType, string location)

          {

              //return $"What is a recommendation for  {recommendationType} near {location}";

              //return "What is a recommendation for  " + recommendationType + " near " + location;

              //return "Give me 5 recommendations for " + recommendationType + " near " + location + " format every response in HTML";

              return "Give me recommendations for " + recommendationType + " near " + location + " format every response in HTML";   

          }

      }

  • We'll register the OpenAI service we just created. This is done in Program.cs.

    • Replace the builder.Build().RunAsync call with the following, the config isn't available until the builder is built, so we register the service then initialize it based on the config once it's available.

    • // Set up the OpenAI client

      OpenAIService svc = new OpenAIService();

      builder.Services.AddSingleton<OpenAIService>(svc);

      
      

      var app = builder.Build();

      var config = builder.Configuration;

      var openAIKey = config["OpenAIKey"]!;           //gets key from appsettings.json

      var openAIEndpoint = config["OpenAIEndpoint"];  //get endpoint from appsettings.json   

      svc.Initialize(openAIKey, openAIEndpoint);      //calls Initialize method (in Service-OpenAIService) to set up OpenAIService instance
      

      await app.RunAsync();

  • ... and lastly we create a simple UI called TouristPage.razor ( use TouristPageCodeSnippet.txt )

    • @page "/touristpage"

      
      

      @using OpenAIRecommendationAppBlazor.Services;

      @using System.ComponentModel.DataAnnotations;

      @inject OpenAIService openAIService;

      
      

      <PageTitle>Index</PageTitle>

      
      

      <h1>Local AI Recommendations</h1>

      
      

      <EditForm OnValidSubmit="OnSubmit" Model="@recommendation">

          <DataAnnotationsValidator />

          <ValidationSummary />

      
      

          <div class="form-group row">

      
      

              <div class="form-group col-5">

                  <label>City</label>

                  <InputText @bind-Value="@recommendation.City" class="form-control"></InputText>

              </div>

      
      

              <div class="form-group col-5">

                  <label>I'd like a recommendation for ...</label>

                  <InputSelect @bind-Value="@recommendation.RecommendationType" class="form-select">

                      <option value="">--Select--</option>

                      <option value="restaurant">Restaurants</option>

                      <option value="hotel">Hotels</option>

                      <option value="attractions">Attractions</option>

                  </InputSelect>

              </div>

      
      

              <div class="form-group col-2">

                  <button type="submit" class="btn btn-success mt-4">

                      Submit

                  </button>

              </div>

      
      

          </div>

      
      

      </EditForm>

      
      

      @* <div>

          <p>@responseText</p>   

      </div> *@

      
      

      <div>

          <p>@((MarkupString)responseText)</p>

      </div>

      
      

      @code

      {

          Recommendation recommendation = new Recommendation();

          string responseText = "";

      
      

          async void OnSubmit()

          {

              responseText = "Working on it...This can take a little while based on the model selected.";

              var message = await openAIService.CallOpenAIChat(recommendation.RecommendationType, recommendation.City);

              responseText = message;

              this.StateHasChanged();

          }

      
      

          //Here we defined our class locally instead of in a separate file

          public class Recommendation

          {

              [Required]

              public string City { get; set; } = "";

              [Required]

              public string RecommendationType { get; set; } = "";

          }

      }

• BlazorHybridOpenAI /BlazorHybridOpenAIappShellXAMLupdate (demo)

  • .NET MAUI Blazor Hybrid version of the Recommendation App using the Azure OpenAI package

    • ... as of Nov 2025 works on .NET 8 but not yet on .NET 9 ... so will not work with Visual Studio 2026

  • Key Highlights

    • No appsettings.json .... so we created a class called Constants.cs

      • //static means that the class and its members can be

        //accessed without creating an instance of the class.

        static internal class Constants

        {

            public const string OpenAIKey = "yourKey";

            public const string OpenAIEndpoint = null;

        }

    • We register the Service (OpenAIService) and call the Initialize method from MauiProgram.cs

      • builder.Services.AddTransient<MainPage>();

        OpenAIService svc = new OpenAIService();

        svc.Initialize(Constants.OpenAIKey, Constants.OpenAIEndpoint);

        builder.Services.AddSingleton<OpenAIService>(svc);

    • We added a xaml page called AppShell.xaml  which routes to the MainPage on startup and adds a Title

      • <Shell

            x:Class="OpenAIRecommendationAppMaui.AppShell"

            xmlns=""

            xmlns:x=""

            xmlns:local="clr-namespace:OpenAIRecommendationAppMaui"

            Shell.FlyoutBehavior="Disabled"

            Title="OpenAIRecommendationAppMaui"

            BackgroundColor="LightGrey">

        
        

            <ShellContent

                Title="Chiarelli OpenAI API Demo"       

                ContentTemplate="{DataTemplate local:MainPage}"

                Route="MainPage" />

        </Shell>

    • Most of the UI is created in  MainPage.xaml with the code behind in MainPage.xaml.cs

      • <ContentPage xmlns=""

                     xmlns:x=""

                     x:Class="OpenAIRecommendationAppMaui.MainPage">

        
        

            <ScrollView

                BackgroundColor="LightGray"

                Padding="10"

                VerticalScrollBarVisibility="Always"

                HorizontalScrollBarVisibility="Never">

               

                <VerticalStackLayout

                    Padding="30,0"

                    Spacing="25">

                    <Label

                        Text="Local AI recommendations"

                        SemanticProperties.HeadingLevel="Level1"

                        FontSize="32"

                        HorizontalOptions="Center" />

                    <Entry

                        x:Name="LocationEntry"

                        Placeholder="Enter your location"

                        PlaceholderColor="Yellow"

                        SemanticProperties.Hint="Enter the location for recommendations"

                        HorizontalOptions="Center"/>

                    <Button

                        x:Name="RestaurantBtn"

                        Text="Get restaurant recommendations"

                        SemanticProperties.Hint="Gets restaurant recommendations when you click"

                        Clicked="OnRestaurantClicked"

                        HorizontalOptions="Center" />

        
        

                    <Button

                        x:Name="HotelBtn"

                        Text="Get hotel recommendations"

                        SemanticProperties.Hint="Gets hotel recommendations when you click"

                        Clicked="OnHotelClicked"

                        HorizontalOptions="Center" />

        
        

                    <Button

                        x:Name="AttractionBtn"

                        Text="Get attraction recommendations"

                        SemanticProperties.Hint="Gets attraction recommendations when you click"

                        Clicked="OnAttractionClicked"

                        HorizontalOptions="Center" />

        
        

                    <Label x:Name="SmallLabel"

                        Text="Click a button for recommendations!"

                        SemanticProperties.HeadingLevel="Level2"

                        FontSize="18"

                        HorizontalOptions="Center" />

                </VerticalStackLayout>

            </ScrollView>

        
        

        </ContentPage>

      • using OpenAIRecommendationAppMaui.Services;

        
        

        namespace OpenAIRecommendationAppMaui

        {

            public partial class MainPage : ContentPage

            {

                OpenAIService openAIService;

                public MainPage(OpenAIService svc)

                {

                    openAIService = svc;

                    InitializeComponent();

                }

                private async void OnRestaurantClicked(object sender, EventArgs e)

                {

                    await GetRecommendation("restaurant");

                }

        
        

                private async void OnHotelClicked(object sender, EventArgs e)

                {

                    await GetRecommendation("hotel");

                }

        
        

                private async void OnAttractionClicked(object sender, EventArgs e)

                {

                    await GetRecommendation("attraction");

                }

        
        

                private async Task GetRecommendation(string recommendationType)

                {

                    if (string.IsNullOrWhiteSpace(LocationEntry.Text))

                    {

                        await DisplayAlert("Empty location", "Please enter a location (city or postal code)", "OK");

                        return;

                    }

                    SmallLabel.Text = "Working on it...This can take a little while based on the model selected.";

                    var message = await openAIService.CallOpenAIChat(recommendationType, LocationEntry.Text);

                    SmallLabel.Text = message;

                }

            }

        }

• Supplementary Demos

  • BlazorAIsemanticKernel

    • This is a Blazor WebAapp .NET 9  (Render Mode Server) that uses the Microsoft Semantic Kernel SDK to connect to the OpenAI API.

      • Just when you thought the  Azure OpenAI NuGet package or the OpenAI .NET Library (from last lecture) made working with the OpenAI API easier than the HttpClient technique along comes the Microsoft Semantic Kernel SDK

    • Key Features and Implementation Steps

      • Install the Microsoft.SemanticKernel NuGet Package

      • Update Program.cs
        using Microsoft.SemanticKernel;

        var builder = WebApplication.CreateBuilder(args);

        builder.Services.AddKernel().AddOpenAIChatCompletion(modelId:"gpt-4o-mini",apiKey: "yourKey");

      • Add a simple class called ChatMessage in the Models folder
        public class ChatMessage

        {

            public string Text { get; set; }   

            public bool IsUserMessage { get; set; }

           
           //Constructor
           //isUserMessage is a boolean that indicates whether the message
           //is from the user (true) or the bot (false)

            public ChatMessage(string text, bool isUserMessage)

            {

                Text = text;

                IsUserMessage = isUserMessage;

            }

        }

      • KernelPage.razor Page

        • Code Section

          private string buttonText = "Send";

          private string progressWidth = "0%";

          private bool isLoading = false;

          private string userInput = "";

          private List<ChatMessage> chatMessages = new List<ChatMessage>();

          private string apiResponse = "";

          
          

          private async Task SendMessage()

          {

              buttonText = "Sending...";

              isLoading = true;

              StateHasChanged();

          
          

              try

              {

                  //Add the user message to the chat messages list true indicates it's a user message
                   chatMessages.Add(new ChatMessage(userInput, true));

          
          

                  apiResponse = "";

                  int messageIndex = -1;

          
          

                  await foreach (var message in kernel.InvokePromptStreamingAsync<string>(userInput))

                  {

                      apiResponse += message.Replace("\n", @"<br>").Replace("\t", @"<tab>");

                     
                     //If the messageIndex is -1, it means that the current message is the first message        //from the API, so we add a new ChatMessage to the chatMessages list with the //apiResponse and set the IsUserMessage property to false to indicate that it is a bot //message. If the messageIndex is not -1, it means that the message is a continuation
                     
                     if (messageIndex == -1)

                      {

                          chatMessages.Add(new ChatMessage(apiResponse, false));

                          messageIndex++;

                      }

                      else

                      {

                          chatMessages.LastOrDefault().Text = apiResponse;

                      }

          
          

                      this.StateHasChanged();

          
          

                  }

          
          

                  userInput = "";

          
          

              }

              catch (Exception ex)

              {

                  apiResponse = "API call failed: " + ex.Message;

              }

              finally

              {

                  buttonText = "Send";

                  isLoading = false;

                  StateHasChanged();

              }

          }

        • HTML Section
          @page "/kernelpage"

          
          

          @using Microsoft.SemanticKernel

          @inject Kernel kernel

          @rendermode InteractiveServer

          
          

          <PageTitle>Home</PageTitle>

          
          

          <h5>Using Microsoft Semantic Kernel SDK</h5>

          
          @*Note the reference to a number of classes ... check out KernelPage.razor.css

              for the styling of these classes

          *@
          

          <div class="chat-container">

              @foreach (var message in chatMessages)

              {

                  <div class="chat-message @(message.IsUserMessage ? "user-message" : "bot-message")">

                      @((MarkupString)message.Text)

                  </div>

              }

          </div>

          
          

          <div class="message-input">   

             

              <input disabled="@isLoading" type="text" @bind="userInput" placeholder="Type your message here..." />

          
          @*Lots going on in this one button ... lots of ternary operators

              to set the class and the disabled attribute.

              The button has a span for the text and a div for the loading bar

              The loading bar is shown when isLoading is true

          *@

              <button disabled="@string.IsNullOrEmpty(userInput)" class="@(string.IsNullOrEmpty(userInput)||isLoading? "disabled":"enabled")"

                      @onclick="SendMessage">

                  <span>@buttonText</span>

                  <div class="loading-bar @(isLoading ? "show":"")"></div>

              </button>

          
          

              <div class="loading-bar @(isLoading ? "show":"")" style="width:@progressWidth"></div>   

          
          

          </div>

        • Check out KernelPage.razor for css coding

  • BlazorKernelVisionWebAppNET9auto/BlazorKernelVisionWebAppUpdate... PreviewImgUpd

    • These series of applications are an extension to the previous one that introduced using the Kernel to perform a simple chat completion task.

    • This time, we will use the Kernel to Analyze Images using the OpenAI API Vision.

    • This is a Blazor Web App Application ... NET 9 Auto Render Mode.This application has a Server Project and WebAssembly Project ... but all the coding is located in the Server Project

    • Make sure to check out the references added to _Imports.razor and Program.cs. The rest of the coding is located in the KernelVisionServer page

      • @*Required for Vision Implementation*@

        @using Microsoft.SemanticKernel.ChatCompletion

        @using Microsoft.SemanticKernel.Connectors.OpenAI

      • using Microsoft.SemanticKernel;

        app.MapRazorComponents<App>()

            .AddInteractiveServerRenderMode()

            .AddInteractiveWebAssemblyRenderMode()

            .AddAdditionalAssemblies(typeof(BlazorKernelVisionWebApp.Client._Imports).Assembly);

    • BlazorKernelVisionWebAppNET9auto

      • This version analyzes a pre-set URL image or allows the user to use the InputFile component to pick any local image

      • Key Elements Code Section

        • ChatMessageContent reply;

          //This will hold the response from the API where //ChatMessageContent is from //Microsoft.SemanticKernel.AI.ChatMessages and is used

          //to represent the content of a chat message.

          
          // This is the method that gets triggered when the user selects an image

          private async Task OnFileChanged(InputFileChangeEventArgs e)

          {

              isTooLarge = false;

             

              var file = e.File;

              imageFileSize = file.Size;

          
          

              if (imageFileSize > 65520) // This is the maximum size that the .NET implementation of Uri can handle

              {

                  isTooLarge = true;                       

              }

          
          

              var buffer = new byte[512000]; //initialize buffer to 512000 bytes

          
          

              if (file.Size > 512000) // This is a separate check for the file size just to display the image on screen

              {

                  // The file is too large

                  resizedFile = await file.RequestImageFileAsync(file.ContentType, 300, 300);

                  buffer = new byte[resizedFile.Size];

          
          

                  //OpenReadStream max allowed 512000 bytes ... so we need to read in a reduced sized image file

                  var stream = await resizedFile.OpenReadStream().ReadAsync(buffer);

              }

              else

              {

                  buffer = new byte[file.Size];

                  var stream = await file.OpenReadStream().ReadAsync(buffer);           

              }

          
          

              image = Convert.ToBase64String(buffer);

             

              imageBase64 = "data:" + file.ContentType + ";base64," + image;   

          }

           

          
          

          private async Task AnalyzeImage(int type)

          {

              //Inject the ChatCompletionService to get the API response from the Kernel

              var chatCompletionService = kernel.GetRequiredService<IChatCompletionService>();

          
          

              //Prompt messages for the chat completion

              string systemPrompt = "You are a friendly chatbot that helps me analyze images. I will show you an image and you will tell me what you see in it.";

              string userPrompt = "Analyze the image and tell me what you see in it.";

          
          

              //Create a new ChatHistory object with the system prompt

              var chatHistory = new ChatHistory(systemPrompt);

          
          

              //type 1 is for image URL and type 2 is for local image

              if (type==1)

              {

                  isImageURL = true;

                 

                  chatHistory.AddUserMessage(new ChatMessageContentItemCollection

                  {

                      new TextContent(userPrompt),

                      new ImageContent(new Uri(imageUri))                 

                  });

              }

              else

              {

                  isImageURL = false;

                 

                  chatHistory.AddUserMessage(new ChatMessageContentItemCollection

                  {

                      new TextContent(userPrompt),

                      new ImageContent(imageBase64) //.NET implementation of Uri has a length limitation (65520 bytes) so it can only process small images.

                     

                   });

              }   

             

              reply = await chatCompletionService.GetChatMessageContentAsync(chatHistory);

          }

        • Key Elements HTML

          • @*This button is used to Analyze an image using it's URL address ... which is hard coded as imageUri in the code section *@

            <button class="btn btn-primary me-3" @onclick="@(() => AnalyzeImage(1))" >Analyze URL Image</button>

            
            

            @*This button is used to load a local image file.

              The standard InputFile component displays a simple button

              ... by surrounding it with the label using the class btn we now have a nice bootstrap button  *@

            <label class="btn btn-primary">

                Choose Local Image

                <InputFile OnChange="OnFileChanged" accept="image/png,image/jpeg" style="display: none;" />

            </label>

          • @if (imageBase64 != null)

            {

                @*Here we are displaying the local image that was selected by the user

                 ... But the .NET implementation of Uri has a length limitation (65520 bytes) so it can only process small images.*@

                <p><strong>Selected Image:@imageFileSize</strong><i>bytes</i> ... NET implementation has a length limitation of 65520 bytes </p>

                <img src="@imageBase64" alt="Selected Image" style="max-width: 300px; max-height: 300px;" />

            
            

                @*If the image is too big to analyze disable the button *@

                <button class="btn btn-primary" disabled="@isTooLarge" @onclick="@(() => AnalyzeImage(2))" style="margin-left: 10px;">Analyze Local Image</button>

            }

            
            

            @if (reply != null)

            {

                <div class="alert alert-info mt-3">

                    <h5 >API Response:</h5>

                    @if(isImageURL)

                    {

                        <img src="@imageUri" alt="Selected Image" style="max-width: 300px; max-height: 300px;" />

                        <br />

                    }       

                    @reply

                </div>

            }

    • BlazorKernelVisionWebAppUpdate

      • In this version we display a list of image links located in the wwwroot folder and allow the user to select (click) the image to analyze.

      • Key Updates Code Section
        
        //List of image files in the wwwroot/images directory

        List<string> fileList = new List<string>();
        
        //create a method to automatically load a set of images on program startup

        protected override async Task OnInitializedAsync()

        {       

            //C# lets you add the @ symbol in front of a string to create a

            //verbatim string literal where backslashes are not interpreted as escape characters

            //... so you can use them to specify file paths               

            string path = Directory.GetCurrentDirectory() + @"\wwwroot\images";

           

            //GetFiles ... returns an array of the full names (including paths) for the files in the specified directory

            //[0] = "C:\\Temp\\Blazor\\Vision\\4-BlazorKernelVisionWebAppUpdate\\BlazorKernelVisionWebApp\\BlazorKernelVisionWebApp\\wwwroot\\images\\beach.jpg"       

            var files = Directory.GetFiles(path);

        
        

            //GetFileName returns the file name and extension of the specified path string ... beach.jpg       

            foreach (var file in files)

            {

                fileList.Add(Path.GetFileName(file));

            }

        }   
        
        //The ReadFile method is reading a file from the local directory (wwwroot/images) that you have clicked on from a list on screen

        //and it accepts a simple STRING parameter (beach.jpg) for instance

        
        

        //The OnFileChanged method is reading the file from ANYWHERE on the user's computer

        //The selected file is stored in the File property of the InputFileChangeEventArgs OBJECT, which is of type IBrowserFile

        //IBrowserFile is an interface . It provides properties and methods to access the file's metadata and content like Size, ContentType, OpenReadStream, etc.

        
        

        private void ReadFile(string file)

        {

            var path = Directory.GetCurrentDirectory() + @"\wwwroot\images\" + file;

            var buffer = File.ReadAllBytes(path);

            image = Convert.ToBase64String(buffer);

            imageBase64 = "data:image/jpeg;base64," + image;

        
        

            //determine the image file size

            FileInfo fi = new FileInfo(path);

            imageFileSize = fi.Length;

        
        

            isTooLarge = false;

        
        

            StateHasChanged();

        }   

      • Key Updates HTML
        <div class="col-md-3">

            @if (fileList.Count > 0)

            {           

                <h5>Select Image </h5>

                <span>wwwroot folder</span>

                <ul style="list-style-type: none;padding:0px;margin:0px;">

                    @foreach (var file in fileList)

                    {

                        <li><span @onclick="@( () => ReadFile(file))" style="cursor:pointer; text-decoration:underline;color:blue">@file</span></li>                   

                        @*Here we are using a Lambda expression since we are calling a method and passing a parameter to it*@

                    }

                </ul>

            }

        </div>

    • BlazorKernelVisionWebAppUpdatePreviewImgUpd

      • In this last version we display a list of image links + the actual images located in the wwwroot folder and allow the user to select (click) the image to analyze.

      • Key update

        <li class="mb-2"><span @onclick="@( () => ReadFile(file))" style="cursor:pointer; text-decoration:underline;color:blue"><img src="/images/@file" height="50" width="50" style="padding:5px" />@file</span></li>

  • BlazorSmartComponents

    • This is a simple application which uses the Smart Components NuGet Package to add 3 different AI powered features to a Blazor Web App (.NET 9 WebAssembly RenderMode)

      The 3 components are:

      • SmartTextArea ... Allows you to configure how text will be autocompleted including tone and URLs

      • SmartComboBox ...upgrades the traditional combobox by making suggestions based on semantic matching ... helps users find what they are looking for faster (as you type a value an autocompleting dropdown appears). The traditional combo box suggests values only based on eact substring matches, but SmartCombobox upgrades this by suggesting matches closely related to meanings.

      • SmartPasteButton ... fills out forms automatically using data pasted in from the user's clipboard ... it just knows where to put the data

    • Here are the Implementation Steps:

      • Smart Components work in .NET 6 or higher ... Here we are using .NET 9 Web App with Render Mode WASM ... you do need to have an ASP.NET Core Server so you cannot use a Blazor WASM Standalone App. This is because you need a server to hold your API keys.

      • In the Server project add the NuGet Package SmartComponents.AspNetCore

      • In the WebAssembly project add the NuGet Package SmartComponents.AspNetCore.Components (not required if you only have a Server project)

      • Update Program.cs in the Server Project with builder.Services.AddSmartComponents();

      • To specifically use SmartPaste and SmartTextArea you need to provide access to a language model backend ... so we need to Configure an OpenAI backend in the Server Project

        • Add the SmartComponents.Inference.OpenAI NuGet Package

        • In Program.cs update the call to AddSmartComponents ... builder.Services.AddSmartComponents().WithInferenceBackend OpenAIInferenceBackend ();

      • Using Azure OpennAI or OpenAI endpoints ... we need to Configure API keys in appsettings.json ... "SmartComponents":{"ApiKey":"your key","DeploymentName":"gpt-4o-mini","Endpoint":null}

    • Now you can use the Smart Components in your Blazor Web App

      • Smart Paste you simplying add SmartPasteButton DefaultIcon

      • Smart ComboBox ... SmartComboBox Url="api/accounting-categories" ... Note here you need to set up an API endpoint matching the Url Setting up the API endpoint

        • In the server's Program.cs file you add an array of candidates

        • In the server's Program.cs you add app.MapSmartComboBox("api/accounting-categories",request=>candidates)

        • This version is not very smart since it will always use the fixed suggestions (stored in the array)

        • ... Adding Semantic Matching by using embeddings ... a way of converting natural language strings into numerical vectors.

        • First we reference and register the embedder Service

        • Install the NuGet Package .. SmartComponents.LocalEmbeddings in the Server project

        • In Program.cs register it ... builder.Services.AddSingleton LocalEmbedder

In this Lecture we will

• Learn about  an open-source tool called Ollama that simplifies running large language models (LLMs) locally on your computer, offering a user-friendly way to access and interact with powerful AI models without relying on cloud services.

  • Ollama acts as a local model manager and runtime, handling the complexities of downloading, installing, and managing LLMs, allowing users to focus on interacting with them

    • Why it's useful:

      • Local AI: It enables running LLMs directly on your own hardware, providing greater control over data and privacy.

      • Open-Source Focus: Ollama primarily works with open-source models, fostering transparency and customization.

      • Simplified Workflow: It streamlines the process, making it easier to get started and experiment with LLMs.

      • Cost-effective: Eliminates the need for cloud infrastructure, potentially saving on recurring costs.

    • Platform Support:

      Ollama supports macOS, Windows, and Linux.

• Learn about Microsoft.Extensions.AI a set of core .NET libraries developed in collaboration with developers across the .NET ecosystem, including Semantic Kernel. These libraries provide a unified layer of C# abstractions for interacting with AI services, such as small and large language models (OpenAI,Azure,Ollama)

• Create a simple Blazor Application that implements  basic Image Recognition functionality (Vision) BlazorImageRecognitionMSextensionsAI (NET 9 Blazor WebApp Interactive Render Mode Server) ... A redo a similar application from the previous Lecture Supplementary Demos that used the Microsoft Semantic Kernel to perform Image Analysis.

  • Leverage Microsoft.Extensions.AI library for image recognition tasks

    • A new package from Microsoft that provides the scaffolding and tools to build AI-powered applications without starting from scratch.

    • Think of Microsoft.Extensions.AI as the evolution of the Microsoft. SemanticKernel Abstractions package that we highlighted in the Supplementary Demos in the last Lecture.

    • The .NET AI Chat template (See Lecture 207 for full details) was built using the Microsoft.Extensions.AI

  • Integrate Ollama in our .NET application to process visual data using a local LLM (llava) eliminating the need for cloud API calls while maintaining privacy and control over our data.

  • First we set up the development environment

    • install Microsoft.Extensions.AI  (NuGet Package)

    • install Microsoft.Extensions.AI.Ollama  (NuGet Package)

      • this library will allow us to interact with an Ollama model we will install locally

    • install Ollama (free open source tool) ... then

      • note all the models supported ... filter based on Vision

      • We are going to use the LLaVA model. It  is a novel end-to-end trained large multimodal model that combines a vision encoder and Vicuna for general-purpose visual and language understanding....

      • we are going to pick the 7b version that has a size of 4.7 GB

      • install model 

        • open up the console  (in windows enter "cmd")

        • We use the command ollama pull llava:7b to download the model from the Ollama Library

        • since we have Ollama installed locally we simply type  ollama run llava:7b to start an interactive chat session with the model

  • Now it's time to add this model to our application ... we do that in the Program.cs file

    • using Microsoft.Extensions.AI;
      // Add Ollama AI service to the container.

      builder.Services.AddChatClient(new OllamaChatClient(new Uri("http://localhost:11434"), "llava:7b"));

      • The port number 11434 is the specific port on which the Ollama AI service is listening for incoming HTTP requests.

      • The Blazor application will communicate with the Ollama AI service at this address to perform tasks such as chat operations using the specified AI model ("llava:7b").

  • Next we add an images folder in the wwwroot folder and add a number of high quality png/jpg files

    • See images.rar in the Resources or use your own

  • All the remaining coding will occur in the Home.razor page (removing all original code)

    • we start off by adding some required declarations

      @rendermode InteractiveServer

      
      

      @using Microsoft.Extensions.AI;

      @inject IChatClient ChatClient

      
      

      @*necessary declaration... allows us to use Directory and Path commands in the code section*@

      @using System.IO

    • We are going to have two methods to pick images ... one method will display on screen the contents of the default images folder or we let the user go anywhere on their computer and click on a image file (InputFile component)

      • Lets prep some of the code first
        
        private string responseMessage;

        private string status; //either Processing or Results

        
        

        private string image; //used to form imageBase64 below

        private string imageBase64;

           

        private bool isTooLarge = false; //for the Analyze Local Image button

        
        

        private long imageFileSize;

        IBrowserFile resizedFile; //Used in the OnFileChanged method to store the resized image file if the original is too large to display/process

        
        

        //List of image files in the wwwroot/images directory

        List<string> fileList = new List<string>();
        

      • Next lets create the task that will automatically load the contents of the images folder for us to display on screen for possible selection (one of two techniques ... this method uses hardcoded images)
        
        protected override async Task OnInitializedAsync()

        {

            //C# lets you add the @ symbol in front of a string to create a

            //verbatim string literal where backslashes are not interpreted as escape characters

            //... so you can use them to specify file paths

            string path = Directory.GetCurrentDirectory() + @"\wwwroot\images";

        
        

            //GetFiles ... returns an array of the full names (including paths) for the files in the specified directory

            //[0] = "C:\\Temp\\Blazor\\Vision\\4-BlazorKernelVisionWebAppUpdate\\BlazorKernelVisionWebApp\\BlazorKernelVisionWebApp\\wwwroot\\images\\beach.jpg"

            var files = Directory.GetFiles(path);

        
        

            //GetFileName returns the file name and extension of the specified path string ... beach.jpg

            foreach (var file in files)

            {

                fileList.Add(Path.GetFileName(file));

            }

        }

      • Now we are going to create two methods to load in the selected image and store it as a Base64  string. The one method will be for when we use the InputFile component and the second method is when we click on the pre-displayed images in the images folder.
        

        • private void ReadFile(string file)

          {

              var path = Directory.GetCurrentDirectory() + @"\wwwroot\images\" + file;

              var buffer = File.ReadAllBytes(path);

              image = Convert.ToBase64String(buffer);

              imageBase64 = "data:image/jpeg;base64," + image;

          
          

              //determine the image file size

              FileInfo fi = new FileInfo(path);

              imageFileSize = fi.Length;

          
          

              isTooLarge = false; //no issues with this technique so Recognize button fully enabled all the time

          
          

              StateHasChanged();

          }

        • private async Task OnFileChanged(InputFileChangeEventArgs e)

          {

             

              var buffer = new byte[512000]; //initialize buffer to 512000 bytes

          
          

              isTooLarge = false;

          
          

              //The selected file is stored in the File property of the InputFileChangeEventArgs object, which is of type IBrowserFile

              var file = e.File;

              imageFileSize = file.Size;

          
          

              if (imageFileSize > 65520) // This is the maximum size that the .NET implementation of Uri can handle

              {

                  isTooLarge = true;

          
          

                  // The file is too large

                  resizedFile = await file.RequestImageFileAsync(file.ContentType, 300, 300);

                  buffer = new byte[resizedFile.Size];       

                  var stream = await resizedFile.OpenReadStream().ReadAsync(buffer);                 

              }

              else

              {

                  buffer = new byte[file.Size];

                  var stream = await file.OpenReadStream().ReadAsync(buffer);
                  //This line reads the contents of the file into the buffer array

              }
          

              //Convert the buffer to a Base64 string and then create the data URI scheme for the image
          

              image = Convert.ToBase64String(buffer);

              imageBase64 = "data:" + file.ContentType + ";base64," + image; 

          
          

              StateHasChanged();

          }

        • The ReadFile method vs OnFileChanged method

          The ReadFile method is reading a file from the local directory (wwwroot/images) that you have clicked on from a list on screen

          and it accepts a simple STRING parameter (beach.jpg) for instance

          
          

          The OnFileChanged method is reading the file from ANYWHERE on the user's computer

          The selected file is stored in the File property of the InputFileChangeEventArgs OBJECT, which is of type IBrowserFile

          IBrowserFile is an interface . It provides properties and methods to access the file's metadata and content like Size, ContentType, OpenReadStream, etc.

      • Now lets setup our UI before we focus on our last task to actually Recognize the Image

        • <h5>Image Recognition with Microsoft AI Extensions</h5>

          
          

          @*This button is used to load a local image file.

            The standard InputFile component displays a simple button

            ... by surrounding it with the label using the class btn we now have a nice bootstrap button  *@

          <label class="btn btn-primary">

             
                Choose Local Image
          

              <InputFile OnChange="OnFileChanged" accept="image/png,image/jpeg" style="display: none;" />

          
          </label>

          
          

          <button class="btn btn-primary" disabled="@isTooLarge" @onclick="RecognizeImage" style="margin-left:10px;">Recognize Image</button>

          
          

          <hr/>

          
          @*Here we are going to divide the screen into two sections

              on the left we will display a list of images from the wwwroot folder

              on the right we will display the selected image*@
          

          <div class="row">

             

              <div class="col-md-3">

                  @if (fileList.Count > 0)

                  {

                      <span style="color:purple; font-size:small;"><b>Select Image from wwwroot</b></span>

          
          

                      <ul style="list-style-type: none;padding:0px;margin:0px;">

                          @foreach (var file in fileList)

                          {

                              <li class="mb-2"><span @onclick="@( () => ReadFile(file))" style="cursor:pointer; text-decoration:underline;color:blue"><img src="/images/@file" height="50" width="50" style="padding:5px" />@file</span></li>

                              @*Here we are using a Lambda expression since we are calling a method and passing a parameter to it*@

                          }

                      </ul>

                  }

              </div>   

             

             

              <div class="col-md-9 mt-3">

                  @if (imageBase64 != null)

                  {           

                      @*Here we are displaying the local image that was selected by the user *@           

                          <strong>Selected Image Original Size:@imageFileSize</strong><i>bytes</i>

                          <br />
                     

                      <img src="@imageBase64" alt="Selected Image" style="max-width: 300px; max-height: 300px;" />

                  }       

              </div>   

          </div>

          
          

          @status

          
          

          <br/>

          
          

          @if (responseMessage != null)

          {

              <div class="alert alert-info mt-3">

                  <h5>API Response</h5>

                  <p>@responseMessage</p>

              </div>   

          }

      • ... and finally the key AI Task  RecognizeImage()

        • private async Task RecognizeImage()

          {   

              if (imageBase64 == null)

              {

                  responseMessage = "Please select an image first.";

                  return;

              }

          
          

              status = "Processing";

          
          

              //Create a new ChatMessage object with the ChatRole.User role and the message "What's in this image?"

              //Add a new DataContent object to the message's Contents collection with the image data and the MIME type "image/jpg"

              //Call the GetResponseAsync method of the ChatClient object with the message as the parameter

              //and store the result in the response variable

              //Set the responseMessage variable to the response's Text property

          
          

          var message = new ChatMessage(ChatRole.User, "What's in this image?");

              message.Contents.Add(new DataContent(imageBase64, "image/jpg"));

              var response = await ChatClient.GetResponseAsync(message);

              responseMessage = response.Text;

          
          

              status = "Results";

          }

      • Test out the application

        • NOTE: Do NOT shut down Ollama ... it MUST be running in the background for proper execution.

      • Compare the speed of execution to a similiar application from the previous lecture implementing the Semantic Kernel and accessing the OpenAI API

        • BlazorKernelVisionWebAppUpdateNoKey

    • Let's experiment a little bit with what we can ask our image (vision)  model

      • Count the number of people in an image (BlazorImageRecognitionMSextensionsAIUpdated)
        var message = new ChatMessage(ChatRole.User, "How many people are in this image?");

      • Get back a Structured response from the model ... ie return an object defined by a class (BlazorImageRecognitionMSextensionsAIUpdatedClass) ... demo only

        • Structured Data Extraction in .NET: Microsoft.Extensions.AI allows us to  extract and deserialize structured information from freeform listings into typed C# objects.

        • we are going to try to determine the number of adults and children in the images and store the results

        • public class PictureResult

          {       

              public int TotalNumberOfPeople { get; set; }     

              public int NumberOfAdults { get; set; }

              public int NumberOfChildren { get; set; }           

          }

        • In  Task RecognizeImage()
          ///Change prompt to try and get a more structured response

          //*******************************************************

          var message = new ChatMessage(ChatRole.User, "What's in this image . Also count the total number of people in the image and how many are adults and how many are children");

          
          

          message.Contents.Add(new DataContent(imageBase64, "image/jpg"));

          var response1 = await ChatClient.GetResponseAsync(message);

          var response2 = await ChatClient.GetResponseAsync<PictureResult>(message);
          //Note:The method overload ... unique to Microsoft.Extensions.AI ... allows you to specify the type of response you are expecting ie an object of type PictureResult

          //The key is using the .Result property to access the PictureResult object

                 

          //Structured response
          //note here we are displaying the full response message and the structured response

          //also note the use of <br/> to format the response message so that the structured response is displayed on a new line

          //... because of this we are using the MarkupString type to display the response message on the HTML section

          //****************************

          responseMessage =  " <b>Standard Response</b> ... response1.Text" + "<br/>" + response1.Text + "<br/>" + "<br/>"+

                              "<b>New Response using PictureResult class </b> ... response2.Text " + "<br/>" + response2.Text + "<br/>"+ "<br/>" +

                              "<b>New Response using PictureResult class </b> ... response2.Result.TotalNumberOfPeople ... " +

                              "<br/> Total number of People: " + response2.Result.TotalNumberOfPeople + " Adults: " + response2.Result.NumberOfAdults + " Children: " + response2.Result.NumberOfChildren;

        • In the HTML section
          @if (responseMessage != null)

          {

              <div class="alert alert-info mt-3">

                  <h5>AI Response</h5>

                  ... to user Query : "What's in this image . Also count the total number of people in the image and how many are adults and how many are children"

                  <hr />

                  <p>@((MarkupString)responseMessage)</p>

              </div>   

          }

• Supplementary Demos

  • BlazorImgRecogMSextensionsAIClassUsingOpenAI

    • This application leverages the Microsoft.Extensions.AI Library for image recognition tasks. BUT, we replace our open source model with the OpenAI API model gpt-4o-mini for better results. Almost all the coding remains the same.

    • The key difference is the use of the Microsoft.Extensions.AI.OpenAI NuGet Library to interact with the OpenAI API. So the only program updates occur in the Program.cs file where we reference OpenAIChatClient instead of OllamaChatClient.

      • using Microsoft.Extensions.AI;

        using OpenAI;
        
        // Add AI service to the container ... OpenAI

        var openAiClient = new OpenAIClient("yourKey");

        builder.Services.AddChatClient(new OpenAIChatClient(openAiClient, "gpt-4o-mini"));
        

    • Biggest difference is the SPEED and somewhat the accuracy compared to the last Lecture demo BlazorImageRecognitionMSextensionsUpdatedClass

  • BlazorHybridMSExtensionsAI (Windows Machine version)

    • .NET MAUI Blazor Hybrid version of previous demo with the addition of a new FormResult class which contains the properties ReportBy,  ReportDate, ReportPhone, AttendantName, AttendantPhone, RecievedDate, EnteredDate. These will be used to store all the components of a given image which displays a business form.

    • Key Highlights

      • MauiProgram.cs
        using Microsoft.Extensions.AI;

        using OpenAI;
        // Add AI service to the container ... OpenAI

        var openAiClient = new OpenAIClient("yourKey");

        builder.Services.AddChatClient(new OpenAIChatClient(openAiClient, "gpt-4o-mini"));

      • We replace this Web App command ...with the .NET MAUI specific line below in both the ReadFile and OnInitializedAsync Tasks
        //var path = Directory.GetCurrentDirectory() + @"\wwwroot\images\" + file;
        

        //.NET MAUI uses the AppDomain.CurrentDomain.BaseDirectory property to get the directory where the application is running   (add @using System.IO in HTML section)
        

        string rootpath = System.IO.Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "wwwroot");

        string path = rootpath + @"\images\" + file;

      • Note the additional support for Image URLs

        • //Use HttpClient to download the image from the URL and then convert it to a base64 string for processing

          var httpClient = new HttpClient();

          var imageBytes = await httpClient.GetByteArrayAsync(imageUrl);

          var base64Image = Convert.ToBase64String(imageBytes);

          var imageData = $"data:image/jpg;base64,{base64Image}";

      • The last and most important update is the implementation of the class FormResult and the extraction of the key elements of a business form image.

        • public class FormResult

          {

              public string ReportBy { get; set; }

              public string ReportDate { get; set; }

              public string ReportPhone { get; set; }

              public string AttendantName { get; set; }

              public string AttendantPhone { get; set; }

              public string ReceivedDate { get; set; }

              public string EnteredDate { get; set; }

          }

        • private async Task RecognizeImage ()
          
          var message = new ChatMessage(ChatRole.User, "What's in this image . Pull out all the information from all the required questions and place them in the related properties called ReportBy, ReportDate,ReportPhone,AttendantName,AttendantPhone,RecievedDate,EnteredDate. If you are unable to extract specific information like that from the image please describe the visual content");

          
          

          message.Contents.Add(new DataContent(imageBase64, "image/jpg"));

          var response1 = await ChatClient.GetResponseAsync(message);

          var response2 = await ChatClient.GetResponseAsync<FormResult>(message);
          
          //Structured response ...

          //note here we are displaying the full response message and the structured response

          //also note the use of <br/> to format the response message so that the structured response is displayed on a new line

          //... because of this we are using the MarkupString type to display the response message on the HTML section

          //****************************

          responseMessage = " <b>Standard Response</b> ... response1.Text" + "<br/>" + response1.Text + "<br/>" + "<br/>" +

                             "<b>New Response using FormResult class </b> ... response2.Text " + "<br/>" + response2.Text + "<br/>" + "<br/>" +

                             "<b>New Response using FormResult class </b> ... response2.Result.ReportBy ... " + "<br/>" + "<hr/>" +

                             " Report By: " + response2.Result.ReportBy + " <br/> Report Date: " + response2.Result.ReportDate

                             + " <br/>  Report Phone: " + response2.Result.ReportPhone + " <br/> Attendant Name: " + response2.Result.AttendantName +

                              "  <br/> Attendant Phone: " + response2.Result.AttendantPhone + " <br/> Received Date: " + response2.Result.ReceivedDate +

                              " <br/> Entered Date: " + response2.Result.EnteredDate;

        • In the HTML section
          @if (responseMessage != null)

          {

              <div class="alert alert-info mt-3">

                  <h5>AI Response</h5>

                  @if (isImageURL)

                  {

                      <span>... to user Query : "What's in this image"</span>

                  }

                  else

                  {

                      <span>... to user Query : "What's in this image . Pull out all the information from all the required questions and place them in the related properties called ReportBy, ReportDate,ReportPhone,AttendantName,AttendantPhone,RecievedDate,EnteredDate"</span>

                  }

          
          

                  <hr />

                  <p>@((MarkupString)responseMessage)</p>

              </div>

          }
          

  • BlazorAIformImageToDB (updated version of  previous demo with Database implementation ... .NET 9 Blazor Web App using Interactive Render Mode WebAssembly ) 

    • Proof of Concept (Home.razor page ... in Server Project)

      • The Home page is our PROOF OF CONCEPT. It implements a class called FormResult (located in Shared Project ... more details below)  to further analyze an image of a form and provide a structured response. To do this we use the GetResponseAsync method with a FormResult type parameter. Note:The method overload ... unique to Microsoft.Extensions.AI ... allows you to specify the type of response you are expecting ie an object of type FormResult .The key is using the .Result property to access the FormResult object properties.The FormResult class has the following properties: ReportBy, ReportDate, ReportPhone, AttendantName, AttendantPhone, ReceivedDate, EnteredDate

    • The Next step is to take the structured response and store the results in a Database

      • Check out the View Current Form DB link in the Navigation Menu and Image AI page for all the details.

      • Note: This application is a .NET 9 Blazor Web App using Interactive Render Mode WebAssembly ... which includes a Server Project where this Home.razor page is located and a Client Project where FormPage.razor (View Current Form DB) and ImageAI.razor (new version of AI Image Recognition) pages are located + a FormEdit.razor page (for editing and deleting form data).

      • The key class FormResult is located in the Shared Project (Class Library).
        public class FormResult

        {

            public int Id { get; set; }

            public string? ReportBy { get; set; }

            public string? ReportDate { get; set; }

            public string? ReportPhone { get; set; }

            public string? AttendantName { get; set; }

            public string? AttendantPhone { get; set; }

            public string? ReceivedDate { get; set; }

            public string? EnteredDate { get; set; }

        }

      • The Microsoft.Extensions.AI and Microsoft.Extensions.AI.OpenAI librarys have been added to both the Server and Client projects

    • Database Implementation Key Teaching Points

      • We added a class library project (BlazorAIforms.Shared) which holds our FormResult class ... then make sure to add a Reference to this Shared  project in the Client and Server Projects

      • In the Server Project we added a folder called Controllers and then right clicked and add an API Controller with actions using Entity Framework. We also updated the appsettings.json (Server Project) and changed name of the Database=BlazorAIforms

      • Then we performed the migration and database update ... add-migration initial ... update-database

      • Check out the database in the Server Object Explorer and add a few records by hand

      • Before running the app we updated the Program.cs file (Server Project) so that it is aware of our use of a Controller api ... builder.Services.AddControllers() ... app.MapControllers()

      • We can now start up the app and enter the controller name (FormsResults) to view the rough Json text file output

      • .. All the prep is now done so we focus on the UI and coding for the remaining pages ...

        • ImageAI.razor (this page ... very similar to Home.razor)

          • HTML Section

            @rendermode @(new InteractiveWebAssemblyRenderMode(prerender:false))

            
            

            @inject HttpClient Http 

            @*Added to Program.cs in client project to implement HttpClient

            builder.Services.AddScoped(sp => new HttpClient { BaseAddress = new Uri(builder.HostEnvironment.BaseAddress) });*@

            
            <button class="btn btn-primary" @onclick="SaveDB" style="margin-left:10px;">Save To DB</button>

          • Code Section
            //an object of type FormResult to store the structured response

            //which will be saved to the database

            private FormResult formResult;
            
            private async Task RecognizeImage()
            { ...
            var message = new ChatMessage(ChatRole.User, "What's in this image . Pull out all the information from all the required questions and place them in the related properties called ReportBy, ReportDate,ReportPhone,AttendantName,AttendantPhone,RecievedDate,EnteredDate");

            
            

            message.Contents.Add(new DataContent(imageBase64, "image/jpg"));

            var response1 = await ChatClient.GetResponseAsync(message);

            var response2 = await ChatClient.GetResponseAsync<FormResult>(message);
            //Store the structured response in the formResult object for saving to the database

            formResult = response2.Result;
            }
            
            private async Task SaveDB()

            {

                //reponse is an HttpResponseMessage object that is used to store the response from the server

                //response is initialized to the result of the PostAsJsonAsync method of the Http object

                HttpResponseMessage response;                     

               

                if (formResult == null)

                {

                    status = "Please recognize the image first.";

                    return;

                }

            
            

                //The PostAsJsonAsync method sends a POST request to the specified URI (our Controller) with the specified value serialized as JSON in the request body

                //and returns a Task<HttpResponseMessage> that represents the asynchronous operation

                response = await Http.PostAsJsonAsync("FormResults", formResult);

            
            

                if (response.IsSuccessStatusCode)

                {

                    status = "Form Result Saved to Database";

                }

                else

                {

                    status = "Error Saving Form Result to Database";

                }

            }
            

        • FormPage.razor (basic display of the FormResult data in a table format)

          • Code Section
            //List of FormResult objects to store the form data from the database

            List<FormResult> FormList = new List<FormResult>();

            
            

            protected override async Task OnInitializedAsync()

            {           

                //this line fetches data from the "FormResults" endpoint (Controller) in the API project.

                //Converts the JSON response into a list of FormResult objects, and stores it in the FormList variable.

                // This allows the Blazor component to use this data for rendering the form list in the UI.

                FormList = await Http.GetFromJsonAsync<List<FormResult>>("FormResults");

            }

            
            

            private async Task HandleEdit(int id)

            {

                string requestUri = "form-edit/" + id.ToString();

                Nav.NavigateTo(requestUri);

            }

          • HTML Section
            @page "/formpage"

            @using BlazorAIforms.Shared

            @inject HttpClient Http

            @rendermode @(new InteractiveWebAssemblyRenderMode(prerender:false))

            @inject NavigationManager Nav

            
            

            @*Stream rendering allows the component to start rendering and sending parts of the response to the client as soon as they are ready,

              rather than waiting for the entire component to be rendered*@

            @attribute [StreamRendering]

            
            

            <h5>Database Application using AI Form Image Recognition </h5>

            
            

            
            

            <div class="row mt-4">

                <div class="col-6">

                    <h5 class="card-title mt-3">FORM LIST</h5>

                </div>

                <div class="col-4 offset-2">

                    <a href="/imageai" class="btn btn-info form-control">Add New Data Via Vision AI</a>

                </div>

            </div>

            
            

            <br />

            
            

            @if (FormList == null)

            {

                <div class="text-center">

                    <p><em>Loading ...</em></p>       

                </div>

            }

            else

            {

                <table class="table table-sm table-bordered table-striped table-hover">

                    <thead>

                        <tr>

                            <th>Action</th>

                            <th>Report By</th>

                            <th>Report Date</th>

                            <th>Report Phone</th>

                            <th>Attendant Name</th>

                            <th>Attendant Phone</th>

                            <th>Received Date</th>

                            <th>Entered Time</th>

                        </tr>

                    </thead>

                    <tbody>

                        @foreach (var form in FormList.OrderBy(m => m.ReportBy))

                        {

                            <tr @onclick="@(() => HandleEdit(form.Id))">

                                <td>

                                    <a class="btn btn-success" href="/form-edit/@form.Id">Edit/Delete</a>

                                </td>

                                <td>@form.ReportBy</td>

                                <td>@form.ReportDate</td>

                                <td>@form.ReportPhone</td>

                                <td>@form.AttendantName</td>

                                <td>@form.AttendantPhone</td>

                                <td>@form.ReceivedDate</td>

                                <td>@form.EnteredDate</td>                   

                            </tr>

                        }

                    </tbody>

                </table>

            }

        • FormEdit.razor (edit and delete functionality)

          • HTML Section
            @page "/form-edit/{id:int}"

            @rendermode @(new InteractiveWebAssemblyRenderMode(prerender:false))

            @using BlazorAIforms.Shared

            @inject HttpClient Http

            @inject NavigationManager Nav

          • Code Section
            [Parameter]

            public int id { get; set; }
            
            // FormResult is a class in the Shared project that represents the form data from the database

            // and is used to bind the form data to the form fields

            private FormResult  form;
            
            protected override async Task OnInitializedAsync()

            {

                // Get the form data from the database using the id parameter from the URL

                // and store it in the form object

            form = await Http.GetFromJsonAsync<FormResult>("FormResults/" + id);

            OpeningTitle = "Edit/Delete Current Form";

            
            

            ready = true;

            }
            
            private async Task HandleValidSubmit()

            {

            HttpResponseMessage response;

            //This line constructs the URI for the HTTP PUT request.

            // It concatenates the base URI "FormResults/" with the Id property of the form object.

            // This Id uniquely identifies the form data that needs to be updated on the server.

            string requestUri = "FormResults/" + form.Id;

            response = await Http.PutAsJsonAsync(requestUri, form);

            
            

            if (response.IsSuccessStatusCode)

            {

            Nav.NavigateTo("formpage");

            }

            else

            {

            error = response.ReasonPhrase;

            }

            }
            
            private async Task HandleReset()

            {

            Nav.NavigateTo("formpage");

            }

            
            

            private async Task DeleteForm()

            {

            string requestUri = "FormResults/" + form.Id;

            var response = await Http.DeleteAsync(requestUri);

            
            

            if (response.IsSuccessStatusCode)

            {

            Nav.NavigateTo("formpage");

            }

            else

            {

            error = response.ReasonPhrase;

            }

            }
            
            

  • BlazorAIformImageToDBupdateURL

    • Adds a couple of extra features to previous demo

    • On the Home.razor page we added the option to enter and analyze an Image URL (not crucial to DB implementation)

    • On the  FormEdit.razor page we added the option to Add New Data Via Vision AI (goes back to the ImageAI.razor page)

    • Added a little more error checking. If we analyze an image that is not a proper form then we don't want a null record added to the Database in the SaveDB Task
      if (formResult.ReportBy != null)

      {

          //The PostAsJsonAsync method sends a POST request to the specified URI (our Controller) with the specified value serialized as JSON in the request body

          //and returns a Task<HttpResponseMessage> that represents the asynchronous operation

          response = await Http.PostAsJsonAsync("FormResults", formResult);

      
      

          if (response.IsSuccessStatusCode)

          {

              status = "Form Result Saved to Database";

          }

          else

          {

              status = "Error Saving Form Result to Database";

          }

      }

      else

      {

          status = "Image does not conform to the Form required";

      }   

  • BlazorAIformImageToDBandTableImage/BlazorAIformImageToDBandTableImageQuickGrid

    • This latest version demostrates how to extract structured data from an image of a table that includes various fields like age, job , marital status, education, balance and housing.

      • First we add a new class in the Shared folder called Table with properties that match the table in the image

      • Then in the ImageAI.razor page we declare a new object called tableList ... a list of type Table

        • //a List of objects of type Table to store the structured response (from an image of  tabular data)

          private List<Table> tableList = new List<Table>(); 

      • We then create a new ChatMessage object with the ChatRole.User role and the message "What's in this image . Extract the data from this table into a structured including fields for age, job, marital status, education, balance, and housing."

      • Our GetResponseAsync command now uses a reference to a List of objects of type Table

        • //Added to extract the data from the image of a table

          var newmessage = new ChatMessage(ChatRole.User, "What's in this image . Extract the data from this table into a structured format including fields for age, job, marital status, education, balance, and housing.");
          
          //Added to extract the data from the image of a table ... note the use of List<Table> in the GetResponseAsync method

          newmessage.Contents.Add(new DataContent(imageBase64, "image/jpg"));

          var response3 = await ChatClient.GetResponseAsync(newmessage); 

          var response4 = await ChatClient.GetResponseAsync<List<Table>>(newmessage); 

          
          //Store the structured response in the tableList object for displaying in a table format

          tableList = response4.Result;

          
          

      • We then display the Result in a table format using a foreach loop

      • Use the image table.jpg in the images folder of the wwwroot folder to test out

  • BlazorAIformImageToDBandTableImageQGvideo

  • BlazorAIformImageToDBandTableImageQGvideoUpdate (1)

  • BlazorAIformImageToDBandTableImageQGvideoUpdTab (2)

  • BlazorAIformImageToDBandTableImageQGvideoUpdTabAudio (3)

  • BlazorAIformImageToDBandTableImageQGvideoUpdTabAudioUpd (4)

  • BlazorAIformImageToDBandTableImageQGvideoUpdTabAudioUpdWbcm (5)

  • BlazAIformImgToDBtableImgQGvidTabAudWbcmImgSimilaritySSIM(6)

    • These newest Updates adds the OpenCvSharp4 and OpenCvSharp4.runtime.win libraries to allow us to easily incorporate Computer Vision and Image processing.

    • This library allows us to connect  to cameras, open video files and save frames easily.

    • In this update we demonstrate how to easily connect to a Webcam , take a picture , save it to the wwwroot folder and analyze it. We also show how to extract frames from a video and analyze them using the Microsoft.Extensions.AI library.

    • We can apply this technology to real-world problems like insurance claim analysis. Automate report generation for vehicle damage videos, detailing the car model, year, damage specifics, and incident descriptions. This demonstrates AI’s transformative potential.

    • The first Updated version extracts frames evenly spaced across the entire video instead of the first 10 consecutive frames and then allows the user to analyze any one of them

    • The second Updated version allows the user to send all extracted frames to OpenAI for analysis at the same time. It also implements a Tabbed Menu.

    • The third Updated version allows you to record audio, save it, and convert it to text (English) using OpenAI's Whisper API. This is basic Transciption. The application will also take non-English audio and convert it to text in the language spoken (See French in wwwroot folder ... food-french.mp3) and also convert it (Translation)  to English Text.

    • The fourth Updated version takes the converted text and adds the capability to read it back in a number of user selected voices.

    • The fifth Updated version modifies the  Home.razor where we have been working on AI Video Analysis. Now we have added the ability to Record full mp4 videos via the Webcam.

    • The sixth Updated version adds the ability to compare images for Similarity using the OpenCvSharp4 library. Specifically we compare the first and last frames of any selected video using two techniques

      • The first technique calculates the mean difference for each color channel (B, G, R) and averages out these values to get a rudimentary score ... low is better high is worse

      • The second more advanced technique uses the Structural Similarity Index (SSIM), which compares images based on luminance, contrast, and structure, providing a value between -1 and 1 (where 1 means identical images).

  • BlazorMEAIchat

    • This application is a redo of several AI Chat Implementations from previous Lectures (.NET 9 Blazor WebApp Interactive Render Mode Server)

      • Recall BlazorAInet9HTMLupdate (HttpClient Technique covered in Lecture 198)

      • Recall BlazorAISemanticKernal (Microsoft Semantic Kernel Library covered in Lecture 199)

    • Here we use the Microsoft.Extensions.AI library to implement a Basic Chat page and a Conversational Chat page

      • Install the Microsoft.Extensions.AI Nuget Package

      • Install the Microsoft.Extensions.AI.OpenAI Nuget Package

      • In Program.cs add the AI Service

        • var openAiClient = new OpenAIClient("yourKey");

        • builder.Services.AddChatClient(new OpenAIChatClient(openAiClient, "gpt-4o-mini"));

      • The rest of the coding is in the ChatPage.razor and ChatConversation.razor (and assocated .css file)

  • BlazorSpeechToTextDemoJS

    • This demo showcases real-time speech-to-text capabilities in a Blazor WebAssembly Standalone app.
      There are many options for speech-to-text:
      Cloud APIs (Azure Cognitive Services, Google, etc.). Native libraries wrapped with gRPC or REST. Browser Web Speech API.
      For a Blazor app that runs in the browser, the Web Speech API is the easiest place to start:
      No server round trips for audio. No external billing. Works directly in the user’s browser. The downside: not all browsers support it (mainly Chromium based ones do: Chrome, Edge, some versions of Opera). You will add a simple IsSupported check and show a friendly message when support is missing.

      The solution is built using:

      • A small JavaScript helper speech-to-text.js that wraps the Web Speech API for browser-based voice input.

      • Real-time transcription with interim and final results.

      • Error handling and browser support checks.

      • A reusable Blazor component (SpeechToText) to drive speech recognition and integrate with your forms.

      • A simple binding pattern (bind-Value) to plug speech-to-text into any form field.

      User Benefits:
      This feature makes your app feel more modern and friendly, especially on mobile devices or for long forms. Users can dictate feedback or other text, making data entry faster and more accessible.

      Technical Highlights:
      

      • Works entirely in the browser—no server round-trips for speech recognition.

      • Easy to reuse: drop the SpeechToText component into any form.

      • Real-time interim and final transcript display.

      • Status line: listening / idle / error message.

      • Clear integration pattern for parent-child communication (e.g., clearing form fields).

      • All the magic will happen in JavaScript, but Blazor will control it via JS interop, and you will get typed callbacks on the .NET side.

      
      

      Try it out: Enter your name and feedback. You can type or use your voice. Click Clear All to reset the form and start again!

  • BlazorSpeechToTextEmailWebAppServerJS

    • This Updated demo showcases real-time speech-to-text capabilities in a Blazor Web App Interactive Render Mode Server with the additional capability of sending the transcribed text to other users via Email
      For speech-to-text we are using:
      Browser Web Speech API.

      This updated solution implements:

      • The Email Sending Library called MailKit

      • MailKit: A popular .NET library for sending and receiving emails using standard protocols like SMTP, IMAP, and POP3

      Technical Highlights:

      • Sending email in a Blazor Server application is generally easier than in a Blazor WebAssembly application due to the fundamental differences in their execution environments.

      • In Blazor Server, your application code runs entirely on the server. This means you have direct access to server-side resources and libraries, including those for sending emails.

      • Direct SMTP Client Access: You can directly use the SmtpClient class from System.Net.Mail or any other .NET email library (e.g., MailKit) to connect to an SMTP server and send emails.

      • To test out our implementation we used the Ethereal Site

      • Ethereal is a fake SMTP service, mostly aimed at Nodemailer and EmailEngine users (but not limited to). It's a completely free anti-transactional email service where messages never get delivered. Instead, you can generate a vanity email account right from Nodemailer, send an email using that account just as you would with any other SMTP provider and finally preview the sent message here as no emails are actually delivered.

      
      

      Try it out: Enter your Name, Receipent Email, Subject Heading and Body Text (Speech to Text Transcribed). You can also type your body content. Click Clear All to reset the form and start again!

In this Lecture we will

• Learn how to Scrape a Website using the Html Agility Pack NuGet package within a Blazor Web App (Interactive Render Mode Server)

  • Web scraping is a technique used in many language like C# to extract data from a website.

  • Is web scraping legal?

    That's a gray zone! There is no official answer about it, and almost every company has some web scraping program. In short, do polite crawling and don't spam a website and everything will be fine. Also some sites block scraping tools.

  • The Html Agility Pack is a .NET library that is used to parse and manipulate HTML documents. It is commonly used in .NET applications for tasks such as web scraping, reading and updating HTML files, and more. Because Blazor is a web framework within the .NET ecosystem, you can use HAP in Server-Side Blazor applications without any issues.

  • However, Html Agility Pack cannot be used in Blazor WebAssembly (Blazor WASM) directly because it relies on .NET APIs that may not be supported or could cause issues when running in the browser's WebAssembly environment. For Blazor WebAssembly, you would need to make an API call to a server-side endpoint (which could be a Blazor Server app, an ASP.NET Core app, Azure Function, etc.) that performs the HTML parsing using HAP and then returns the results to the Blazor WebAssembly application.

• Learn how to feed a Scraped website to an AI model (using Microsoft.Extensions.AI library)  and have it return specific details about the site ... including site overview ,specifics about images and more

  • The Scraped data cannot be added to the message.Contents command like it was done with images
    message.Contents.Add(new DataContent(imageBase64, "image/jpg")) in the previous lecture

  • ... so instead you must  attach (concatenate) the scraped site data (extractedData) right in the ChatMessage command
    var message = new ChatMessage(ChatRole.User, "Give me an overall idea what this site is about nicely format your response in HTML " + extractedData);

• Create a simple application that implements the stated objectives above  BlazorServerWebScrapeAI (Blazor .NET 9 Web App Interactive Render Mode Server)

  • We start off by installing the HTML Agility Pack NuGet Package . It is a .NET code library that allows you to parse  HTML pages.

  • Next we install the Microsoft NuGet Packages

    • Microsoft.Extensions.AI and Microsoft.Extensions.AI.OpenAI

    • ... back to Program.cs
      using Microsoft.Extensions.AI;

      using OpenAI;
      
      // Add AI service to the container ... OpenAI

      var openAiClient = new OpenAIClient("yourKey");

      builder.Services.AddChatClient(new OpenAIChatClient(openAiClient, "gpt-4o-mini"));
      
      builder.Services.AddHttpClient();
      

  • Let's focus on implementing our Web Scraper first

    • We create a folder called Services and create a class (service) called HtmlScraperService.cs
      
      using HtmlAgilityPack;

      using System.Net.Http;

      using System.Threading.Tasks;   

      
      

      namespace BlazorServerWebScrapeAI.Services

      {

          public class HtmlScraperService

          {

              private readonly HttpClient _httpClient;

      
      

              //Constructor
              public HtmlScraperService(HttpClient httpClient)

              {

                  _httpClient = httpClient;

              }

      
      
      //The result of the GetAsync call is an HttpResponseMessage object, which is stored in the        //response variable. The HttpResponseMessage contains details about the HTTP response, such as //the status code, headers, and the response content.
      //ReadAsStringAsync reads the HTTP response body (content) as a string and ensures it is fully //converted into a string format which is useful for processing the HTML
      

              public async Task<HtmlDocument> LoadHtmlFromUrlAsync(string url)

              {

                  var response = await _httpClient.GetAsync(url);

                  var pageContent = await response.Content.ReadAsStringAsync();

      
      //These commands are part of the HtmlAgilityPack library
      //Here we create a new instance of the HtmlDocument, which is used to parse and manipulate        //HTML content in a structured way
      //The LoadHtml method takes a string containing raw HTML and parses it into the HtmlDocument object. After this step the object will contain a tree-like structure of the HTML allowing you to query or manipulate specific elements (eg. tags, attribures, text...called ParsedText)
      

                  var htmlDocument = new HtmlDocument();

                  htmlDocument.LoadHtml(pageContent);

      
      

                  return htmlDocument;

              }

          }

      }

    • Now we go back to Program.cs
      //declare the HtmlScraperService for future dependency injection

      builder.Services.AddScoped<HtmlScraperService>();   

      
      

  • The rest of our code will be on the Home.razor page

    • First some variable declarations in our code section
      
      private string targetUrl = "";

      
      

      private string extractedData = "";

      private MarkupString ExtractedDataMarkup => new MarkupString(extractedData); // Convert to MarkupString for HTML rendering

      // ... this is a second method you can use

      // instead of the (casting) technique used in the HTML above

      
      

      private bool isLoading = false; //used to disable and enable buttons

      
      

      private string? responseMessage;

      private string? statusScrape;

      private string? statusAI;

      
      

      private string? imageBase64;

      private long imageFileSize;

      private string? imageUrl;

      private bool isImageURL = false;
      
      //This is the list of image URLs extracted from the scraped data

      //used in the Task ServiceScrapeAndProcess()

      private List<string> imageUrls = new List<string>();
      

    • Now we work on our Web Scraper Task and the associated ExtractImageUrls method (used as a proof of concept to extract every image on the webpage ...  will be fully utilized  in the Supplementary Demo  BlazorServerWebScrapeAIupdated where we will allow the user to  analyze  any selected image)
      
      protected async Task ServiceScrapeAndProcess()

      {

          if (string.IsNullOrEmpty(targetUrl))

          {

              statusScrape = "Please enter a URL to scrape.";

              return;

          }

          else

          {

              isLoading = true;

              responseMessage = "";

              statusAI = "";

              statusScrape = "Processing...";

         

              //call the HtmlScraperService (in Service folder) to scrape the URL

              var ScraperResponse = await HtmlScraperService.LoadHtmlFromUrlAsync(targetUrl);

             

              //Extract data from the response so we can display it on our HTML page

              extractedData = ScraperResponse.ParsedText;   

         

              // Extract image URLs using regular expressions

              // These will be added to the end of the scraped data (extractedData) so they can be displayed

              imageUrls = ExtractImageUrls(ScraperResponse.ParsedText);

             
             foreach (var url in imageUrls)

              {

                  extractedData += " <br/><img src=' " + url + " ' width='100' height='100' /> ";

              }

      
      

              imageUrl = imageUrls.FirstOrDefault(); // Get the first image URL to display and Analyze (ProcessUrLImage method below)

      
      

              StateHasChanged(); // Refresh the UI to show the new data

      
      

              isLoading = false;

              statusScrape = "Done";

          }   

      }
      
      //Generated with help from GitHub Copilot (see ExtractImageUrls-codesnippet)
      ... live demo of using GitHub Copilot with  query "How do I extract all the images from the scraped site and store them into a list "
      
      private List<string> ExtractImageUrls(string htmlContent)

      {

          //This method uses a regular expression to extract image URLs from the HTML content and returns them as a list of strings.

          //The regular expression looks for <img> tags and captures the value of the src attribute.

          //The regex pattern "<img[^>]+?src=[\"'](?<url>.*?)[\"']" is used to match the <img> tags and extract the URLs.

          //The pattern uses a named group "url" to capture the URL value.

          //The RegexOptions.IgnoreCase option is used to make the regex case-insensitive.

      
      

          var urls = new List<string>();

          var regex = new Regex("<img[^>]+?src=[\"'](?<url>.*?)[\"']", RegexOptions.IgnoreCase);

          var matches = regex.Matches(htmlContent);

          foreach (Match match in matches)

          {

              urls.Add(match.Groups["url"].Value);

          }

          return urls;

      }
      

    • Next we add some HTML so that we can test out the first part of our application ... Web Scraping

      • Some initial declarations ... some we need now some we will use later
        
        @page "/"

        
        

        @using HtmlAgilityPack

        @using System.Net.Http

        @using System.Text.Json

        @using System.Threading.Tasks

        @using System.Text

        
        

        @using BlazorServerWebScrapeAI.Services

        
        

        @rendermode InteractiveServer

        
        

        @*Implements HTML Agility Pack*@

        @inject HtmlScraperService HtmlScraperService   

        
        

        @using Microsoft.Extensions.AI;

        @inject IChatClient ChatClient

        
        

        @*Required for Regex commands in ExtractImageUrls method below*@

        @using System.Text.RegularExpressions
        

      • Now the UI  (see HTMLcodesnippet)  ... some of the HTML we need now, and  some we will use later ... before pasting in the snippet ... create a folder called images in the wwwroot folder and copy over loading.gif (in Resources)
        
        <PageTitle>Home</PageTitle>

        
        

        <h3>Web Scraping AI ... Implementing HTML Agility Pack and Microsoft.Extensions.AI</h3>

        
        

        <br/>

        
        

        <h5 style="text-align:center">

            <b>Generate Responses using the OpenAI API</b>

        </h5>

        
        

        <div class="row">

            <textarea @bind="targetUrl" rows="2" placeholder="Enter Website Address Here"></textarea>

        </div>

        
        

        <div style="text-align:center">

            <button @onclick="ServiceScrapeAndProcess" class="btn btn-primary mt-3" disabled="@isLoading">Web Scrape URL</button>

            <button @onclick="@( () => ProcessUrLImage(imageUrl))" class="btn btn-primary mt-3" disabled="@isLoading">First Image AI Analysis</button>

            <button @onclick="ScrapeAndProcessMSextensions" class="btn btn-primary mt-3" disabled="@isLoading">Generate Webpage AI Analysis</button>

        
        

            @if (statusAI == "Processing...")

            {

                <img src="images/loading.gif" width="40" height="40" style="margin-left: 10px;" />

                <em>@statusAI</em>

            }

            @if (isImageURL)

            {       

                <br />

                <img src="@imageUrl" alt="Selected Image" style="max-width: 300px; max-height: 300px;" />

                <strong>Selected Image Original Size:@imageFileSize</strong>

                <i>bytes</i>

            }

        
        

        </div>

        
        

        <div class="alert alert-info mt-3">

            <p>

                AI Analysis of Scraped Data using MS Extensions:@statusAI

                <br />

                @((MarkupString)responseMessage)

            </p>

        </div>

        
        

        <div class="alert alert-light mt-3" style="width: 800px;">

            <p>

                Scraped Data:@statusScrape

                <br/>

                @ExtractedDataMarkup

            </p>

        </div>

      • OK time to test out our Web Scraper (comment out the two buttons that reference ProcessUrLImage and ScrapeAndProcessMSextensions ... for now)

    • Now that we have our extractedData and imageUrl (first image only) we are going to feed these into our OpenAI API model via our Microsoft.Extensions.AI library of commands.

      • First we will analyze the image stored in imageUrl ... leveraging our knowledge of image (Vision) analysis from the previous lecture. (see ProcessImageCodeSnippet.txt)
        
        private async Task ProcessUrLImage( string imageUrl)

        {

            isImageURL = false;

            imageBase64 = null;

            isLoading = true;            

        
        

            if (imageUrl == null)

            {

                responseMessage = "Need a valid Image URL ...Try Scraping first";

                isLoading = false;

                return;

            }

        
        

            //check if valid URL

            if (!Uri.IsWellFormedUriString(imageUrl, UriKind.Absolute))

            {

                responseMessage = "Not a valid Image URL.";

                isLoading = false;

                return;

            }

        
        

            isImageURL = true;

            statusAI = "Processing...";

            responseMessage = null;

                   

            var message = new ChatMessage(ChatRole.User, "What's in this image");

        
        

            //Use HttpClient to download the image from the URL and then convert it to a base64 string for processing

            var httpClient = new HttpClient();

        
        //This method sends an HTTP GET request to the specified imageUrl (a string containing the URL of an image). It retrieves the response body as a byte array. This is useful for downloading binary data, such as images, files, or other non-text content.

            var imageBytes = await httpClient.GetByteArrayAsync(imageUrl);
        
        

            var base64Image = Convert.ToBase64String(imageBytes);

            var imageData = $"data:image/jpg;base64,{base64Image}";

        
        

            //determine the image file size

            imageFileSize = imageBytes.Length;

        
        

            message.Contents.Add(new DataContent(imageData, "image/jpg"));

            var response1 = await ChatClient.GetResponseAsync(message);

           

            responseMessage = response1.Text;

        
        

            statusAI = "Done";

            isLoading = false;

           

        }

      • ... and finally we Analyze the entire scraped site
        
        private async Task ScrapeAndProcessMSextensions()

        {

            if (!string.IsNullOrEmpty(extractedData))

            {

                isLoading = true;

                isImageURL = false;

                responseMessage = "";

                statusAI = "Processing...";

        
        

                //Create a new ChatMessage object with the ChatRole.User role and an appropriate message

                //Call the GetResponseAsync method of the ChatClient object with the message as the parameter

                //and store the result in the response variable

                //Set the responseMessage variable to the response's Text property           

        
        

                //Change prompt to try and get a more detailed or relevant response

                //*****************************************************************

        
        

                //var message = new ChatMessage(ChatRole.User, "Tell me what's on this webpage and format the response in HTML " + extractedData );

                //var message = new ChatMessage(ChatRole.User, "Tell me how many pictures of bread there are on this site " + extractedData);

                //var message = new ChatMessage(ChatRole.User, "Tell me how many times the word Sourdough appears on the site " + extractedData);

                //var message = new ChatMessage(ChatRole.User, "Give me an overall idea what this site is about " + extractedData);

                //var message = new ChatMessage(ChatRole.User, "Count the number of images on the site by counting the number of img tags " + extractedData);       

               

                var message = new ChatMessage(ChatRole.User, "Give me an overall idea what this site is about nicely format your response in HTML " + extractedData);

               

                // ***** I Tried here to add the Scraped data to the message but not allowed ... so I fed scraped site data above right in the ChatMessage *********

                //message.Contents.Add(new DataContent(imageBase64, "image/jpg"));

        
        

                var response1 = await ChatClient.GetResponseAsync(message);

                responseMessage = response1.Text;

        
        

                isLoading = false;

                statusAI = "Done";           

            }

            else

            {

                responseMessage = "Please Scrape the URL first ";

            }

        }

  • Test out application ...

    • ... Some sites block users from web scraping .

• Supplementary Demos

  • BlazorServerWebScrapeAIupdated

    ... Updated with Complete Site Image Analysis and enhanced error checking

    • HTML updates

      • <button @onclick="@( () => ProcessUrLImage(imageUrl,1))" class="btn btn-primary mt-3" disabled="@isLoading">First Image AI Analysis</button>

      • @*This handles the hard coded capture of the first image on the site*@

        @if (isImageURL1)

        {       

            <br />

            <img src="@imageUrl" alt="Selected Image" style="max-width: 300px; max-height: 300px;" />

            <br/>

            <strong>Selected Image Original Size:@imageFileSize</strong>

            <i>bytes</i>

        }

        @*This handles any image clicked on from the entire list displayed in the div section below*@

        else if (isImageURL2)

        {

            <br />

            <img src="@newImageURL" alt="Selected Image" style="max-width: 300px; max-height: 300px;" />

            <br/>

            <strong>Selected Image Original Size:@imageFileSize</strong>

            <i>bytes</i>

        }

      • @*This displays all the images on the site that was scraped ...

            Some may be placeholder images or not relevant to the site ... but they are all displayed

            They are all clickable and will call the ProcessUrLImage method to analyze the image

        *@

        <div class="alert alert-light">

            @if (imageUrls.Count > 0)

            {           

                <span>Click on Images below for AI Analysis</span>

                <br/>

        
        

                @foreach (var img in imageUrls) 

                { 

                    if (!string.IsNullOrEmpty(img)) 

                    { 

                        <span @onclick="@(() => ProcessUrLImage(img,2))" style="cursor:pointer;"> 

                            <img src="@img" height="50" width="50" style="padding:5px" /> 

                        </span> 

                    } 

                }

            }

        </div>

    • Code Update

      • Note the use of try catch  constructs in the ServiceScrapeAndProcess Task ... here we are worried about a site whose url is well formed BUT  may not actually exist .
        
        try

        {

            // Call the HtmlScraperService to scrape the URL

            var ScraperResponse = await HtmlScraperService.LoadHtmlFromUrlAsync(targetUrl);

        
        

            // Extract data from the response so we can display it on our HTML page

            extractedData = ScraperResponse.ParsedText;

        
        

            // Extract image URLs using regular expressions

            imageUrls = ExtractImageUrls(ScraperResponse.ParsedText);

        
        

            foreach (var url in imageUrls)

            {

                extractedData += "<br/><img src='" + url + "' width='100' height='100' />";

            }

        
        

            imageUrl = imageUrls.FirstOrDefault(); // Get the first image URL to display and analyze

        
        

            statusScrape = "Done";

        }

        catch (Exception ex)

        {

            // Handle any errors that occur during the scraping process

            statusScrape = "An error occurred while scraping the URL...." + ex.Message;

            extractedData = "";

            responseMessage = ""; // Optionally, you can log the exception message

            statusAI = "";

            imageUrls.Clear(); // Clear the list of image URLs

            imageUrl = null; // Clear the image URL

            isImageURL1 = false;

            isImageURL2 = false;

        }

        finally

        {

            isLoading = false;

            StateHasChanged(); // Refresh the UI to show the new data

        }

      • ProcessUrLImage Task now has two parameters
        private async Task ProcessUrLImage(string imageUrl,int type)

        { .....
          if (type==1) //First Image Analysis button was clicked

        {

            isImageURL1 = true;

        }

        else if (type == 2) //Image clicked on from the list of images displayed ... store the URL for analysis in a different variable

        {

            isImageURL2 = true;

            newImageURL = imageUrl;

        }
        ..... }

      • Note the use of try catch construct in the ScrapeAndProcessMSextensions Task ...here we may run into an issue with the Openai API due to a token limit error ... HTTP 429 (tokens: rate_limit_exceeded) Request too large for gpt-4o-mini ...
        
        try

        {

            var response1 = await ChatClient.GetResponseAsync(message);

            responseMessage = response1.Text;

        }

        catch (Exception ex)

        {

            // Handle the exception and provide feedback to the user

            responseMessage = $"An error occurred while processing the AI response: {ex.Message}";

            statusAI = "Error";

        }

        finally

        {

            isLoading = false;

            statusAI = "Done";

            StateHasChanged(); // Refresh the UI to show the new data

        }

  • BlazorServerWebScrapeAIupdatedOtherElements 

    • Instead of just pulling out images this Updated Version illustrates how to extract any desired part of a site.

      • For this demo app we use the site  which contains a table listing selected cities in the world by nominal gross domestic product (GDP). The table has four columns ... City, Country, GDP, Population

    • We must take a look at the site before hand via the DevTools (F12) and specifically look through the page Elements to get a good idea how the site is structured (html tags) ... we will use this knowledge later with our query selectors when we extract objects ... like city, population etc

    • Next we create a class to mirror what objects we want to extract ... CityByGdpData
      public class CityByGdpData

      {       

          public string City { get; set; }

          public string Country { get; set; }

          public int GdpByBillionUsd { get; set; }

          public int Population { get; set; }           

      }

    • Then we create a new Task/Method (public List GetCitiesByGdpDatas(string url)) within our current Service to scrape the site and use Query Selector(s) to pinpoint where the desired objects we want to extract are located. Note: to use the Query Selector commands you must additionally install the HtmlAgilityPack.CssSelectors.NetCore NuGet Package

      • //This Task below is for scraping the GDP data from the Wikipedia page

        //Before using the QuerySelector(s) you must take a deep dive into the HTML structure of the page

        //You can do this by right clicking on the page and selecting "Inspect" in your browser.

        //This should open the developer tools and show you the HTML structure of the page ... from there the fun begins

        //Narrow in on the table you want to scrape and check out the HTML tags and classes

        public List<CityByGdpData> GetCitiesByGdpDatas(string url)

        {

             //Here is declare the list of CityByGdpData objects ... which will be used to store the scraped data (specific to the GDP data)

             var citiesByGdpDataList = new List<CityByGdpData>();

        
        

             //Load the HTML document from the specified URL using HtmlAgilityPack load command and then query the document for the table rows

             //... because the information we want from the Wikipedia page is in a table format

             //The QuerySelectorAll method is used to select all the table rows (tr) in the HTML document

            var htmlDoc = new HtmlWeb().Load(url);

             var rows = htmlDoc.QuerySelectorAll("table tr");

        
        

             //The first three rows are headers and we want to skip them ... the actual header for the table is obvious

             //but it appears that the two boxed sections on the page are also part of the row structure

             //The way I discovered this was to add a break here and hover over the rows variable above and observe the contents of rows[0] rows[1] rows[2]

             for (int i=3;i<rows.Count;i++)

             {

                var cells = rows[i].QuerySelectorAll("td"); //here I grab the table cells (td) in the row (city,country,gdp,population)

        
        

                 if (cells.Count==4) //parse data only if there are 4 cells in a row for city,country,gdp,population

                 {

                     //Here I am using the ReplaceNonBreakingSpace method to replace non-breaking spaces with regular spaces

                     //The ReplaceNonBreakingSpace method is defined below and is used to clean up the data

                     //.. this is only discovered after exectuting the code and seeing the results

                    var city = ReplaceNonBreakingSpace(cells[0].InnerText.Trim());

                     var country = ReplaceNonBreakingSpace(cells[1].InnerText.Trim());

        
        

                     //Here I am using the ExtractNumericPart method to extract the numeric part of the GDP and population data

                     //The ExtractNumericPart method is defined below and is used to clean up the data

                     //.. this is only discovered after exectuting the code and seeing the results

                     //We use TryParse to convert the string to an integer and handle any parsing errors

                    int.TryParse(ExtractNumericPart(cells[2].InnerText.Trim()), out int gdpByBillionUsd);                   

                     int.TryParse(ExtractNumericPart(cells[3].InnerText.Trim()),out int population);

        
        

                     //Here I am creating a new CityByGdpData object and adding it to the list (citiesByGdpDataList)

                     //The CityByGdpData class is defined in the Models folder and contains the properties for the city, country, GDP, and population

                     //The CityByGdpData class is used to store the scraped data in a structured way

                    citiesByGdpDataList.Add(new CityByGdpData

                     {

                         City = city,

                         Country = country,

                         GdpByBillionUsd = gdpByBillionUsd,

                         Population = population                         

                     });

                 }

        
        

                 //The ReplaceNonBreakingSpace method is used to replace non-breaking spaces with regular spaces

                string ReplaceNonBreakingSpace(string input)

                 {

                     return input.Replace("&#160;", " ");//non-breaking space characters (encoded as &#160; in HTML)

                 }

        
        

                 //The ExtractNumericPart method is used to extract the numeric part of the GDP and population data

                string ExtractNumericPart(string input)

                 {

                     //^: Ensures the match starts at the beginning of the string.

                     //[\d,.]: Matches any digit(\d), a comma(,), or a period(.).

                     //+ Ensures one or more of the preceding characters(digits, commas, or periods) are matched.

                     var match = Regex.Match(input, @"^[\d,.]+");

                     if (match.Success)

                     {

                         return match.Value.Replace(",", string.Empty).Replace(".", string.Empty);

                     }

        
        

                     return null;

                 }

             }

        
        

             //Sort the list of CityByGdpData objects by country name and return the sorted list

             return citiesByGdpDataList.OrderBy(x => x.Country).ToList(); 

        }

    • ... and finally we extract and process (in this case) the GDP table data from Wikipedia on the Home.razor page.

      • HTML Section
        @using Microsoft.AspNetCore.Components.QuickGrid
        
        <h5 style="text-align:center">

            <b>Advanced Scraping ... </b>

        </h5>

        
        

        <div class="row">

            <textarea @bind="newtargetUrl" rows="2" placeholder="Enter Website Address Here"></textarea>

        </div>

        
        

        <button @onclick="NewServiceScrapeAndProcess" class="btn btn-primary mt-3" disabled="@isLoading">Web Scrape URL ... Other Data Points (Objects)</button>

        <button @onclick="@( () => GetCityWithTheHighestPopulation(cityByGdpData))" class="btn btn-primary mt-3" disabled="@isLoading">Highest Population </button>

        <br/>
        
        <div class="alert alert-light mt-3" style="width: 800px;">

            <p>

                Scraped Data:@statusScrape ... @HighestPopulationCity

                <br/>

                @ExtractedDataMarkup

            </p>

        </div>
        
        @if (cityByGdpData.Count >0)

        {       

            <QuickGrid Class="table table-sm table-bordered table-striped table-hover" Virtualize="true" Items="@CityQ"  Pagination="@pagination">

        
        

                <PropertyColumn Property="@(p => p.City)" Title="City" Sortable="true" />

                <PropertyColumn Property="@(p => p.Country)" Title="Country" Sortable="true" />

                <PropertyColumn Property="@(p => p.GdpByBillionUsd.ToString("C", System.Globalization.CultureInfo.CurrentCulture))" Title="GDP" Sortable="true" />

                <PropertyColumn Property="@(p => p.Population)" Title="Population" Sortable="true" />

        
        

            </QuickGrid>

        
        

            <Paginator State="@pagination"/>

        }
        
        //Note the implementation of the QuickGrid with column sorting and page due to the size of the data scraped

      • Code Section

        //Wikipedia URL

        private string newtargetUrl = "";
        
        //Wikipedia URL ... List for the newtargetURL

        private List<CityByGdpData> cityByGdpData = new List<CityByGdpData>();
        
        //Required for the use of QuickGrid

        IQueryable<CityByGdpData>? CityQ;

        PaginationState pagination = new PaginationState { ItemsPerPage = 10 };

        
        

        //used in the GetCityWithTheHighestPopulation method below

        private string HighestPopulationCity = string.Empty;   

        
        //This method is called when the user clicks the "Web Scrape URL ... Other Data Points (Objects)" button

        protected async Task NewServiceScrapeAndProcess()

        {

            if (string.IsNullOrEmpty(newtargetUrl))

            {

                statusScrape = "Please enter a URL to scrape.";

                extractedData = "";

                responseMessage = "";

                statusAI = "";

                imageUrls.Clear(); // Clear the list of image URLs

                imageUrl = null; // Clear the image URL

                isImageURL1 = false;

                isImageURL2 = false;

        
        

                return;

            }

            else if (!Uri.IsWellFormedUriString(newtargetUrl, UriKind.Absolute))

            {

                statusScrape = "Not a valid URL.";

                extractedData = "";

                responseMessage = "";

                statusAI = "";

                imageUrls.Clear(); // Clear the list of image URLs

                imageUrl = null; // Clear the image URL

                isImageURL1 = false;

                isImageURL2 = false;

                return;

            }   

            else

            {

                isLoading = true;

        
        

                responseMessage = "";

                extractedData = ""; // Clear the extracted data

        
        

                imageUrls.Clear(); // Clear the list of image URLs

                imageUrl = null; // Clear the image URL

                isImageURL1 = false;

                isImageURL2 = false;

        
        

                statusAI = "";

                statusScrape = "Processing...";

           

                try

                {

                    //This Method call within the Service was used to extract the GDP data from the Wikipedia page

                    //it returns a List of CityByGdpData objects

                    cityByGdpData = HtmlScraperService.GetCitiesByGdpDatas(newtargetUrl);

                    CityQ = cityByGdpData.AsQueryable(); // Convert the list to an IQueryable for QuickGrid   

        
        

                    statusScrape = "Done";

                }

                catch (Exception ex)

                {

                    // Handle any errors that occur during the scraping process

                    statusScrape = "An error occurred while scraping the URL...." + ex.Message;

                    extractedData = "";

                    responseMessage = ""; // Optionally, you can log the exception message

                    statusAI = "";

                    imageUrls.Clear(); // Clear the list of image URLs

                    imageUrl = null; // Clear the image URL

                    isImageURL1 = false;

                    isImageURL2 = false;

                }

                finally

                {

                    isLoading = false;

                    StateHasChanged(); // Refresh the UI to show the new data

                }

        
        

            }

        }

        
        

        public void GetCityWithTheHighestPopulation(List<CityByGdpData> data) 

        { 

           if (data == null || data.Count == 0) 

           { 

               return; // Return if the list is empty or null 

           } 

        
        

            // Filter city with highest population using LINQ and lambda expression

           var cityWithHighestPopulation = data           

               .OrderByDescending(city => city.Population) 

               .FirstOrDefault(); 

        
        

           // Store the result in the HighestPopulationCity variable 

           HighestPopulationCity = cityWithHighestPopulation != null 

               ? "City with Highest Population: " + cityWithHighestPopulation.City + " Population: " + cityWithHighestPopulation.Population.ToString("N0")

               : "No data available for cities in the United States."; 

        }

      
      

  • BlazorScrapingAndDB/ tabbed menu layout BlazorScrapingAndDBupdated

    • Further updated  Web Scraping demo application  now with a Database implementation

      • These New Versions illustrates how to extract any desired part of a site and feed the results into a DB with full CRUD Ability.

      • We will be again be using the Wikipedia page "List of cities by GDP"

        • Recall we must take a look at the site before hand via the DevTools (F12) (right click on table and choose inspect) and specifically look through the page Elements to get a good idea how the site is structured (html tags) ... we will use this knowledge later with our query selectors when we extract objects ... like city, population etc

      • This version is a .NET 9 Blazor WebApp using Interactive Render Mode WebAssembly

        • Recall that this template creates a Server Project and Client (WebAssembly) project. We must manually add a new Class Library project which we will call BlazorScrapingAIandDB.Shared

      • First we create a class to mirror what objects we want to extract ... CityByGdpData with the addition of an Id property so we can implement a DB ... See the Shared project
        public  class CityByGdpData

        {

            public int Id { get; set; }

            public string City { get; set; }

            public string Country { get; set; }

            public int GdpByBillionUsd { get; set; }

            public int Population { get; set; }

        }

      • Then we create a new Task/Method (public List GetCitiesByGdpDatas(string url)) within our current Service to scrape the site and use the SelectNodes command instead of Query Selector(s) to pinpoint where the desired objects we want to extract are located (to use Query Selectors you must additionally install the HtmlAgilityPack.CssSelectors.NetCore NuGet Package
        public async Task<List<CityByGdpData>> GetCitiesByGdpDatas(string url)

        {

            //Here we declare a new object ... which will be a List of type CityByGdpData class

            //we will use it to store the data we scrape from the website

            var citiesByGdpDataList = new List<CityByGdpData>();

        
        

            //Here we load the HTML document from the URL

            var htmlDoc = new HtmlWeb().Load(url);

        
        

            // Replace QuerySelectorAll with SelectNodes

            //var rows = htmlDoc.QuerySelectorAll("table tr");           

           var rows = htmlDoc.DocumentNode.SelectNodes("//table//tr");

        
        

            if (rows != null)

            {

                for (int i = 3; i < rows.Count; i++)

                {

                    //var cells = rows[i].QuerySelectorAll("td");  ... old technique

                    //here a grab the table cells (td) in the row (city,country,gdp,population)//

                    var cells = rows[i].SelectNodes("td");

        
        

                    if (cells != null && cells.Count == 4)

                    {

                        var city = ReplaceNonBreakingSpace(cells[0].InnerText.Trim());

                        var country = ReplaceNonBreakingSpace(cells[1].InnerText.Trim());

        
        

                        int.TryParse(ExtractNumericPart(cells[2].InnerText.Trim()), out int gdpByBillionUsd);

                        int.TryParse(ExtractNumericPart(cells[3].InnerText.Trim()), out int population);

                       

                        citiesByGdpDataList.Add(new CityByGdpData

                        {

                            City = city,

                            Country = country,

                            GdpByBillionUsd = gdpByBillionUsd,

                            Population = population

                        });

                    }

                }

            }

        
        

            return citiesByGdpDataList.OrderBy(x => x.Country).ToList();

        
        

        
        

            // Helper methods to clean up the data

            // These are local functions ... methods that are only used in this class

            // and are not accessible outside of this class

        
        

            string ReplaceNonBreakingSpace(string input)

            {

                return input.Replace("&#160;", " ");

            }

        
        

            string ExtractNumericPart(string input)

            {

                var match = Regex.Match(input, @"^[\d,.]+");

                if (match.Success)

                {

                    return match.Value.Replace(",", string.Empty).Replace(".", string.Empty);

                }

        
        

                return null;

            }

        }

      • Here are the key updates to the Program.cs file in the Server Project
        // Add services to the container... InteractiveServerComponents ... after the fact

        builder.Services.AddRazorComponents()

            .AddInteractiveWebAssemblyComponents()

            .AddInteractiveServerComponents();

        
        

        //added to implement HttpClient for Service HtmlScraperService 

        builder.Services.AddHttpClient();

        
        

        //added so that the app can use the database via the controller

        builder.Services.AddControllers();

        
        

        //inject the HtmlScraperService

        builder.Services.AddScoped<HtmlScraperService>();
        
        //added after the fact for the controller to work

        app.MapControllers();

      • Database Implementation Key Teaching Points

        • We added a class library project which holds our CityByGdpData class ... make sure to add a Reference to this Shared  project in the Client and Server Projects

        • In the Server Project we added a folder called Controllers and then right clicked and add an API Controller with actions using Entity Framework. We also updated the appsettings.json (Server Project) and changed name of the Database = BlazorScrapingAIandDB

        • Then we performed the migration and database update ... add-migration initial ... update-database

        • Check out the database in the Server Object Explorer and add a few records by hand

        • Before running the app we updated the Program.cs file (Server Project) so that it is aware of our use of a Controller api ... builder.Services.AddControllers() ... app.MapControllers()

        • We can now start up the app and enter the controller name (CityByGdpDatas) to view the rough Json text file output

        • .. All the prep is now done so we focus on the UI and coding for the remaining pages ...

      • Home.razor page

        • HTML code
          <div class="row">

               <textarea @bind="newtargetUrl" rows="2" placeholder="Enter Website Address Here"></textarea>

          </div>

             

          <button @onclick="@( () => NewServiceScrapeAndProcess(1))" class="btn btn-primary mt-3" disabled="@isLoading">Web Scrape URL and Save ALL to DB</button>

          <button @onclick="@( () => NewServiceScrapeAndProcess(0))" class="btn btn-primary mt-3" disabled="@isLoading">JUST ... Web Scrape URL </button>   

          <button @onclick="@( () => GetCityWithTheHighestPopulation(cityByGdpData))" class="btn btn-primary mt-3" disabled="@isLoading">Highest Population </button>

          <br/>
          
          @if (cityByGdpData.Count >0)

          {       

              <QuickGrid Class="table table-sm table-bordered table-striped table-hover" Virtualize="true" Items="@CityQ"  Pagination="@pagination">

          
          

                  <PropertyColumn Property="@(p => p.City)" Title="City" Sortable="true" />

                  <PropertyColumn Property="@(p => p.Country)" Title="Country" Sortable="true" />

                  <PropertyColumn Property="@(p => p.GdpByBillionUsd.ToString("C", System.Globalization.CultureInfo.CurrentCulture))" Title="GDP" Sortable="true" />

                  <PropertyColumn Property="@(p => p.Population)" Title="Population" Sortable="true" />

          
          

              </QuickGrid>

          
          

              <Paginator State="@pagination"/>

          }

        • Code Section
          
          protected override void OnInitialized()

          {

              // Set the base address for the API request in the NewServiceScrapeAndProcess Task below

              // var response = await Http.PostAsJsonAsync("CityByGdpDatas", city);

              //... solved issue with the "API URI not set correctly" error message in this Server Project

              //This was not required in the Blazor WebAssembly project razor pages

              Http.BaseAddress = new Uri(Nav.BaseUri);

          }
          //This method is called when the user clicks either the "Web Scrape URL and Save to DB" ... OR ..."JUST scrape URL" button

          //... the scrapeFlag is used to determine if we want to save the data to the DB or not

          protected async Task NewServiceScrapeAndProcess(int scrapeFlag)

          {

              if (string.IsNullOrEmpty(newtargetUrl))

              {

                  statusScrape = "Please enter a URL to scrape.";

                  return;

              }

              else if (!Uri.IsWellFormedUriString(newtargetUrl, UriKind.Absolute))

              {

                  statusScrape = "Not a valid URL.";

                  return;

              }   

              else

              {

                  isLoading = true;

                  statusScrape = "Processing...";

          
          

                  //----------- Updated Technique to handle possibly Scraper Errors -----------

                  try

                  {         

          
          

                      //This Method call within the Service was used to extract the GDP data from the Wikipedia page

                      //it returns a List of CityByGdpData objects

                      cityByGdpData = await HtmlScraperService.GetCitiesByGdpDatas(newtargetUrl);

                      CityQ = cityByGdpData.AsQueryable(); // Convert the list to an IQueryable for QuickGrid

          
          

                      //We only want to save to the Database once! it is not necessary to save the same data multiple times

                      if (scrapeFlag==1)

                      {

                          foreach(var city in cityByGdpData)

                          {

                              //save to database

                              if(city.Id==0)

                               {

                                  //This line would not work until the API URI was set correctly in the OnInitialized method above

                                   var response = await Http.PostAsJsonAsync("CityByGdpDatas", city);                       

                               }                     

                          }

                      }               

          
          

                      statusScrape = "Done";

                  }

                  catch (Exception ex)

                  {

                      // Handle any errors that occur during the scraping process

                      statusScrape = "An error occurred while scraping the URL...." + ex.Message;               

                  }

                  finally

                  {

                      isLoading = false;

                      StateHasChanged(); // Refresh the UI to show the new data

                  }

              }

          }

      • In the Client project we have the WikiPage.razor (used to Display the current contents of the database with the option to Edit/Delete) and the  GdpEdit.razor page which is used to actually edit and delete records.

      • WikiPage.razor

        • HTML section
          @page "/wikipage"

          @using BlazorScrapingAIandDB.Shared

          @inject HttpClient Http

          @rendermode @(new InteractiveWebAssemblyRenderMode(prerender:false))

          @inject NavigationManager Nav

          @using Microsoft.AspNetCore.Components.QuickGrid
          
          @if (cityByGdpData.Count > 0)

          {

              <QuickGrid Class="table table-sm table-bordered table-striped table-hover" Virtualize="true" Items="@CityQ" Pagination="@pagination">

          
          

                  <TemplateColumn Context="p" Title="Action">

                      <a class="btn btn-warning" href="/gdpedit/@p.Id">Edit/Delete</a>

                  </TemplateColumn>

                 

                  <PropertyColumn Property="@(p => p.City)" Title="City" Sortable="true" />

                  <PropertyColumn Property="@(p => p.Country)" Title="Country" Sortable="true" />

                  <PropertyColumn Property="@(p => p.GdpByBillionUsd.ToString("C", System.Globalization.CultureInfo.CurrentCulture))" Title="GDP" Sortable="true" />

                  <PropertyColumn Property="@(p => p.Population)" Title="Population" Sortable="true" />

          
          

              </QuickGrid>

          
          

              <Paginator State="@pagination" />

          }

        • Code Section
          protected override async Task OnInitializedAsync()

          {   

              cityByGdpData = await Http.GetFromJsonAsync<List<CityByGdpData>>("CityByGdpDatas");

              CityQ = cityByGdpData.AsQueryable();

          }

      • GdpEdit.razor

        • HTML Section
          @page "/gdpedit"

          @page "/gdpedit/{id:int}"
          ...
          <EditForm Model="@gdp" OnValidSubmit="@HandleValidSubmit" @onreset="HandleReset">
          
          <div class="text-center">

          <button type="submit" class="btn btn-primary" style="margin:2px;">Save</button>

              @if (id != 0) // if we are editing an existing database record we show the delete button

          {

          <button class="btn btn-danger " title="Delete task" style="margin:2px;" @onclick="@DeleteForm">

          <i class="bi bi-fire"></i>

          Delete

          </button>

          }

          <button type="reset" class="btn btn-secondary" style="margin:2px;">Cancel</button>

          </div>

        • Code Section
          [Parameter]

          public int id { get; set; }
          

          private CityByGdpData gdp;
          
          protected override async Task OnInitializedAsync()

          {

          // Get the form data from the database using the id parameter from the URL

          // and store it in the form object

          
          

          // If the id is 0, create a new form object

          // If the id is not 0, get the form data from the database using the id parameter
          if (id==0)

          {

          gdp = new CityByGdpData();

              OpeningTitle = "Create New Form";

          }
          else

          {

          gdp = await Http.GetFromJsonAsync<CityByGdpData>("CityByGdpDatas/" + id);

          OpeningTitle = "Edit/Delete Current Form";

          }
          }

    • Suggested Exercises: (WebScrapingExercise.pdf)

      • Scrape the 100 best American movies list from the Wikipedia site

        • 
          

        • BlazorScrapingAndDBupdatedMovies

      • Scrape a fictional E-Commerce Site and extract the product name, price, image URL and product URL

        • 
          

        • BlazorScrapingAndDBupdateNextEcom (1 page scrape)

        • BlazorScrapingAndDBupdatedNextEcomFull (full site scrape)

In this Lecture we will

• Learn that in the latest release of Visual Studio (17.4) we now can enable Agent mode for GitHub Copilot. Now GitHub Copilot not only works with you (synchronous), but works for you (asynchronous).

• Learn that Copilot agent mode is the next evolution in AI-assisted coding. Acting as an autonomous peer programmer, it performs multi-step coding tasks at your command — analyzing your codebase, reading relevant files, proposing file edits,  and implementing the changes that you agree on right in your code. It's like a real time coding partner. Not only will the AI recommend changes, it will do some testing to validate that the changes actually work and even run terminal commands. A major step up from simple code completions.

  • Agent mode will actually look in your all files including all your components pages( .razor)  stylesheets (.css) , class (.cs) and more. 

  • ... But don't blindly accept all changes, that's why it is still necessary to have a good grounding in C#, HTML,  CSS  and ASP.NET core concepts.

    • There is growing gap between those who learned to code before AI tools and those who started with them. The former group tends to value methodical, hands-on coding, while the latter sometimes uses AI to bypass deeper learning. Despite these differences, most developers agree on one point: responsibility lies with the human, not the machine.

      GitHub Copilot can be a powerful ally, but only when used with intention and care. In the end, AI should enhance human decision-making—not replace it.

    • Experienced programmers can take AI and use it as a tool to help them with mundane parts of their coding assignments. They will have the expertise to fix things that the AI gets wrong and be better prepared to make cogent requests that will get results closer to the desired outcome. Attempting to code complex systems without such background/knowledge with AI would be like watching a full season of some TV medical drama and thinking you’re ready to do surgery.

• Learn how to activate Agent Mode

  • Tools...Options...GitHub...Enable Agent mode in the chat pane

  • In the GitHub Copilot Chat Window you will now see under Ask, two options

    • Ask ... this will give you a direct answer (similar to chat) in the side bar and then you can use this information in your application. You may end up copying and pasting specific code  where necessary.

    • Edit ... focuses on code suggestions and improvements within a file, where you provide context and prompts for edits  ( actually blended into Agent functionality) ... so Ask and Agent are the two key Modes now in GitHub Copilot

    • Agent ... this is a more autonomous mode that can handle more complex tasks. In this mode it will do all the steps necessary to implement your request right inside your application, modifying all the required files , installing dependencies if necessary and actually building the project. If will determine the "context" of a request (if none is provided)  and search over your entire code base.

  • Resetting Copilot back to Ask mode

• Practice using Agent mode and some more complex GitHub Copilot techniques in some Blazor Applications

  • In an ideal world, you would just care about the final output of Copilot agent mode, but it can sometimes make mistakes and go off track. To easily intervene and undo in those situations, every  invocation is transparently displayed in the UI, and each solution response requires approval, or you can undo and revert to the state before the last edit was applied.

  • Instead of relying on one long, detailed prompt to yield a perfect solution, you should iterate with Copilot – the UI is built for iterations, giving you full control over the process ensuring the final result is better.

  • AiAgentModeDemoApp1

    • Blazor Web App (.NET 9 Render Mode WebAssembly) ... working with the default template

    • Remove About Heading

      • Please remove the top border that contains the About Link

    • Change Colour Theme (see Color Hunt for possible color palette or use the discretion of Agent with some prompting)

      • Change the colour scheme of the entire site to your discretion please do not use the blue scheme at all including the left side menu

    • Add a Tab Style menu along a top bar (... again some trial and error until the desired menu look is achieved)

      • Update the application menu system, do not use the NavMenu but rather implement a tabbed menu style in the MainLayout using bootstrap

      • That's really good but I want the menu all on the top bar by itself and the display of the component pages directly below

  • AiAgentModelDemoApp2

    • Things don't always work out ... adding QuickGrid to  Weather page ... issues

      • Note: Pre- Rendering in Server Project Weather.razor page to prevent the page from loading twice

        • @rendermode @(new InteractiveServerRenderMode(prerender:false))

    • On the home.razor page put a nice summary of what this application can do section it off with an alert-info bootstarp box

  • Prompting with Images Using GitHub Copilot Vision (still using our current demo as the base of operations)

    • AiAgentModelDemoApp3a/AiAgentModelDemoApp3b/AiAgentModelDemoApp3c

      • using pizzaTemplate.png as image prompt

        • Can you help me build a page that looks like this (3a)

          • using pizza.png in images folder for mainpage (Home.razor)

        • Please keep the original tabbed menu along the top intact and add a completely separate menu right in the Home.razor page along the top with the choices Menu Order Online Book a Table About Hours & Locations (3b)

        • Now please create generic pages for the menu links Menu, Order Online, Book a Table, About , Hours & Locations. Place these new pages in the Client project pages folder (3c)

        • Make sure to make the links active on the Home page to these newly create razor component pages

      Copilot can understand all kinds of image content including for instance a rough drawing of a database layout ie Tables and connections and it will attempt to create this database

  • Using Agents to Fix and Style an existing Blazor Application

    • ToDoStatusUpdateConditionalOriginal

    • ToDoStatusUpdateConditionalAgentUpdate1

      • Redo the Todo.razor page to a more styled and colourful display when the user enters a Todo display the status in a table format with 3 columns the first column called Remove and second column description and the third column IsComplete The is complete column should contain a checkbox when clicked it would inidcate the todo is complete and then would automatically display a garbage pail icon in the Remove column which would allow the user to delete the current todo

      • Choose a different Agent Model (GPt-4.1 ... Claude 3.7 Sonnet)

    • ToDoStatusUpdateConditionalAgentUpdate2

      • small tweak using Ask mode : "Use the iconic icon class to display the trash can".

      • Can't use Bootstrap icon class here because this application was created in .NET 3.2

  • Create a Blazor Hybrid Application ... that mirrors the Song Database (SQLite) application (Lecture 169)

    • HybridSQLiteSongNET8Original

    • HybdSngAgent

      • I want to create an application that helps me tracks  my favourite songs , artists . I want to update the main page so it's a grid and on the top of the grid it lists Songs and Artist Name. On top of the grid I want a section for Song entry and Artist entry and a button to Add the latest entry. I want these new entries to add into a SQLite database. I want the model ie class to be in a separate file  in a folder called models and the database to be in its own separate file in a folder called database

      • Update the current application so that a user can edit a record and delete a record

      • When a user presses the Delete button please prompt the user with "Are you sure" and give them the option to cancel the delete

• Suggested Exercise

  • Create a simple calculator application (Blazor WebAssembly Standalone App) . Add a new razor component page called Calc make sure you can get to it from the navmenu. The application should have two input boxes for the user entry and 4 buttons add, subtract, multiply and divide. The result should appear in a final result text box

  • Take a previously create CRUD application and recreate a similar version using GitHub Copilot Agent Mode

    • ToDoListDbNET8Lect180

      
      

• Summary ... Here are some of the ways developers are using GitHub Agent Mode

  • Refactoring code

  • Migrating a project to another programming language or to another tech stack

  • Writing tests

  • Modernizing legacy code (maybe so you don’t have to learn an outdated language)

  • Autofixing code gen errors

  • Adding new features to an application

  • Prototyping a new app based on a functional spec or UI sketch

  • Implementing non-functional requirements or boilerplate code

  • Scoping and planning the work for implementing a new feature

  • Non-code generation tasks, like documentation

One important thing to note: Copilot is powered by LLMs, which are nondeterministic, and that means its suggestions may differ even with the same prompt and context.

In this Lecture we will

• Highlight a number of ways to access specific functions/capabilities of the Android Phone (Emulator) and the Windows Machine within a  .NET MAUI Blazor Hybrid App. We then leverage some of  these capabilities  to implement AI Analysis of Images, Videos (frame images) and Audio (transcribed text) ...

  HyPhone (.NET MAUI Blazor Hybrid App) ... deep dive into demo no hands on ... but suggested exercises (See Lecture 205 Resource Links )

• Part 1 ...  Phone specific (Andriod Emulator) applications

  • Begin our deep dive into Blazor Hybrid applications  focusing on Android Emulator specific implementations including :

    • Accessing phone contacts lists

    • Creating Push Notifications

    • Biometric Authentication

    • Simulating Swipe Gestures

• Contacts (TheContacts.razor)

  • Access the full Contact list on the Android Phone (Emulator) and allow the user to select a specific contact's full details

    • Note: The use of a service (ContactService) to access device contacts. The Nuget Package Microsoft.Maui.Essentials was installed but with the latest version of .NET MAUI, the Contacts API is now part of the Microsoft.Maui.Devices namespace... actually Microsoft.Maui.Essentials will lead to errors so it has been removed from the .csproj for all the demos in this 3 part Lecture series
      

      • MyContact.cs (in Model folder)
        public  class MyContact

        {

            //This class represents a contact in the phone book.
           //The Contact API returns many properties, but we will only use a few of them.

            public string? Name { get; set; }   

            public string? PhoneNumber { get; set; }

            public string? Email { get; set; }   

        }

      • IContact.cs (in Services folder)
        public interface IContact

        {

            Task<List<MyContact>> GetAll();

            Task<MyContact> SelectContact();

        }

      • ContactService (in Services folder)
        using Microsoft.Maui.ApplicationModel.Communication;
        //This service implements the IContact interface to provide functionality for accessing contacts on the device.

        public class ContactService : IContact

        {

            // This method retrieves all contacts from the device and returns them as a list of MyContact objects.

            public async Task<List<MyContact>> GetAll()

            {

                bool permission = await IsGranted();    // Check if the app has permission to access contacts (method below)

                if (!permission)

                {

                    throw new UnauthorizedAccessException("You are not allowed");

                }

        
        

                // Get all contacts from the device using the Contacts API in .NET MAUI.

                var allContacts = await Microsoft.Maui.ApplicationModel.Communication.Contacts.Default.GetAllAsync();

               
               // Map the retrieved contacts to a list of MyContact objects.

              // MyContact is a custom class that represents a contact in the phone book.

              // Here we only take values for the properties Name, PhoneNumber, and Email.
             // We use the FirstOrDefault() method to get the first email and phone number if they exist,   otherwise we use an empty string.

                var contactList = allContacts.Select(x => new MyContact

                {

                    Email = x.Emails.FirstOrDefault()?.EmailAddress ?? string.Empty,       

                    PhoneNumber = x.Phones.FirstOrDefault()?.PhoneNumber ?? string.Empty,   

                    Name = x.GivenName ?? string.Empty                                     

                }).ToList();

        
        

                return contactList;

            }

        
        

            // This method allows the user to select a single contact from the device's contacts.

            public async Task<MyContact?> SelectContact()

            {

                bool permission = await IsGranted();

                if (!permission)

                {

                    throw new UnauthorizedAccessException("You are not allowed");

                }

        
        

                // Open the contact picker to allow the user to select a contact

                var contact = await Microsoft.Maui.ApplicationModel.Communication.Contacts.Default.PickContactAsync(); 

             

                return contact is null ? null : new MyContact

                {

                    Email = contact.Emails.FirstOrDefault()?.EmailAddress ?? string.Empty,

                    PhoneNumber = contact.Phones.FirstOrDefault()?.PhoneNumber ?? string.Empty,

                    Name = contact.GivenName ?? string.Empty

                };

                //We need to make sure we return a non-nullable MyContact object otherwise the compiler will complain.

            }

        
        

            // This method checks if the app has permission to access contacts.

            // The Permissions class is part of the Microsoft.Maui.Essentials namespace.

            // In.NET MAUI, Essentials APIs(like permissions, contacts, device info, etc.) are integrated directly into the Microsoft.Maui libraries.

            // So you no longer need to install a separate NuGet package for Essentials in .NET MAUI projects, as you did with Xamarin.Essentials in Xamarin projects.

            // The Permissions API is available via using Microsoft.Maui.ApplicationModel;
        

            private async Task<bool> IsGranted()

            {

                var permission = await Permissions.RequestAsync<Permissions.ContactsRead>();    // Request permission to read contacts

                return permission == PermissionStatus.Granted ? true : false;                   // Return true if permission is granted, otherwise false

            }

        
        

            // Don't forget to declare the ContactService in your MauiProgram.cs file to register it as a service.

            // builder.Services.AddScoped<IContact, ContactService>();   

        }

    • TheContacts.razor
      @page "/thecontacts"
      <h6><strong>Phone Contacts</strong></h6>
      
      @*Key service referenced in the code section below *@

      @inject IContact contactService 

      
      

      <button class="btn btn-primary" style="font-size:0.85rem;" @onclick ="SelectContact">Select Contact</button>

      
      

      @*This will display key info specific to chosen contact*@

      @if (SelectedContact != null) 

      { 

          <ul> 

              <li>Name: @SelectedContact.Name</li> 

              <li>Phone: @SelectedContact.PhoneNumber</li> 

              <li>Email: @SelectedContact.Email</li> 

          </ul> 

      } 

      
      

      @*This displays all the contacts in your phone ... just the names and phone numbers*@

      @if (Contacts.Count > 0) 

      { 

          <h3>Contacts</h3> 

          <ul> 

              @foreach (var contact in Contacts) 

              { 

                  <li>@contact.Name - @contact.PhoneNumber</li> 

              } 

          </ul> 

      } 

      else 

      { 

          <p>No contacts available.</p> 

      }

      
      

      @status

      
      

      
      

      @code {

             

          MyContact? SelectedContact = new();                 // Selected contact from the phone where MyContact is a class that holds contact details

          List<MyContact> Contacts = new List<MyContact>();   // List to hold all contacts

      
      

          private string status = "";

      
      

          async Task SelectContact() 

          {

              status = "";

      
      

              if (Microsoft.Maui.Devices.DeviceInfo.Platform == Microsoft.Maui.Devices.DevicePlatform.Android)              // Check if the platform is Android ... don't want to run this on iOS or Windows

              {

                  SelectedContact = await contactService.SelectContact();         // Call the service to select a contact from the phone (contact picker dialog)

                  StateHasChanged();                                              // Refresh the UI to show the selected contact 

              }

              else

              {

                  status = "Only Available when running Android Emulator";

              }

      
      

          } 

      
      

           protected override async Task OnInitializedAsync() 

           { 

              Contacts = await contactService.GetAll();                          // Get all contacts from the phone using the service

           } 

      
      

      }

    • For Android implementation you must also go to Platforms/Android/AndroidManifest.xml and check off "READ_CONTACTS"

    • You must connect to your Google Mail account to actually see your  contacts

• Local Push Notifications (Notify Button ...on Home.razor page of Android Emulator)

  • using the Plugin.LocalNotication Nuget Package...Only for Android Emulator (not visible in Win App)

    • MauiProgram.cs
      var builder = MauiApp.CreateBuilder();

      builder

           .UseMauiApp<App>()     

           .UseLocalNotification()

           .ConfigureFonts(fonts =>

           {

               fonts.AddFont("OpenSans-Regular.ttf", "OpenSansRegular");

           });

    • Home.razor
      @using Plugin.LocalNotification

      @inject INotificationService notificationService

    • @*Make Notify button only visible in Mobile apps ... not visible in WinUI *@

      @if (DeviceInfo.Platform == DevicePlatform.Android || DeviceInfo.Platform == DevicePlatform.iOS)

      {

          <button class="btn btn-info" @onclick="NotifyUser">Notify</button>

      }

    • private void NotifyUser()

      {       

          //Display an alert here to inform the user that they must turn on the notifications in the Android Emulator settings for the notification to work

          //Specifically they need to allow notifications for the BlazHybridPhoneDemos app in the Android Emulator settings

          App.Current.Windows[0].Page.DisplayAlert("Warning", "You need to allow notifications for the BlazHybridPhoneDemos app in the Android Emulator settings before the Notify button will work. If set properly you will hear a little sound. To actually view the Notification pull down the top of the screen on the Emulator" , "OK");

         

          // Show a notification with a title, description, and scheduled time

          // The notification will appear in the Android Emulator's notification bar after 5 seconds

          notificationService.Show(new NotificationRequest

          {

              NotificationId = 1,

              Title = "BlazHybridPhoneDemos",

              Description = "This is a test notification from BlazHybridPhoneDemos",

              Schedule = new NotificationRequestSchedule

              {

                  NotifyTime = DateTime.Now.AddSeconds(5)

              }

          });

      }

    • Need to add new Platform permission POST_NOTIFICATIONS in AndroidManifest.xml file

    • When executing the Android Emulator , after you press Notify button , you will see a notification in the Notification Bar of the Emulator (you may need to swipe down to see it)

    • All the coding is in the method NotifyUser() on the Home.razor page

• Biometric Authentication

  • Use the Plugin.Maui.Biometric Nuget Package to add Biometric Authentication to the Blazor Hybrid App.

  • The Biometric Authentication is only for the Android Emulator

    • You will need to add the Biometric permission (USE_BIOMETRIC) in the AndroidManifest.xml file

  • Almost all of the coding is in the MainPage.xaml (BlazorWebView initially invisible) and MainPage.xaml.cs files (most of the coding is here where PIN verification is performed)

    • MainPage.xaml 
      <BlazorWebView

           x:Name="blazorWebView"

           IsVisible="False"

           HostPage=""> <!-- The HTML host page for Blazor -->

           <BlazorWebView.RootComponents>

               <!-- Register the root Blazor component to render at the #app selector -->

               <RootComponent

                   Selector="#app"

                   ComponentType="{x:Type components:Routes}" />

           </BlazorWebView.RootComponents>

      </BlazorWebView>

    • Since the Android Emulator does not have a fingerprint sensor or face recognition we will prompt the user to enter a PIN code (12345) to simulate the biometric authentication process.

    • MainPage.xaml.cs
      using Plugin.Maui.Biometric;
      protected override async void OnAppearing()

      {

           base.OnAppearing();

      
      

           //To ensure the biometric check only happens the very first time the app is run (an issue when closing XAML pages),

           //we use a persistent flag (e.g., in Preferences) to track if authentication has already occurred.

           //Preferences is a class provided by the Microsoft.Maui.Storage namespace in .NET MAUI.

           //It allows you to store and retrieve simple key-value pairs of data (such as strings, numbers, or booleans) in a persistent way on the device.

           //This is commonly used for saving user settings, flags, or small pieces of data that need to persist between app launches.
      //This is slightly different from Cookies which are typically used for web applications to store session data in the browser.

      
      

          //Preferences.Get("HasAuthenticated", false) tries to retrieve a value associated with the key "HasAuthenticated" from the app's persistent storage.

           //If the key does not exist(for example, the first time the app runs), the method returns the default value you provide in this case, false.

           //So the second parameter is the default value to return if the key does not exist.

           //But if the key exists and has a value, it will return that value (true) instead of false.
      //Note: in .NET MAUI, values set using Preferences.Set are persisted across app restarts and device reboots.

      //on the Home.razor page I have added a button to clear the Preferences, so you can test the authentication flow again.

      //Preferences.Remove("HasAuthenticated");

      
      

           if (Preferences.Get("HasAuthenticated", false))

           {

               blazorWebView.IsVisible = true;

               return;

           }

      
      

           // Only check authentication on Android if not running on Android, show the BlazorWebView directly and exit.

           if (DeviceInfo.Platform != DevicePlatform.Android)

           {

               blazorWebView.IsVisible = true;

               return;

           }

      
      

           // If running on Android, proceed with biometric authentication

           var biometric = BiometricAuthenticationService.Default; // Get the default biometric authentication service

           bool isAuthenticated = false;                           // Flag to track if the user is authenticated

      
      

           if (biometric.IsPlatformSupported)

           {

               var authenticationRequest = new AuthenticationRequest   // Create an authentication request

               {

                   Title = "Unlock to continue",

                   NegativeText = "Cancel",

                   Subtitle = "Subtitle",

                   Description = "Test",

                   AllowPasswordAuth = true,

               };

      
      

               // Attempt to authenticate using biometrics

               AuthenticationResponse response = await biometric.AuthenticateAsync(authenticationRequest, CancellationToken.None);

      
      

               // Check the response status

               // If the response status is Success, show a success alert and set the authenticated flag

               // If the response status is not Success, fall back to PIN authentication

               if (response.Status == BiometricResponseStatus.Success)

               {

                   await DisplayAlert("Success", "Auth success", "OK");

                   isAuthenticated = true;

               }

               else

               {

                   // Fallback to PIN for any non-successful biometric authentication

                   while (true)

                   {

                       string pin = await DisplayPromptAsync("PIN Required", "Enter your PIN to continue:", "OK", "Cancel", "PIN", maxLength: 10, keyboard: Keyboard.Numeric);

                       if (string.IsNullOrEmpty(pin))

                       {

                           // User cancelled

                           break;

                       }

                       if (pin == "12345")

                       {

                           await DisplayAlert("Success", "PIN correct. Auth success", "OK");

                           isAuthenticated = true;

                           break;

                       }

                       else

                       {

                           await DisplayAlert("Auth Failed", "Incorrect PIN", "OK");

                       }

                   }

               }

           }

           else

           {

               // No biometric support at all, fallback to PIN

               while (true)

               {

                   string pin = await DisplayPromptAsync("PIN Required", "Enter your PIN to continue:", "OK", "Cancel", "PIN", maxLength: 10, keyboard: Keyboard.Numeric);

                   if (string.IsNullOrEmpty(pin))

                   {

                       // User cancelled

                       break;

                   }

                   if (pin == "12345")

                   {

                       await DisplayAlert("Success", "PIN correct. Auth success", "OK");

                       isAuthenticated = true;

                       break;

                   }

                   else

                   {

                       await DisplayAlert("Auth Failed", "Incorrect PIN", "OK");

                   }

               }

           }

      
      

           // Set the visibility of the BlazorWebView based on authentication status

           // If authenticated, the BlazorWebView will be visible; otherwise, it will be hidden.

           // BlazorWevView displays the Blazor content in the app

      
      

          if (isAuthenticated)

           {

               Preferences.Set("HasAuthenticated", true);  // Store the authentication status in preferences

           }

      
      

           blazorWebView.IsVisible = isAuthenticated;

      }

    • Home.razor
      @if (DeviceInfo.Platform == DevicePlatform.Android || DeviceInfo.Platform == DevicePlatform.iOS)

      {

          <button class="btn btn-info btn-sm mb-2" @onclick="NotifyUser">Notify</button>

          <button class="btn btn-info btn-sm mb-2" @onclick="ClearAuth">Clear HasAuthenicated Preference </button>

      }
      
      private void ClearAuth()

      {       

          // Clear the authenticated preference for testing purposes

          Preferences.Remove("HasAuthenticated");

          App.Current.MainPage.DisplayAlert("Info", "HasAuthenticated preference cleared for testing purposes.", "OK");

      }

• Swipe Gestures (Swipe.razor)

  • Swipe left to display other button actions

    • Uses the Javascript library hammer.min.js to handle swipe gestures on the Android Emulator or Windows Machine App.

    • The js is kickstarted with the file script.js  located in the scripts folder

      • from there it will create a Hammer instance and attach the swipe event to the swipe-item div located on the Swipe.razor page

      • // Adds a swipe left gesture handler to a DOM element with the given swipeItemId.

        // When swiped left, the element is translated left and associated action buttons are shown.

        function addSwipeHandler(swipeItemId, buttonsId) {

            const swipeItem = document.getElementById(swipeItemId); // Get the swipeable item element

            const actionButtons = document.getElementById(buttonsId); // Get the action buttons element

        
        

            if (!swipeItem || !actionButtons)

                return;

        
        

            const hammertime = new Hammer(swipeItem); // Initialize Hammer.js on the swipe item

            hammertime.on("swipeleft", function () {

                swipeItem.style.transform = "translateX(-150px)"; // Move the item left

                swipeItem.style.transition = "transform 0.3s ease-in-out"; // Animate the movement

                actionButtons.classList.add("visible"); // Show the action buttons

            });

        }

        
        

        // Restores the swipeable item and hides the action buttons.

        // Resets the transform and transition styles to their original state.

        function restoreSwipe(swipeItemId, buttonsId) {

            const swipeItem = document.getElementById(swipeItemId); // Get the swipeable item element

            const actionButtons = document.getElementById(buttonsId); // Get the action buttons element

        
        

            if (!swipeItem || !actionButtons)

                return;

        
        

            swipeItem.style.transform = "translateX(0)"; // Reset the item's position

            swipeItem.style.transition = "transform 0.3s ease-in-out"; // Animate the reset

            actionButtons.classList.remove("visible"); // Hide the action buttons

        }

    • Swipe.razor
      @page "/swipe" 

      @inject IJSRuntime js 

      @inject NavigationManager NavigationManager 

      
      

      <div class="alert alert-primary" role="alert" style="font-size: 0.85em;">

          <strong>Purpose:</strong> This page demonstrates how to implement <b>Swipe Gestures</b> in a .NET MAUI Blazor Hybrid app.<br />

          <strong>How it works:</strong>

          <ol class="mb-0">

              <li>Swipe left on the item to reveal action buttons.</li>

              <li>Click <b>Click Here</b> to trigger a sample alert.</li>

              <li>Click <b>Go to Counter Page</b> to navigate to the Counter demo.</li>

              <li>Click <b>Cancel</b> to restore the swipe item to its original state.</li>

              <li>Swipe gestures are handled via JavaScript (Hammer.js) and custom logic in <code>script.js</code>.</li>

          </ol>

      </div>

      
      

      @*swipe-container and swipe-item are defined in the css file Swipe.razor.css

          swipe-item is the div that contains the swipe item and is the element that will be swiped

          action-buttons is the div that contains the action buttons and will be shown when the swipe item is swiped left

          these are both referenced in the JavaScript function addSwipeHandler and restoreSwipe in the OnAfterRenderAsync Task below

      *@

      <div id="swipe-container" class="swipe-container"> 

          <div id="swipe-item" class="swipe-item"> 

              <strong>Swipe left for all the options</strong>

          </div> 

      
      

          <div id="action-buttons" class="action-buttons"> 

              <button id="delete-button" class="delete-button btn-sm" @onclick="SampleItem" style="padding-top: 0.25rem; padding-bottom: 0.25rem; font-size: 0.70em;">Click Here</button> 

              <button id="link-button" class="btn btn-secondary btn-sm" @onclick="LinkItem" style="padding-top: 0.25rem; padding-bottom: 0.25rem; font-size: 0.70em;">Go to Counter Page</button> 

              <button id="cancel-button" class="cancel-button btn-sm" @onclick="CancelItem" style="padding-top: 0.25rem; padding-bottom: 0.25rem; font-size: 0.70em;">Cancel</button>

          </div> 

      </div> 

      
      

      @code { 

          private async Task SampleItem() 

          { 

              await App.Current.Windows[0].Page.DisplayAlert("Info", "You could execute a method here", "OK"); 

          } 

      
      

          private async Task LinkItem() 

          { 

              NavigationManager.NavigateTo("/counter"); 

          } 

      
      

          // This method is called when the cancel button is clicked

          // It will restore the swipe item to its original state

          // swipe-item is the id of the div above that contains the swipe item

          // action-buttons is the id of the div above that contains the action buttons

          private async Task CancelItem() 

          { 

              await js.InvokeVoidAsync("restoreSwipe", "swipe-item", "action-buttons"); 

          } 

      
      

          //Here we are using the OnAfterRenderAsync method to add the swipe handler after the component has rendered for the first time

          //this is necessary to ensure that the JavaScript function is available when the component is rendered

          //addSwipeHandler is a JavaScript function defined in the script.js file that handles the swipe gestures

          //from there it will create a Hammer instance and attach the swipe event to the swipe-item div

          //script.js should be included in the index.html file along with the hammer.min.js library

      
      

          protected override async Task OnAfterRenderAsync(bool firstRender) 

          { 

              if (firstRender) 

              { 

                  await js.InvokeVoidAsync("addSwipeHandler", "swipe-item", "action-buttons"); 

              } 

          } 

      }

In this Lecture we will

• Part 2 ... Windows Machine
  Continue our deep dive into Blazor Hybrid applications using Razor pages focusing on Windows only applications with AI implementations dealing with:

  • Images (Camera/Webcam access) + AI Analysis returning a structured response (class based)

  • Audio (Microphone access) with  Transcription and  AI Analysis of Translation

  • Video (Webcam access) with AI Analysis of Video frames and Similarity scoring

  
  

•   Camera (TheCamera.razor)  ... primarily a Windows specific  App with some features available in  the Android Emulator implementation.

  • Access the Camera / Webcam , Take a Photo, Save,Load and perform AI Analysis  (including a custom class for specific property extraction if image conforms )

    • For Android implementation you must also go to:

      Platforms/Android/AndroidManifest.xml and check off "CAMERA" and "WRITE_EXTERNAL_STORAGE"

    • For Windows implementation we are using the FileSystem.AppDataDirectory as our default saving/loading directory

      Photo saved in C:\Users\chiar\AppData\Local\User Name\com.companyname.blazhybridphonedemos\Data\images

    • Also note the use of the CameraService and the Tasks PickAPhotoAsync (loads in any selected photo using MediaPicker ... In Android will open up the Phone Photo Gallery) and TakePhotoAsync (in Windows this will open the local Webcam if available to take snapshot ... in Android Emulator you may get a response that no camera is available to take a picture). Try the Camera (XAML) page for a better implementation which uses the CommunityToolkit.Maui.Camera NuGet Package and the CameraView.

      •   //This service provides a method to load photos via a media picker

                //In Windows you can pick photos from the file system

                //In Android you can pick photos from the phone/camera photo gallery

                public async Task<FileResult> PickAPhotoAsync()

                {

        #if ANDROID || IOS || MACCATALYST || WINDOWS

                    if (MediaPicker.Default.IsCaptureSupported)

                    {

                        return await MediaPicker.Default.PickPhotoAsync(); 

                    }

                    else

                    {

                        throw new NotSupportedException("Photo access is not supported on this device.");

                    }

        #else

                    throw new NotSupportedException("Photo access is not supported on this device.");

        #endif

                }

      •         // This method captures a photo using the device's camera.

               // In Android, it checks for camera and storage permissions.

              // In iOS, macOS, and Windows, it uses the MediaPicker to capture a photo.
                public async Task<FileResult> TakePhotoAsync()

                {

        #if ANDROID

                    var cameraStatus = await Permissions.RequestAsync<Permissions.Camera>();

                    var storageStatus = await Permissions.RequestAsync<Permissions.StorageWrite>();

                    if (cameraStatus != PermissionStatus.Granted || storageStatus != PermissionStatus.Granted)

                        throw new Exception("Camera or storage permission not granted.");

                    var context = Android.App.Application.Context;

                    if (!context.PackageManager.HasSystemFeature(PackageManager.FeatureCameraAny))

                        throw new NotSupportedException("No camera available on this device.");

                    var photo = await MediaPicker.Default.CapturePhotoAsync(new MediaPickerOptions

                    {

                        Title = $"photo_{DateTime.Now:yyyyMMdd_HHmmss}.jpg"

                    });

        #elif IOS || MACCATALYST || WINDOWS

                    if (!MediaPicker.Default.IsCaptureSupported)

                        throw new NotSupportedException("Camera capture is not supported on this device.");

                    var photo = await MediaPicker.Default.CapturePhotoAsync();

        #else

                    throw new NotSupportedException("Camera capture is not supported on this device.");

        #endif

                    if (photo == null)

                        throw new OperationCanceledException("Photo capture was canceled.");

                    return photo;

                }

      • Don't forget to declare the service in MauiProgram.cs so it can be injected in TheCamera.razor page
        builder.Services.AddScoped<CameraService>();

      • TheCamera.razor
        @inject CameraService cameraService 
        @inject BlazHybridPhoneDemos.Model.WebCamState webCamState 
        
        private async Task TakePhoto()

        {   

        
        

            ClearAll(); // Clear any previous state before capturing a new photo

        
        

            if (Microsoft.Maui.Devices.DeviceInfo.Platform == Microsoft.Maui.Devices.DevicePlatform.WinUI)

            {

                if (App.Current?.Windows?.Count > 0 && App.Current.Windows[0]?.Page != null)

                {

                    await App.Current.Windows[0].Page.DisplayAlert("Info", "You are running a Windows Machine App ... I will try to open a Webcam if available", "OK");

                }

                else

                {

                    errorMessage = "Unable to display alert. The application window or page is null.";

                }

                try

                {

                    var photo = await cameraService.TakePhotoAsync();       // TakePhotoAsync opens the Webcam on Windows and allows user to take a photo

                    if (photo != null)

                    {

                        lastPhoto = photo;                                  // Store the captured photo in lastPhoto ... used by CanSavePhoto property above and SavePhoto method below

                        photoDataUrl = await GetPhotoDataUrlAsync(photo);   // Convert the photo to a data URL for display

                        webCamState.IsCameraAvailable = true;               // Set the WebCam state to active

                        webCamState.NotifyStateChanged();                   // Notify NavMenu or other  subscribers of the state change

                    }

                }

                catch (Exception ex)

                {

                    errorMessage = "Error taking Photo: " + "<br /><hr />" + ex.Message;

                    webCamState.IsCameraAvailable = false;      // Set the WebCam state to inactive if an error occurs

                    webCamState.NotifyStateChanged();           // Notify NavMenu or other subscribers of the state change

        
        

                }

            }       

            else if (Microsoft.Maui.Devices.DeviceInfo.Platform == Microsoft.Maui.Devices.DevicePlatform.Android)

            {

                if (App.Current?.Windows?.Count > 0 && App.Current.Windows[0]?.Page != null)

                {

                    await App.Current.Windows[0].Page.DisplayAlert("Info", "You are running on a mobile device ( Android Emulator) ... so taking a photo with the Camera may not be available using MediaPicker.Default.CapturePhotoAsync ... try the Camera (XAML) version which implements the CommunityToolkit.Maui.Camera NuGet Package and the CamerView", "OK");

                }

                else

                {

                    errorMessage = "Unable to display alert. The application window or page is null.";

                }

                try

                {

                    var photo = await cameraService.TakePhotoAsync();       // TakePhotoAsync opens the Camera on Android if available

                    if (photo != null)

                    {

                        lastPhoto = photo;                                  // Store the captured photo in lastPhoto ... used by CanSavePhoto property above and SavePhoto method

                        photoDataUrl = await GetPhotoDataUrlAsync(photo);   // Convert the photo to a data URL for display

                    }

                }

                catch (Exception ex)

                {

                    errorMessage = "Error taking Photo: " + "<br /><hr />" + ex.Message;

                }

            }

        }

      • WebCamState.cs
        //used to make Camera(XAML) page available (for Windows) only if camera is active on //computer running application. There is no issue when running Android Emulator.
        public class WebCamState

        {

            public bool IsCameraAvailable { get; set; } = false; // Default to false, assuming camera is not available

        
        

            //See NavMenu.razor and TheCamera.razor for Implementation

        
        

            // Event to notify subscribers when the camera availability changes

            public event Action? OnChange;

        
        

            // Method to set the camera availability and notify subscribers

            public void NotifyStateChanged() => OnChange?.Invoke();
        }

    • Note: The AI Image Analysis uses Microsoft.Extensions.AI and is done using the ChatClient and the GetResponseAsync method. The AI will analyze the image and try to return a structured response (based on the FormResult class ... declared within the Code Section) with the information extracted from the image.

      Note: The AI analysis is done using a custom prompt to extract specific information from the image. 

      • Don't forget to install the Nuget Packages

        • Microsoft.Extensions.AI
          Microsoft.Extensions.AI.OpenAI

      • Don't forget the necessary declarations in the MauiProgram.cs    

        
        using Microsoft.Extensions.AI;

        using OpenAI;
        
        // Add AI service to the container ... OpenAI

        var openAiClient = new OpenAIClient("yourKey");

        builder.Services.AddChatClient(new OpenAIChatClient(openAiClient, "gpt-4o-mini"));

        // Register OpenAIServices with HttpClient factory

        builder.Services.AddHttpClient<OpenAIServices>();

      • TheCamera.razor
        @using Microsoft.Extensions.AI

        @inject IChatClient ChatClient

        
        

        @*IChatClient is an interface provided by the Microsoft.Extensions.AI library.

        It represents a client for interacting with AI-powered chat services, such as large language models (LLMs).

        In the context of our .NET MAUI Blazor app, it is used to send messages (including text and images) to an AI service and receive responses.*@

• Audio ... Windows only (Sound.razor)

  • This page allows you to record audio using your device's microphone, play back recordings, save them as MP3 files (file System AppDir) , and load existing audio files (from any local directory) . You can also select and play MP3 files from the app's audio folder (file system App Directory).

  • It also implements some AI integration by Converting Audio to Text (Transcribing), Converting Foreign Audio to English Text (Translating) and Playing back text to audio in a number of voices. For point of comparison the built-in MAUI TextToSpeech.Default.SpeakAsync command is also demonstrated.

  • Make note of how we are leverage the transcribe/translation capabilities to Analyze the audio-to-text (via AI) and pull out specific information according to some predefined properties we declared in a class called MyTasks (see the Analyze Audio button ). We are basically taking unstructured voice input and producing structured data.

  • Programming Details

    • Audio recording and playback is powered by the audioRecorder.js JavaScript module (in the scripts folder of wwwroot).

      • The script provides the microphone access, recording(pause/resume/stop), playback

        • audioRecorder.initRecorder is used by Task InitRecorder()

        • audioRecorder.startRecording is used by Task StartRecording()

        • audioRecorder.pause is used by Task PauseRecording()

        • audioRecorder.resumeRecording is used by Task ResumeRecording()

        • audioRecorder.stopRecording is used by Task StopRecording()

        • audioRecorder.getAudioDataUrl is used by Task SetAudioPlayerSource()  and returns the recorded audio as a base64 data URL (data:audio/mp3;base64,...)

        • audioRecorder.playAudio is used by Task PlayAudio() to play audio that was just recorded as opposed to selected audio from a list where we use
          await JS.InvokeVoidAsync("eval", "document.getElementById('audioPlayer').play();");

      • Note the reference in the index.html
        <script src="scripts/lame.min.js"></script>

        <script src="scripts/audioRecorder.js"></script>

    • Sound.razor

      • Note: This page make a number of Javascript calls to the audioRecorder.js script , implements AI for Analysis (Microsoft.Extensions.AI) and  uses a Text to Speech OpenAI endpoint (stored in the OpenAIServices.cs file in the Services folder)
        @inject OpenAIServices OpenAIServices

        @using Microsoft.JSInterop

        @using Microsoft.Extensions.AI

        @inject IChatClient ChatClient

      • Key Methods/Tasks

        • private async Task StopRecording()

          {

              await JS.InvokeVoidAsync("audioRecorder.stopRecording");

               isRecording = false;

               isPaused = false;

               hasAudio = true;                //referenced in the PlayAudio method below to indicate that we have recorded audio to play and this also enables the Save button

               canPlayAudio = true;

               isPlayingSelectedMp3 = false;   // Reset the isPlayingSelectedMp3 flag since we are not playing a selected MP3 file but rather the recorded audio

               isSaved = false;                // Reset the isSaved flag, this will be turned on when we reload the audio file after saving it (in the PickAndLoadAudio method)

               // for AI Transcribing and Translating

               // The issue is, that after we record, we have a Blob URL eg.

               // but we need to have a base64 URL in Blazor. Unfortunately C# cannot directly access browser memory blobs so we need to

               // re-access the audio file after saving it to the File System App Directory ... using the PickAndLoadAudio method below (clicking the Load Audio File button)

          
          

               status = "Stopped. Ready to play or save.";

               await SetAudioPlayerSource();   // Set the audio player source to the recorded audio

          }

        • // Converts the current Recorded audio to a base64 data URL and sets mUrl ... only used when we record audio

          private async Task SetAudioPlayerSource()

          {       

              mUrl = await JS.InvokeAsync<string>("audioRecorder.getAudioDataUrl");                       // Use the new JS function to get a base64 data URL

              await JS.InvokeVoidAsync("eval", $"document.getElementById('audioPlayer').src='{mUrl}';");  // Set the audio player source to the recorded audio data URL

              canPlayAudio = true;

              isPlayingSelectedMp3 = false;

              //<audio id="audioPlayer" src="@mUrl" controls style="display:@(canPlayAudio ? "inline-block" : "none"); vertical-align: middle; max-width: 220px;"></audio>

          } 

        • // This method plays the selected MP3 file or the Audio Recording. 

          private async Task PlayAudio()

          {

               if (isPlayingSelectedMp3 && !string.IsNullOrEmpty(selectedMp3)) // Check if we are playing a selected MP3 file from the list

               {

                   //This is a standard JavaScript method for HTML audio/video elements.

                   //It tells the browser to start playing the audio from its current source (src) which is stored in mUrl that came from the SelectMp3 method.

                   await JS.InvokeVoidAsync("eval", "document.getElementById('audioPlayer').play();");

                   status = $"Playing: {selectedMp3}";

               }

               else if (hasAudio)                                              // Check if we have recorded audio to play

               {

                   //This line tells the browser to play the audio that was recorded using the JavaScript audio recorder module.

                   //It's typically used after recording audio, so the user can listen to their recording.

                   await JS.InvokeVoidAsync("audioRecorder.playAudio");

                   status = "Playing recorded audio...";

               }

          }

        • private async Task SaveSysMp3()

          {

              //This method saves the recorded audio to the File System App Directory

              //If the directory does not exist, it will be created and named audio

              //The audio file will be saved with the name provided in the fileName variable

              //It saves the audio file in mp3 format

        • private async Task ConvertAudioToText(int type)

          • //The OpenAI API endpoints for audio transcription and translation

            if (type == 1)      // Check if the type is 1 for transcription

            {           

                response = await httpClient.PostAsync("transcriptions", content);   //This is the endpoint for audio transcription

                Heading = "Transcription"; // Set the heading for transcription

            }

            else if (type == 2) // Check if the type is 2 for translation

            {           

                response = await httpClient.PostAsync("translations", content);     //This is the endpoint for audio translation

                Heading = "Translation"; // Set the heading for translation   

            } 

        • // This method plays the audio from the transcription text using the selected voice

          private async Task PlaySpeech()
          //fetch the new speech URL based on the transcription text (obtained from the ConvertAudioToText method) and selected voice

          AudioUrl = await OpenAIServices.ConvertTextToSpeechAsync(mTranscription,SelectedVoice);

        • public  class OpenAIServices

          //The POST request is sent to the OpenAI API using the HttpClient.PostAsync method.

          //This will use the image api model called dall-e-2 to generate images

          var response = await _httpClient.PostAsync("generations", requestContent);   

          
          

• Video ... Windows only (VideoPage.razor)

  • Create, Save and Load Videos and single rame Images from a live feed  video in a Windows Machine App using a local Webcam . Pull out single frames (images) or multiple frames from the video for AI Analysis and Similarity Scoring (a numerical representation of how alike two images are)

    • There are no video AI analysis buttons on screen (except for the basic single frame/image AI button)

    • Once a user saves and then re-loads a video , 10 evenly spaced frames of the video appear and anyone of them can be clicked on for Analysis or a new button label "Analyze All Captured Frames" can be clicked.

  • This application implements  the OpenCvSharp4 NuGet Package ... specifically  the OpenCvSharp4.runtime.win library which allows us to easily incorporate Computer Vision and Image processing.

    • This library allows us to connect  to cameras/Webcams, take a picture , save the picture (image/frame) , record videos, save videos and re-open video and image files

    • We demonstrate how to  analyze the video (frames) using the Microsoft.Extensions.AI library.

      • We extracts frames evenly spaced across the entire video and then allow the user to analyze any one of them

      • We also allow  the user to send all extracted frames to OpenAI for analysis at the same time.

      • We also add the ability to compare images for Similarity using the OpenCvSharp4 library. Specifically we compare the first and last frames of any selected video using two techniques

        • The first technique calculates the mean difference for each color channel (B, G, R) and averages out these values to get a rudimentary score ... low is better high is worse

        • The second more advanced technique uses the Structural Similarity Index (SSIM), which compares images based on luminance, contrast, and structure, providing a value between -1 and 1 (where 1 means identical images).

    • We can apply this technology to real-world problems like insurance claim analysis. Automate report generation for vehicle damage videos, detailing the car model, year, damage specifics, and incident descriptions. This demonstrates AI’s transformative potential.

  • Programming Details

    • Install the NuGet Package ... OpenCvSharp4.Windows

    • All the Coding is located in the VideoPager.razor file

      • Initial Declarations
        @using Microsoft.Extensions.AI;

        @inject IChatClient ChatClient

        
        

        @*allows us to use Directory and Path commands in the code section*@

        @using System.IO

        
        

        @*allows us to use the OpenCvSharp4 library*@

        @using OpenCvSharp

      • Key Initial Variable Declarations in the Code Section
        //Variables to check if the application is running on Android used in the

        //InitializeMediaAsync Task to display a warning message

        private bool isAndroid = false;

        private string? androidWarning;   

        
        

        //Image related variables

        //---------------------------------------------------------------------------------

        //image  is used only in the ReadFile method:Stores the base64-encoded string of the image file's bytes.

        //Immediately after, imageBase64 is set to a data URL using the value of image.

        private string? image;                                               

        private string? imageBase64; 

        
        

        private string? selectedImageBase64;        //used to store the selected image as a base64 string  ... in RecognizeFrameImage(string img) method

                                                    //and HTML section where we display it beside the first frame image

        
        

        List<string> fileList = new List<string>(); //List of image files in the wwwroot/images directory (debug folder)

        private long imageFileSize;                 //used to store the image file size   

        
        

        //----------------------------------------------------------------------------------

        
        

        //Video related variables   

        private string? firstFrameBase64; //used to store the first frame of the video ... used in ReadVideoFile(string file) method and then copied to imageBase64 variable for display and possible AI Analysis

        private string? videoBase64;      //used to store the video as a base64 string ... used in OnFileChangedVideo(InputFileChangeEventArgs e) method and ReadVideoFile(string file) method

        private long videoFileSize;       //used to store the video file size

        private List<string> videoFileList = new List<string>(); //used to store the list of video files in the wwwroot/videos directory (debug folder)

        private string? VideoFileName;    //used to store the name of the video file

        
        

        //Camera related variables

        private Mat? currentFrame;          // used to store the current frame from the camera (OpenCvSharp Mat type)

        private string? currentFrameBase64; // used to store the current frame as a base64 string

        private VideoCapture? videoCapture; // used to capture video from the camera (OpenCvSharp VideoCapture type)

        private bool isCameraActive = false;// used to track if the camera is active

        private string? capturedImageBase64; //used to store the captured image from the camera in base64 format

        private List<string> videoFramesBase64 = new List<string>(); // List to store frames as base64 strings

        private string? frameStatus;        //used to display the number of frames captured from the video

        private bool isFrame = false;       //used to track if the image is a frame from a video or a regular image selected from the left side list or taken from the camera

                                            //Set to false when analyzing a regular image, Set to true when analyzing a frame from a video

        
        

        //Camera Recording related variables

        private bool isRecording = false;   //used to track if the camera is recording

        private VideoWriter? videoWriter;   // used to write video to file (OpenCvSharp VideoWriter type) ... used in RecordVideo() and StopRecording() methods

        private string? recordedVideoFileName; // used to store the name of the recorded video file   

        
        

        //Variables for the Image Recognition AI Form

        private string? responseMessage;

        private string? status;

        
        

        //Class to store the structured response from the AI located in the Shared Project

        private FormResult? formResult;

      • Pre-loading currently saved videos and images
        private async Task InitializeMediaAsync()

        {

            //C# lets you add the @ symbol in front of a string to create a

            //verbatim string literal where backslashes are not interpreted as escape characters ... so you can use them to specify file paths

        
        

            // Detect Android at startup

            isAndroid = OperatingSystem.IsAndroid();

            if (isAndroid)

            {

                androidWarning = "Warning: Webcam access only available in Windows Machine ... Not Android";

            }

            else

            {

                videoFileList.Clear(); // Clear the Video list to avoid duplicates

                fileList.Clear();      // Clear the Image list to avoid duplicates

        
        

                //Video specific code

                //----------------------------------------------------------------------------------

                string path = Directory.GetCurrentDirectory() + @"\wwwroot\videos";           

                var files = Directory.GetFiles(path, "*.*")

                                     .Where(file => file.EndsWith(".mp4") || file.EndsWith(".mov"))

                                     .ToArray();           

        
        

                //Here we are storing the video file names in a list that we can use to display the video files on the left side of the screen

                //and to play the video when the user clicks on the file name

                foreach (var file in files)

                {

                    videoFileList.Add(Path.GetFileName(file));

                }

        
        

                //GetFiles ... returns an array of the full names (including paths) for the files in the specified directory

                //[0] = "C:\\Temp\\Blazor\\Vision\\4-BlazorKernelVisionWebAppUpdate\\BlazorKernelVisionWebApp\\BlazorKernelVisionWebApp\\wwwroot\\images\\beach.jpg"

                //GetFileName returns the file name and extension of the specified path string ... beach.jpg

        
        

                //Image specific code

                //----------------------------------------------------------------------------------

                string Imgpath = Directory.GetCurrentDirectory() + @"\wwwroot\images";

                var Imgfiles = Directory.GetFiles(Imgpath);

        
        

                //Here we are storing the image file names in a list that we can use to display the image files on the left side of the screen

                //and display the images when the user clicks on the file name

                foreach (var file in Imgfiles)

                {

                    fileList.Add(Path.GetFileName(file));

                }             

            }       

        }   

      • Key Tasks/Methods

        • //This Task/method is used to handle the video file chosen via the Blazor InputFile component 

          //It is called when the user clicks on the "Blazor InputFile ... not .NET MAUI MediaPicker" button

          private async Task OnFileChangedVideo(InputFileChangeEventArgs e)

        • //This Task used the .NET MAUI FilePicker
          private async Task PickAndLoadVideo()

        • //The ReadFile method is reading a file from the local directory (wwwroot/images) that you have clicked on from a list on screen

          //and it accepts a simple STRING parameter (beach.jpg) for instance
          private void ReadFile(string file)   

        • //The ReadVideoFile method is reading a file from the local directory (wwwroot/video)

          //that you have clicked on from the list on the left side of the screen

          //it accepts a simple STRING parameter (cake.mp4) for instance

          private void ReadVideoFile(string file)

          {

               Clear();

               frameStatus = ""; //used to display the number of frames captured from the video

               
               //var path = Path.Combine(Directory.GetCurrentDirectory(), "wwwroot", "videos", file);

               var path = Directory.GetCurrentDirectory() + @"\wwwroot\videos\" + file;

               var buffer = File.ReadAllBytes(path);

               videoBase64 = "data:video/mp4;base64," + Convert.ToBase64String(buffer); //Convert the byte array to a base64 string and set the videoBase64 variable to the base64 string

          
          

               FileInfo fi = new FileInfo(path);

               videoFileSize = fi.Length;

          
          

               // Extract the first frame of the video

               //-------------------------------------------------------------------------------------------

               using var videoCapture = new VideoCapture();            // Create a new VideoCapture object (part of OpenCvSharp library)

               videoCapture.Open(path, VideoCaptureAPIs.ANY);          // Open the video file ... expecting a file path or a stream, NOT a base64 string or a web URL.

               //videoCapture.Open(videoBase64, VideoCaptureAPIs.ANY);

          
          

               //declare firstFrameBytes here so it is in scope for the file size calculation

               byte[] firstFrameBytes = null; //used to store the first frame of the video

          
          

               if (videoCapture.IsOpened())        // Check if the video file is opened successfully

               {

                   using var frame = new Mat();    // Create a new Mat object to store the frame ... Mat is a class in OpenCvSharp that represents an image or a video frame

                   if (videoCapture.Read(frame))   // Read the first frame

                   {

                       firstFrameBytes = frame.ToBytes(".jpg");                                        // Convert the frame to byte array

                       firstFrameBase64 = "data:image/jpeg;base64," + Convert.ToBase64String(firstFrameBytes); // Convert to base64 string   

                       imageBase64= firstFrameBase64;                                                  //copy first frame to imageBase64 variable for possible Image Analysis (see RecognizeImage Task)

                       imageFileSize= firstFrameBytes.Length;                                          //determine the image file size

                   }             

               }

               else

               {

                   responseMessage = "Unable to open the video file.";

                   return;

               }

          
          

               // Evenly Spaced Frame Extraction

               //There was an issue here with videoCapture.Read(frame) not working properly with the base64 string

               //OpenCvSharp.VideoCapture.Open() ... located above ...  expects a file path or a stream, not a base64 string or a web URL.

               //Solution:

               //Pass the actual file path to the video file, not the base64 string.

               //Where path is declare at the top of the method ... var path = Directory.GetCurrentDirectory() + @"\wwwroot\images\" + file;

               //videoCapture.Open(path, VideoCaptureAPIs.ANY);       

          
          

               int totalFrames = (int)videoCapture.Get(VideoCaptureProperties.FrameCount); // Get the total number of frames in the video

               int framesToExtract = 10;                                                   // Number of frames to extract evenly spaced from the video

               videoFramesBase64.Clear();

          
          

               if (totalFrames > 0 && framesToExtract > 0)

               {

                   for (int i = 1; i < framesToExtract; i++)

                   {

                       int frameIndex = (int)((i * (totalFrames - 1)) / (framesToExtract - 1));                // Calculate the frame index to extract

                       videoCapture.Set(VideoCaptureProperties.PosFrames, frameIndex);                         // Set the frame position to extract

                       using var newframe = new Mat();

          
          

                       if (videoCapture.Read(newframe) && !newframe.Empty())                                   // Read the frame

                       {

                           var frameBytes = newframe.ToBytes(".jpg");                                          // Convert frame to byte array

                           var frameBase64 = "data:image/jpeg;base64," + Convert.ToBase64String(frameBytes);   // Convert to base64

                           videoFramesBase64.Add(frameBase64);                                                 // Add to list 

                       }

                   }

               }

               frameStatus = "<b>Captured:</b> " + videoFramesBase64.Count.ToString() + " evenly spaced frames from video...<b>Click on any of these images for simple AI Analysis</b> <br/>You can also choose the <b>Analyze First Frame </b> button to analyze the larger image (first frame) below";

               frameStatus += "<hr/>";

          
          

               //Checking frame images for Similarity using OpenCvSharp

               //Compare the first and last images in the list videoFramesBase64 using OpenCvSharp

               //This will give us a simple similarity score based on the absolute difference between the two images

               //This is a simple metric; you might want to use a more sophisticated method for real applications

               //For example, you could use structural similarity index (SSIM) or other metrics depending on your needs

          
          

               if (videoFramesBase64.Count >= 2)

               {

                   // Decode base64 to byte arrays

                   

                   //... for instance , videoFramesBase64[0] could be a string like:

                   //data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD... (Base64 data)

                   //.Split(',')[1] Splits the string at the comma.

                   //[0] is "data:image/jpeg;base64" (the header)

                   //[1] is the actual Base64-encoded image data

          
          

                   var firstImageBytes = Convert.FromBase64String(videoFramesBase64[0].Split(',')[1]);

                   var lastImageBytes = Convert.FromBase64String(videoFramesBase64[^1].Split(',')[1]); // Using ^1 to get the last item in the list

          
          

                   // Load as Mat objects

                   using var firstMat = Cv2.ImDecode(firstImageBytes, ImreadModes.Color);

                   using var lastMat = Cv2.ImDecode(lastImageBytes, ImreadModes.Color);

          
          

                   // Resize to same size if needed

                   if (firstMat.Size() != lastMat.Size())

                   {

                       Cv2.Resize(lastMat, lastMat, firstMat.Size());

                   }

          
          

                   //Method 1 - using a simple absolute difference metric

                   //---------------------------------------------------------------------------------------

          
          

                   // Calculate absolute difference between the first and last frames

                   // This will give us a Mat object that contains the absolute difference between the two images

                   using var diff = new Mat();

                   Cv2.Absdiff(firstMat, lastMat, diff);

          
          

                   // Calculate a simple similarity metric (mean difference)

                   // This will give us a Scalar object that contains the mean difference for each channel (B, G, R)

                   var meanDiff = Cv2.Mean(diff);

          
          

                   // The meanDiff is a Scalar, so we access its values using Val0, Val1, Val2

                   // The Val0, Val1, Val2 properties correspond to the B, G, R channels respectively

                   // and we calculate the average of these values to get a similarity score 

                   // A low number indicates high similarity, while a high number indicates low similarity

                   // Note: This is a simple metric; you might want to use a more sophisticated method for real applications

                   // For example, you could use structural similarity index (SSIM) or other metrics depending on your needs

          
          

                   double similarityScore = (meanDiff.Val0 + meanDiff.Val1 + meanDiff.Val2) / 3.0;

          
          

                   // Optionally, display or log the similarity score ... attach to current contents of frameStatus

                   frameStatus += $"<b> ... Checking Images for Similarity : Similarity score (first vs last frame):</b> {similarityScore:F2}";

                   frameStatus += "(low number high similarity, a high number low similarity)"; // Add explanation

          
          

                   // Method 2 - Using Structural Similarity Index (SSIM)

                   //--------------------------------------------------------------------------

                   double ssimScore = CalculateSSIM(firstMat, lastMat);

                   frameStatus += $"<br/><b> ... Structural Similarity Index (SSIM) similarity score (first vs last frame):</b> {ssimScore:F4}";

                   frameStatus += "(1.0 = identical, closer to 0 = less similar)";

               }

               StateHasChanged();

          
          

          }

        • // Structural Similarity Index (SSIM), which compares images based on luminance, contrast, and structure,

          // providing a value between -1 and 1 (where 1 means identical images).

          // Helper method to compute SSIM between two Mat images

          private double CalculateSSIM(Mat img1, Mat img2)

        • //This method is called when the user clicks on the "Analyze First Frame from Captured Video or Image from Photo Taken or Selected Image from wwwroot" button

          //It is used to analzye a selected Image (from the list) OR from the Take Picture and Save (button) Or the Default first frame of a selected Video

          //all these sources store the image in the variable imageBase64 so we can use it here

          private async Task RecognizeImage()   

        • //This method is called when the user clicks one of the video frames displayed on the screen (form approx 9-10 frames)

          private async Task RecognizeFrameImage(string img)

        • //This method is called when the user clicks the "Analyze All Captured Frames of Video" button which is located to the right of the  9/10 frames of the video

          //Note that this method is similar to the RecognizeFrameImage method but it processes a List of images (frames) instead of a single image

          //The list of images (frames) is passed to this method as a parameter ... List<string> frameImgs

          //RecognizeFrameImages(videoFramesBase64))

          private async Task RecognizeFrameImages(List<string> frameImgs)

          {         

              status = "";

          
          

              if (frameImgs == null || frameImgs.Count==0)

              {

                  responseMessage = "Please select an video first to break apart into frames.";

                  status = "Error1";

                  return;

              }

          
          

              isFrame = true;

              status = "Processing";

              responseMessage = null;

          
          

              //Remember here we are sending a LIST of images (frames) to the AI service not a single image

              var message = new ChatMessage(ChatRole.User, "Look through all the frame images and summarize the contents, display the results in nicely formatted HTML");

          
          

              // Add all images in frameImgs to the message contents

              foreach (var img in frameImgs)

              {

                  message.Contents.Add(new DataContent(img, "image/jpg"));

              }

              //message.Contents.Add(new DataContent(img, "image/jpg"));       

          
          

              var response1 = await ChatClient.GetResponseAsync(message);

              responseMessage= response1.Text; //display the response message

          
          

              status = "Results"; //update the status to show results       

          
          

          }   

        • private async Task StartCamera()

          {

              if (isCameraActive)         //this is a boolean flag to track if the camera is already active it is set to true when the camera is started below

              {

                  return;

              }

              Clear();

          
          

              isCameraActive = true;

              videoCapture = new VideoCapture(0); // Open the default camera

          
          

              if (!videoCapture.IsOpened())       // Check if the camera is opened successfully

              {

                  status = "Unable to access the camera";

                  isCameraActive = false;

                  return;

              }

          
          

              //This loop will run until the camera is stopped

              //it will read frames from the camera and convert them to base64 strings for display on the screen

              while (isCameraActive)

              {

                  //The Mat type is from the OpenCvSharp library, which is a .NET wrapper for the popular OpenCV computer vision library.

                  using var frame = new Mat();

                  videoCapture.Read(frame);   // Read a frame from the camera

          
          

                  if (!frame.Empty())

                  {

                      currentFrame = frame.Clone();                                                           // Clone the frame to avoid memory issues

                      var frameBytes = currentFrame.ToBytes(".jpg");                                          // Convert the frame to a byte array

                      currentFrameBase64 = "data:image/jpeg;base64," + Convert.ToBase64String(frameBytes);    // Convert the byte array to a Base64 string to display it in the browser

                      StateHasChanged();                                                                      // Update the UI with the new frame

                  }

          
          

                  await Task.Delay(33); // ~30 FPS

              }

          }

        • private void StopCamera()

        • //This method is called when the user clicks the "Take Picture" button

          //It captures the current frame from the camera and saves it to a file in the wwwroot/images directory

          private async Task TakePicture()

          {       

          
          

              //Make sure we have a valid filename for the image file

              if (string.IsNullOrEmpty(VideoFileName))

              {

                  status = "Please enter Image File Name to Save";

                  await App.Current.Windows[0].Page.DisplayAlert("Warning", "Please Enter a File Name for the Image", "OK");

                  return;

              }

          
          

              if (currentFrame == null)

              {

                  return;

              }

          
          

              var frameBytes = currentFrame.ToBytes(".jpg");                                       // Capture the current frame and convert it to a base64 string

              capturedImageBase64 = "data:image/jpeg;base64," + Convert.ToBase64String(frameBytes);// Convert the byte array to a Base64 string

              imageBase64 = capturedImageBase64;                                                   //copy current frame to imageBase64 variable for display and possible Img Analysis

          
          

              imageFileSize = frameBytes.Length;

          
          

              // Save the captured image to a file in the wwwroot/images directory (debug folder) with a unique name  and a timestamp and the prefix "captured_"
          //As opposed to the Sound.razor page where we saved to the AppData folder

          //Path.Combine(FileSystem.AppDataDirectory, "audio");
          

              var fileName = "captured_" + VideoFileName + "-" + DateTime.Now.ToString("yyyyMMdd_HHmmss") + ".jpg";         

              var filePath = Path.Combine(Directory.GetCurrentDirectory(), "wwwroot", "images", fileName);

              await File.WriteAllBytesAsync(filePath, frameBytes);    // Save the image to the file

          
          

              await InitializeMediaAsync(); // Reinitialize media files to include the newly captured image

              StateHasChanged();

          }

        • //This method is called when the user clicks the "Record Video" button

          //It captures video from the camera and saves it to a file (with a user provided filename) in the wwwroot/videos directory

          private async Task RecordVideo()

          {

              //We go through a number of error checks first before we start recording

          
          

              //this is a boolean flag to track if the camera is already recording it is set to true when the camera is started below

              if (isRecording)           

                  return;

          
          

              //Make sure we have a valid filename for the video file

              if (string.IsNullOrEmpty(VideoFileName))

              {

                  status = "Please enter Video File Name to Save";

                  return;

              }

              imageBase64 = null; //clear the image base64 variable to avoid confusion with the video recording

              StateHasChanged();

          
          

              status = "";

          
          

              //1.Check if the camera is already open:

              //videoCapture == null checks if the videoCapture object hasn't been created yet.

              //!videoCapture.IsOpened() checks if the camera isn't currently open.

             

              //2.Open the default camera:

              //videoCapture = new VideoCapture(0); tries to open the default camera (usually the first webcam).

             

              //3.Handle failure to open:

              //If the camera still isn't open after trying, it sets a status message:

              //status = "Unable to access the camera"; and exits the method with return;.

             
          if (videoCapture == null || !videoCapture.IsOpened())

              {

                  videoCapture = new VideoCapture(0);

                  if (!videoCapture.IsOpened())

                  {

                      status = "Unable to access the camera";

                      return;

                  }

              }

          
          

              isRecording = true;

          
          

              //You use VideoWriter to save a sequence of images (frames) as a video file (e.g., .mp4).

              //You must specify the output file path, codec(like H.264), frame rate, and frame size when creating it.

              //Each time you call .Write(frame), it adds that frame to the video.

              //When done, you call.Release() or.Dispose() to finish and close the file.

             

              // H.264 offers better performance and fewer crashes compared to some older codecs .

              // The resulting .mp4 files are widely compatible.

             

              //var fourcc = VideoWriter.FourCC('M', 'P', '4', 'V'); // MP4 codec

              var fourcc = VideoWriter.FourCC('X', '2', '6', '4'); // H.264 codec ... better performance no crashes       

             

              int frameWidth = (int)videoCapture.Get(VideoCaptureProperties.FrameWidth);

              int frameHeight = (int)videoCapture.Get(VideoCaptureProperties.FrameHeight);

              double fps = videoCapture.Get(VideoCaptureProperties.Fps);

              if (fps <= 0) fps = 20; // fallback if FPS not detected

          
          

              // Generate a unique name for the recorded video file including the date and the file name and the prefix "recorded_"

              recordedVideoFileName = "recorded_" + VideoFileName + "-" + DateTime.Now.ToString("yyyyMMdd") + ".mp4";

              var filePath = Path.Combine(Directory.GetCurrentDirectory(), "wwwroot", "videos", recordedVideoFileName);

          
          

              videoWriter = new VideoWriter(filePath, fourcc, fps, new OpenCvSharp.Size(frameWidth, frameHeight));

          
          

              // Check if the video writer is opened successfully ... VideoWriter is part of OpenCvSharp library       

              if (!videoWriter.IsOpened())

              {

                  status = "Unable to create video file";

                  isRecording = false;

                  return;

              }

          
          

              status = "Recording started...";

              StateHasChanged();

          
          

          
          

              // Capture frames from the camera and write them to the video file

              // This loop will run until the camera is stopped or the recording is stopped

              // The Task.Run is used to run the video capture in a separate thread to avoid blocking the UI thread

             

              await Task.Run(async () =>

              {

                  using var frame = new Mat();

                  while (isRecording && videoCapture.IsOpened())  //if we press the Stop Recording button, isRecording will be set to false

                  {

                      videoCapture.Read(frame);

                      if (!frame.Empty())

                      {

                          videoWriter.Write(frame);               // Write the frame to the video file

                      }

                      await Task.Delay(33); // ~30 FPS

                  }

              });

          
          

              // Stop the recording and release the video writer

              videoWriter.Release();

              videoWriter.Dispose();

              videoWriter = null;

          
          

              // Add to video list for UI

              await InvokeAsync(() =>

               {

                  if (!string.IsNullOrEmpty(recordedVideoFileName))

                      videoFileList.Add(recordedVideoFileName);     // Add the recorded video file to the list of video files

                  status = "Recording stopped. Video saved.";

          
          

                  InitializeMediaAsync(); // Reinitialize media files to include the newly recorded video

                  StateHasChanged();

              });

          }

        • private void StopRecording()

        • private void Clear()

        • //class used in the RecognizeImage method to hold the result of the possible form image that was recognized by the AI.

          public class FormResult

          {

              public string? ReportBy { get; set; }

              public string? ReportDate { get; set; }

              public string? ReportPhone { get; set; }

              public string? AttendantName { get; set; }

              public string? AttendantPhone { get; set; }

              public string? ReceivedDate { get; set; }

              public string? EnteredDate { get; set; }

          }

      • Key HTML (See HTML snippet)