Hey there! I love talking about the principles of application lifecycle management (ALM) because it’s a crucial aspect of developing successful applications. I know it might sound boring, but trust me, it’s not! Understanding and implementing ALM from the beginning can simplify the development, usage, and management of your applications. This is especially important for Generative AI applications. I discovered that Azure AI Studio can be a great tool to get started with ALM for your Generative AI solutions.

The classic ALM process includes definition, development, test, and production. It’s not a surprise, all these steps are covered by Azure AI Studio. Firstly, the AI Studio provides support during the definition and development of your large language model (LLM) applications.

This means I can start directly with ideation and definition of my AI solution here in the AI studio.

Defining my Use Case

When I ‘m defining a Generative AI application, it’s like starting any other software project. I start by defining my use case, which means I’m describing the problem I want to solve, the process I want to optimize or automate, and the value I hope to generate. It’s like setting a goal for myself and figuring out how to achieve it using the power of AI!

Remember, my blog post Getting started with Azure AI Studio where I showed you, how I created a new project in AI Studio. AI Studio helped me from the beginning to set up AI projects in the right way.

Microsoft says: first I should define my objectives for my AI application. Afterwards, it’s a good practice to prepare sampling data and create a basic prompt for the AI. Finally, I have to design my solution.

In practice, defining the objectives helps me and my team to keep my project focused and on track. Preparing a sample dataset allows me right from the beginning to test and measure my AI. In addition, I can grab a basic prompt from the Prompt Catalog and experiment in the playground before I finally draft my AI solution.

Objective

Let’s start by defining the objective for my AI project! I’ll give you a well know example: “As a business owner I want to categorize incoming support requests.” My objective is to find out how many requests are coming in and what types of requests they are. In detail, I want to categorize these requests into technical support, billing inquiries, or product feedback.

Based on this, my objective is: I want to implement a system that can accurately categorize incoming support requests for a business into technical support, billing inquiries, or product feedback. The system should be able to track and report the number of requests received for each category, enabling the business owner to better understand the types of issues customers are facing and allocate resources accordingly.

Practically, I create a new project ai-request-categorizer in Azure AI Studio:

…and store my objective as project description:

Done!

Sampling Dataset

Next, I need data for my Sampling Dataset. In detail, I collect examples for each of my request categories technical support, billing inquiries, or product feedback. For this I can use real world examples, or I use Copilot to generate these examples for me.

Here are some examples:

Technical Support Request:

Hi Team,

I’m having trouble accessing my account on the website. Every time I try to log in, I get an error message saying my credentials are incorrect, even though I’m sure I’m entering the correct username and password. Can you please assist me with resolving this issue?

Thanks,
Customer

Billing Inquiry

Hi Team,

I noticed a charge on my account for $50 that I don’t recognize. Can you please provide more information about this charge and help me understand why it was made?

Thanks,
Customer

Product Feedback

Hi Team,

I’ve been using your product for a while now, and I have some suggestions for improving the user interface. I think adding a search feature would make it easier to find specific items in the application. I’d love to discuss this further with your team.

Thanks,
Customer

Based on this, I generate my sampling dataset in JSON line format. This means, I create a file with a JSON in each line that represents one request to categorize. Furthermore, I store the email content as text and the category as label in my JSON:

{"text": "Hi Team,\n\nI'm experiencing an issue with the product where it keeps crashing whenever I try to open a specific file. I've tried restarting the application and my computer, but the problem persists. Can you please help me troubleshoot this issue?\n\nThanks,\nCustomer", "label": "technical_support"}
{"text": "Hi Team,\n\nI'm having trouble accessing my account on the website. Every time I try to log in, I get an error message saying my credentials are incorrect, even though I'm sure I'm entering the correct username and password. Can you please assist me with resolving this issue?\n\nThanks,\nCustomer", "label": "technical_support"}
{"text": "Hi Team,\n\nI noticed a charge on my account for $50 that I don't recognize. Can you please provide more information about this charge and help me understand why it was made?\n\nThanks,\nCustomer", "label": "billing_inquiry"}
{"text": "Hi Team,\n\nI recently upgraded my subscription plan, but I was charged twice for the same upgrade. Can you please look into this issue and refund the extra charge?\n\nThanks,\nCustomer", "label": "billing_inquiry"}
{"text": "Hi Team,\n\nI've been using your product for a while now, and I have some suggestions for improving the user interface. I think adding a search feature would make it easier to find specific items in the application. I'd love to discuss this further with your team.\n\nThanks,\nCustomer", "label": "product_feedback"}
{"text": "Hi Team,\n\nI wanted to share some feedback on the latest update to your product. I really like the new features that were added, but I noticed a bug in the search functionality that needs to be fixed. Can you please look into this issue?\n\nThanks,\nCustomer", "label": "product_feedback"}

Now, I start uploading this data to Azure AI Studio:

First, I’m selecting my local file request-examples.jsonl:

Afterwards, I give my data the name request-examples and start the upload:

That’s it for now. Later, I will use this data set to evaluate and to measure my AI solution during the development and in my test phase.

Basic Prompt

The basic prompt for my AI application defines my purpose, which is to categorize my request messages into technical support, billing inquiries, or product feedback. Furthermore, the basic prompt also clarifies the role of the LLM and details of the expected outcome.

I start with this prompt:

<role>
You are a support agent for a software company. You receive the requests from customers. Categorize the requests into `technical_support`, `billing_inquiries`, or `product_feedback`.
</role>

Categorize the following requests into `technical_support`, `billing_inquiries`, or `product_feedback`:

# Request:
'''
{{text}}
'''

# Category:

You see from my example; I use a mix of HTML-Tags and Markdown to structure this prompt. That helps my LLM to understand better the given text. In addition, I added here {{text}} as placeholder for the request message.

Now, I’m navigating to my Playground in Azure AI Studio. First, I must select the right Mode to solve my problem:

I use as mode Completions. This is because my LLM is only supposed to fill in the prompt and infer the category. Now, I copy my created basic prompt into the Playground and replace my variable {{text}} with one of my request messages. Next, I’m clicking on Generate and see this result:

Great, the LLM completed with product_feedback.

Solution Design

Now it’s time to finish my Use Case Definition phase with my solution design. As mentioned, I receive emails that should be categorized into technical support, billing inquiries, or product feedback.

This means, simplified, my AI solution gets the received email as input content and provides as output the category:

To fully cover my objective, I need something more. Right now, my AI solution covers only the first part of my objective: I want to implement a system that can accurately categorize incoming support requests for a business into technical support, billing inquiries, or product feedback. In detail, I have designed a solution that only provides label information for my requests. In other words, just the AI service (endpoint).

Right, my solution doesn’t cover the second part, which is: The system should be able to track and report the number of requests received for each category, enabling the business owner to better understand the types of issues customers are facing and allocate resources accordingly.

To complete the picture, I draft a very simple end-to-end solution with Power Automate. First, I automate monitoring my mailbox. That means, my flow is triggered from each incoming email. Afterwards, I extract the content of my message in my flow and use my future AI endpoint to categorize my email. This AI endpoint will be there as soon my solution passes the test phase in my Azure AI Studio ALM process. Finally, I store the categorized content in a Dataverse table within my flow.

Here is the full picture:

As you also see from the solution overview, I draft here a Power BI Dashboard to provide a detailed overview about the categorized requests to my business owner. Moreover, Copilot in Power BI can help to gain insight into these requests.

Summary

Getting started with ALM for AI solutions is not so difficult, because Azure AI Studio supports us. Today I gave you an overview of the ALM process for AI solutions. In detail, I explained the 4 phases of the ALM process which are defining, development, testing, and production. With this introduction I started into the first phase.

First, I defined my objective. Here I used Azure AI Studio and created a new project. In addition, I added my objective as project description. This helps me and my team to clarify the purpose of my AI solution. Next, I prepared sampling data based on my business case. Furthermore, I uploaded this sampling data into Azure AI Studio. Later, this sampling data will help me to develop my solution, because I can evaluate and measure my AI generated outcome. Finally, I drafted a solution blueprint for my business case. In other words, I defined the target picture for a productive use of my AI solution.

Now I’m ready for the next phase, the development phase of my AI solution. Wait for my next blog post and learn, how Azure AI Studio also supports this part of the ALM process…