Exploring AI Assistants
Hypothesis: AI Assistants can be configured to represent business personas, reflecting their knowledge bases, system interactions and ways of working.
To start probing this hypothesis, we need to answer the following question:
What is an AI Assistant, and how is it distinct from a standard GPT or a fine-tuned model?
To understand this, let’s go through some of the high-level building blocks of how we interact with an AI.
First off, you’ve got the model. Your Claude Opus, GPT 4o, Grok, Llama and others. That underlies everything and is where the real depth of training and intelligence sits, which really drives the effectiveness of any elements that sit on top of it. For instance, an unintelligent model may need for more specific instructions to perform effectively.
Models have specific characteristics including their speed, context windows, and performance across industry benchmarks:
Speed: how quickly the model can compute/generate a response.
Context window: the working memory of the AI - the amount of information that the model can retain at a given point in time. Newer models support over 128,000 tokens, augmented by technologies such as Retrieval Augmented Generation (RAG), which enables the model to dynamically search for content rather than being limited by their context window. This is what allows models to trawl through large file attachments when compared to the shorter character limits of chat.
Industry benchmarks: a standard set of generally accepted tests across multiple domains (including language, code and mathematics) which models are compared through. A key concept here is zero-shot vs. one-shot and higher - this is in reference to how many times the model has been exposed to the same or similar questions before, and therefore an important factor in determining if the model is genuinely solving the problem or relying on memory. It is worth noting that, given the models are not public domain, we have no way of truly knowing that they are genuinely zero-shot tests, or whether the models have been configured to perform well in the tests.
What we can do directly to the model is fine-tune. Fine-tuning really is the perfect term, as it is very similar to how we fine-tune equipment such as knives. We can sharpen it and make it more fit for a task, but it is a process that needs to be repeated periodically and with precise inputs. If done too much, it can actually reduce the overall capability of the model - an issue discussed by Andrej Karpathy. There are some great real-world applications of fine-tuning, such as where Open AI worked with Korean Teleco SK Telecom, fine-tuning their GPT-4 model and subsequently gaining a customer satisfaction score improvement from 3.6/5 (GPT-4) to 4.5/5 (GPT4 fine-tuned). We’re going to get further into fine-tuning in a future post.
From this point on, nothing is actually directly altering the model, so whilst we may use terms like train, knowledge base and persona, these aspects are more like instructions to a general model, rather than adjustments or aspects of the model itself. An analogy would be that a model is a character, whereas any instructions it receives make it an actor playing a character. What it fundamentally is: the data it was trained on, how it operates etc, fundamentally is the same.
When we message a GPT, there are really three key components it is considering:
The message: what we are saying, as well as any files we may have attached.
System instructions/role: in what capacity we want the model to act in responding to the message. The default instructions are typically “you are a helpful assistant”.
Previous context: this may include previous messages in the thread, or ‘AI memory’ carried between threads (a feature on Chat GPT).
In its computations, the model can then respond, call functions, or even delegate to specific sub-agents.
When we are configuring AI Assistants, we are predefining what goes into (2) - the system instructions for each message, as well as the functions that the model can call when responding to us.
So, what does this all mean for Assistants?
Two things:
Every time we use an assistant, we are providing the entire instruction set to the AI model. This is important from a commercial perspective. Consider an exaggerated scenario: if a model costs 1 cent to compute 1000 tokens, and I have a 30,000 token assistant, I’m paying 30 cents per response for that assistant.
The idea of a true knowledge base and behaviours comes into question. We’re effectively layering on these instructions, with a specific weight of importance mind you, to a pre-trained model. For instance, I may configure an assistant to speak in professional Australian language, but I’m pretty sure I’m still going to see the occasional ‘z’ in its responses, just due to the sheer aggregate of US data the model was trained on. Similarly, I may tell a model the specifics of the KFC recipe, but it may blend that with a generalised knowledge of how to cook chicken.
We don’t need to give up on accuracy though. Remember, with incredible functions like Retrieval Augmented Generation, the model can still pull together information directly from content provided.
Based on this, it’s reasonable to apply the following practices when implementing assistants:
Be concise: put precisely what you want from the assistant into the configuration. It’s not about knowing everything but knowing what’s important. For deep and extensive knowledge functions, files can be provided to the assistant, or functions for searching.
Promote accuracy, but expect some hiccups: whilst consistency and accuracy should be key to the instructions provided to assistants, we are not fundamentally changing the models. As such, they are still going to be subject to the aggregate of data they were initially trained on. As the intelligence of models improve, we can expect that their ability to follow instructions also improves. Even so, we can only use instructions to set a direction, not to directly control responses.
Where higher precision is required, consider a fine-tuned model: if applying AI to a highly sensitive area such as law, you may want to at the very least consider fine-tuning, which directly effects the model itself. Like with hiring highly trained specialists, fine tuning has its own cost and maintenance implications, but is necessary in some areas.
Let’s do an experiment.
Let’s compare a standard, uninformed GPT-4o model against GPT 4o with a configured Assistant layered on top of it.
Scenario
An Australian D365 administrator is inquiring about how they can configure a view.
Assistant configuration
You are a Dynamics 365 CRM implementation consultant in Australia, who communicates directly with customers in a concise and professional manner. You use Australian English.
Files attached:
Plain and inclusive language | Digital.NSW
Create and customize views | Microsoft Learn
Prompt
How can I update this view so that it only shows Accounts in Redmond and has a column between account name and main phone for ‘account number’?
GPT 4o Response
The standard GPT-4o response is pointing in the right direction, though it seems to be pulling some information from older versions of D365 CRM and giving the administrator a way to configure a ‘personal’ view that they can share, though some of the navigation steps are wrong. Having said that, if an administrator followed these instructions, they would see where they need to go. I’ll give it a 5/10.
To update your view in Dynamics 365 to show only accounts in Redmond and add an 'Account Number' column between 'Account Name' and 'Main Phone', follow these steps:
Adding a Filter for Redmond:
1. Navigate to the View Editor:
- Click on the three dots `...` or `More Commands` in the command bar.
- Select `Advanced Find`.
2. Modify Criteria:
- In the Advanced Find window, set the `Look For` dropdown to `Accounts`.
- In the `Use Saved View` dropdown, select the current view you are using (e.g., Active Accounts).
- Click on `Edit Columns`.
- In the `Criteria` section, add a new condition: `Address 1: City` equals `Redmond`.
- Apply the filter and click `Results` to ensure it's working.
Modifying Columns:
1. Edit Columns:
- In the Advanced Find window, click on `Edit Columns`.
2. Add 'Account Number' Column:
- Click on `Add Columns`.
- Choose `Account Number` from the list of available columns.
3. Rearrange Columns:
- Select the `Account Number` column in the list.
- Use the arrows to move it between `Account Name` and `Main Phone`.
4. Save Changes:
- Click `OK` to close the column configuration window.
- Click `Save As` to save the modified view with a new name, so you can reuse it easily.
5. View Results:
- Click `Results` to see the updated view and ensure it meets your criteria.
After following these steps, you should have a view that only displays accounts located in Redmond and includes an `Account Number` column between `Account Name` and `Main Phone`.
GPT 4o Assistant Response
The assistant response is much better. It is completely aligned with the knowledge base with the specific steps needed to configure an update to a view, per the prompt. There is some room for improvement - a real consultant may have asked the customer whether they wanted this to be a new view (e.g. ‘Redmond Accounts’) and if it was just for the customer or intended to be shared across the organisation. Furthermore, we can see that US language has been used throughout, despite the direction to use Australian English. I’m going to give this an 8/10, because it’s still a substantial result and ultimately solved the stated prompt well.
To update the view in Dynamics 365 CRM to meet your requirements, follow these steps:
Step 1: Filter the View to Show Only Accounts in Redmond
1. Navigate to the View Editor:
2. In your Dynamics 365 CRM, go to Settings > Customizations > Customize the System.
3. Expand Entities > Account.
4. Select Views > Active Accounts (or the view you want to modify).
5. Add the Filter for Redmond:
6. Open the view to edit it.
7. In the view editor, find the Filter Criteria section.
8. Add a new condition:
Field: Address 1: City
Operator: Equals
Value: Redmond
9. Save and Publish:
Save the changes to the view.
Click Publish to apply the changes.
Step 2: Add the 'Account Number' Column
1. Modify the Columns in the View:
In the same view editor, under the Columns section, select Add Columns.
Choose the field Account Number from the list and add it to your view.
2. Reorder the Columns:
Use the arrow buttons to move the Account Number column to the desired position, between Account Name and Main Phone.
3. Save and Publish Again:
After reordering the columns, save the view.
Click Publish to apply these changes as well.
Step 3: Verify the Changes
Go back to the Sales > Accounts section to check if the view now displays only accounts in Redmond and includes the Account Number column in the correct position.
By following these steps, you will have a view in Dynamics 365 CRM that shows only accounts located in Redmond and includes an Account Number column between the Account Name and Main Phone columns.
Please let me know if you need any troubleshooting or further customization assistance.
Comparison results
This is effectively 0-shot vs. many-shot - I’ve broken the rules of the game, because I know the kinds of content the AI is going to get. But you know what? That’s allowed, and in this case, quite helpful. If we know the areas where we want to direct AI, it’s worth setting those guidelines to improve its relevance and use. There is still progress to be made - for instance with providing meaningful challenge to the prompter, or in complying to specific nuances and expectations in communication for a specific audience/industry. But we are seeing good, practical outcomes, and that is a major step forward.
To conclude - can an Assistant be configured to represent business personas, reflecting their knowledge bases, system interactions and ways of working? It depends Absolutely.
This is certainly a bold statement coming off an 8/10, highly specific scenario, and there’s more we will dig into here on future posts. However, what we’ve demonstrated here is that we are able to emulate specific business personas, now it’s just a question of extent.
There’s more to explore here to start getting a fuller picture, including:
Fine-tuning: how does this really fit in the picture with assistants? How do their capabilities stack? Where should we apply it? Where not?
Chain-of-thought: how can we break content up and sequence it to get the best quality outputs from AI?
For now, signing off.
